ChatGPT vs Google

We Evaluated ChatGPT vs. Google on 500 Search Queries

ChatGPT crushes Google on coding queries and ties it on general informational queries – despite not being optimized for a search experience at all. OpenAI’s existential threat to Google, and reach out to [email protected] if you’re interested in exploring the data.

ChatGPT vs. Google

• Imagine you want to recursively delete all the Python files in a folder.
• Google misinterprets your query and returns a 404 error, prompting you to switch to chatGPT
• The sexy upstart is hyper-customized to your.py need, and provides the perfect tip you never thought to ask.

ChatGPT Pros

• Synthesizes information from a variety of sources into a single, coherent whole
• Minimal interface
• Ability to understand complex queries

Cons

• Hallucinations and inaccuracies
• The AI struggled with anything that wasn’t very common knowledge

Google’s Existential Threat

• Google’s 24 years of search expertise have been toppled by a new technology that soon even small startups will be able to use
• Newer search engines like Neeva, You.com, Kagi, and Bing already move faster, with the freedom to explore new products and UIs that Google can’t
• Imagine AI models just as smart as Google’s or smarter! – in their hungrier, more product-focused hands

Large Language Models and the Google-Killer

• ChatGPT is the definition of the future of Search.
• Twitter has been running amok with examples of how it crushes Google
• But is this representative?
• Large language models might be the Google killer.

Human evaluation

• 100 users were asked to pull up their search history and extract their 5 most recent “informational” queries
• They reissued the same query on Google and posed it to ChatGPT
• The results? Despite not being optimized for a search experience at all, chatGPT already matches or slightly beats Google’s performance
• Google loses out 70% of the time on coding-specific queries
• The Chatbot was better in terms of delivering a concise answer that anyone (myself included) could unambiguously digest.
• Google, full of empty filler and irrelevant ads, was not even close to as nice as the Chatbot.

Source