Two years into the AI race, Google is still struggling to catch up. After ChatGPT’s 2022 release sent shockwaves through the tech world, Google declared a “code red” and rushed to launch Bard, its answer to OpenAI’s chatbot. But Bard’s debut was rocky—errors in a promotional demo wiped billions off Google’s market value. Since then, Google has rebranded Bard to Gemini, but issues continue to plague the product, even in its paid version, Gemini Advanced.
Despite claims of superior technology, Gemini Advanced is far from the polished AI assistant Google promised. From factual mistakes in product comparisons to unexpected errors in basic tasks like recipe generation, the chatbot has repeatedly failed under scrutiny. At $20 per month—the same price as ChatGPT Plus—users expect reliability and precision. But Gemini Advanced often delivers neither.
A Troubled Launch and Rebrand
Google’s initial foray into AI chatbots began with Bard, which was unveiled under pressure in early 2023. According to reports, the company was blindsided by OpenAI’s momentum. Bard’s rushed launch led to factual blunders that impacted investor confidence. Rebranded as Gemini later that year, Google hoped to pivot the narrative toward innovation.
However, the rebrand did not fix core problems. Users continued to report embarrassing failures in Gemini’s outputs. This included incidents where its image generator produced racially inappropriate depictions and its AI Overviews in Search recommended unsafe advice, like eating rocks or adding glue to pizza. While most of these issues were confined to free tools, they cast a shadow on the reliability of the entire Gemini ecosystem.
High Expectations for Gemini Advanced
Google markets Gemini AI Advanced as a premium service powered by the Gemini 1.5 Pro model. It features a massive 2 million-token context window, allowing users to input up to 3,000 pages of text. This technical feat places it ahead of ChatGPT Plus, which maxes out at about 4,000 tokens.
Yet in real-world tests, this advantage doesn’t always translate to meaningful improvements. Gemini Advanced includes a Double Check feature that lets users cross-reference outputs with Google Search. But this feature is only useful if users already suspect an error. As CNET’s testing revealed, most users don’t double-check information they assume is correct. This highlights a fundamental usability flaw.
Poor Performance on Shopping Queries
One of the biggest tests for AI chatbots is delivering accurate and timely information, especially when users seek guidance on new tech products. During CNET’s evaluation, Gemini Advanced faltered significantly in comparing two flagship phones: the Motorola Razr Plus 2024 and the Samsung Galaxy Z Flip 6.
Gemini incorrectly stated that the Razr Plus had a smaller display than the Z Flip 6. It also claimed the Z Flip 6 was bulkier when, in fact, it’s slimmer. Even the chipset specifications were misrepresented. These are not minor inaccuracies—they affect consumer decisions.
When ChatGPT Plus was asked the same questions, it also made mistakes. This shows that parsing up-to-date product data remains a challenge for large language models. But Gemini’s Double Check feature could have helped if users had known they were being misled in the first place.
Recipe Generation Glitches
AI chatbots often perform well in generating recipes, a relatively simple creative task. Free models like ChatGPT 3.5, Claude, and base-level Gemini can provide decent culinary suggestions.
Yet Gemini Advanced unexpectedly failed when asked to generate a basic recipe. It started preparing a response, only to erase it and reply, “I’m not programmed to assist with that.” When the user tried again, Gemini repeated the behavior before finally generating a usable response on the third attempt.
In one case, a Thai-inspired tres leches cake was requested. Gemini initially refused to complete the recipe, but later returned a basic version that was missing key Thai ingredients. The result felt underwhelming, particularly from a paid product. According to testers, dedicated recipe websites offer a better, more consistent experience.
Inconsistent Article Summarization
Summarizing content is a core use case for AI tools. Whether it’s distilling long articles, emails, or reports, users expect reliable summaries from advanced chatbots. This is especially true for paid services like Gemini Advanced.
However, CNET’s evaluation showed inconsistent behavior. When directly asked to summarize a CNET article, Gemini claimed it couldn’t access the site. But when the link was pasted without any summary prompt, Gemini returned a list of bullet points. The output was accurate and captured the article’s essence.
This inconsistency in response depending on prompt phrasing raises questions about the system’s logic. According to CNET, Google isn’t blocked from crawling their website. However, the publication has opted not to allow its journalism to train Google’s large language models. Despite that, Gemini can still extract information—but only under unclear circumstances.
Travel Planning Is a Strength
Travel planning is one of Gemini Advanced’s stronger features. When asked to create a three-day itinerary for Columbus, Ohio, the chatbot performed well. It recommended locations, restaurants, and entertainment spots with updated information.
Thanks to Google Maps integration, Gemini can pinpoint nearby attractions. For example, when asked about retro video game stores, it produced a map highlighting five local options. It is also linked to external sites like Tripadvisor and includes images for visual context.
This functionality is present in both the free and paid versions of Gemini. While useful, it’s not exclusive enough to justify the $20 subscription fee. For most travel queries, the free Gemini still gets the job done.
Weak Performance in Philosophical Reasoning
To test higher-level reasoning, CNET presented both ChatGPT 4 and Gemini Advanced with a complex philosophical prompt. The task required synthesizing ideas from multiple schools of thought while tying them to modern technology.
ChatGPT 4 handled the challenge well, offering a thoughtful, structured analysis. In contrast, Gemini Advanced gave a superficial response and redirected the user to Google Search for further reading. This redirection undermines the purpose of using a chatbot.
Reviewers likened Gemini’s response to a student writing a book report after glancing only at the cover. This highlights a broader issue: despite its technical capacity, Gemini Advanced struggles with abstract and layered queries.
Gemini AI Advanced Feels Unfinished
Even with its massive token capacity and integration with Google services, Gemini Advanced doesn’t meet expectations. Its inconsistent performance, factual errors, and unexpected glitches make it unreliable for everyday use.
In its review, CNET testers underscored that even though Google has a history of AI expertise, the Gemini Advanced lacks the refinement that one might hope with the company that had contributed to the innovation of the neural networks architecture. The chatbot tends to resort to the advice to go to Google and search it, which undermines its main purpose.
The product has a rush atmosphere. Its flops remind of previous misses with Bard and raise the question of whether Google can ever compete in the market for AI assistants.
