Don't Pay for Google Gemini Advanced
Google's "most capable AI model" is not as good as ChatGPT 4
Update: Turns out that I was mistaken. Gemini Advanced aka Gemini Ultra is as good as ChatGPT4. More details here.
Earlier this week, Google made their “most capable AI model”, Gemini Advanced (earlier known as Gemini Ultra), available to users as a paid service. It costs approximately ₹2000 per month (you get a free trial of 2 months, but you have to give credit card details for that).
Earlier, I had written about Gemini Pro, the version one level below Gemini Ultra, and found that it is good for summarizing YouTube videos and asking questions about the videos, but other than that, it wasn’t as good as ChatGPT4 (the paid version).
When Gemini Ultra became available, I immediately jumped at the opportunity to try it out. I continue to believe that for people who can afford it, paying the ₹2000 to get access to the best LLM is absolutely worth it. Unfortunately, Gemini Ultra does not fit that description. It is nowhere as good as ChatGPT4. If you remember, Google had released a very impressive video of a demo of Gemini Ultra’s capabilities, which later turned out to be manipulated and misleading. Now that we can actually try out Gemini Ultra ourselves, I can say that it is really not that good.
In other words, use your money to pay for ChatGPT4. Continue using the free Gemini Pro for video summarization and use ChatGPT for everything else. And those who can’t afford to pay for this, use Bing/Chat or Claude.
Update:
So, Ethan Mollick has a post saying Gemini Ultra is approximately as good as ChatGPT4. However, that hasn't been my experience. Ethan does a whole bunch of things with LLMs including playing games and convoluted tests of abilities. I just do simpler normal things.
For example, the fact that Gemini doesn't read PDFs is a serious shortcoming for me. For image reading and generation Gemini flaked out much more often ("Sorry I can't do this ...")
Here is an example. I tried to ask Gemini to explain this handwritten maths to me:
This complaint is typical of Gemini. I know for sure that it can read images. In fact, Ethan points out that this is one of its strengths vis a vis ChatGPT4. And yet, here is Gemini gaslighting me. Similar things happen randomly with YouTube video summarization.
Here is another example:
I asked it to find an article based on a vague description and instead of finding it for me, it again gaslighted me. By contrast, ChatGPT4 found the answer in one shot.
There are more example, but I’m getting bored of waiting for Google to produce something that is better than ChatGPT4.