OpenAI’s GPT-4o mini model is battling Gemini Flash and Claude Haiku

It’s been quite a summer for OpenAI. First, pushing the artificial intelligence (AI) capabilities’ envelope with the new GPT-4o model. Voice conversations, browsing the web to answer your queries, writing code, analysing data charts, text to image, troubleshooting with just a photo are all on the agenda. And that wasn’t even it. You can converse with the assistant via the phone’s camera, giving AI more of a context about the world around you. All this was even before the announcement at Apple WWDC, where we saw the first glimpses of ChatGPT finding a home within iOS, iPadOS, and macOS. OpenAI isn’t done for the summer.

The GPT-4o mini (its mini, not Mini) is keeping the size and weight in check, as is often the case with AI models and how they’re used, for broader relevance. Even then, OpenAI insists the cost-effective small model is better than even GPT-4 in many tasks, whilst being significantly less costly as well. Competitors too, on the MMLU and MMMU benchmarks. For instance, in the MMLU benchmark that tests multimodal reasoning, GPT-4o mini scores an 82% accuracy rate, while Gemini Flash (77.9%) and Claude Haiku (73.9%) are far behind.