I built a blind AI voting tool – ChatGPT isn’t winning (Claude Sonnet 4.5 leads at 56%)

Developing a Blind AI Evaluation Tool Reveals Task-Dependent Performance Trends

In the rapidly evolving landscape of artificial intelligence, selecting the most suitable AI model for specific tasks can be challenging. To address this, a developer recently created a bespoke testing platform aimed at objectively comparing different AI systems without bias. This initiative highlights the nuances of AI performance across diverse applications and emphasizes the importance of tailored model selection.

Introducing a Blind AI Comparison Platform

Recognizing the frustration of choosing an AI without clear, comparative insights, the developer designed a straightforward, anonymous response comparison tool. Users submit tasks—such as coding, writing, or reasoning—and then review two separate AI-generated responses without knowing which model produced which. The user selects the response they find superior, and over time, this crowdsourced voting accumulates into meaningful performance data.

This process is repeated across various categories, with individual preferences aggregated to offer a personalized ranking of AI models. After approximately 50 votes in different domains, the platform provides users with tailored recommendations based on their interactions.

Current Leaderboard and Performance Insights

The data collected so far reveals intriguing performance trends. The top-performing models in this ongoing comparison include:

• Claude Sonnet 4.5 with an overall success rate of 56.0%
• GPT-5.2 closely behind at 55.0%
• Claude Opus 4.5 at 54.9%

While these percentages indicate competitive performance, they also underscore that no single model is universally superior. Instead, different AI engines excel in different areas.

Task-Dependent AI Preferences

A notable insight from the results is the variability in model effectiveness depending on the task:

• GPT-5.2 shows superior prowess in coding-related tasks.
• Claude models tend to outperform in creative writing and more subjective, expressive tasks.

This suggests that the optimal AI choice isn’t a matter of overall ranking but hinges on the specific application.

Implications for AI Usage and Future Exploration

Rather than seeking a one-size-fits-all solution, users can leverage these insights to select models tailored to their needs. The developer emphasizes that this tool, created initially for personal use, is freely available to the community at llmatcher.com. The platform encourages users to experiment with different tasks and discover which AI aligns best with their requirements.

Concluding Thoughts

As AI models continue to evolve, ongoing, unbiased evaluations become essential for making informed choices.