Today, from powering conversational agents to assisting in complex decision-making processes, LLMs are revolutionizing industries and redefining the possibilities of AI. In my last article, I listed the rapid evolution of LLM in the last few years. In this blog, I analyzed 983 hot posts from Reddit: diving into the notable LLM names that frequently surface: GPT, Claude, Llama, Gemini and Mixtral, and learn the usage, sentiments and key concerts raised by developers and users when engaging with these models.

Frequent Mentions of the LLMs

Within the 983 hot posts, Llama is the most frequently mentioned LLM (124 posts), which indicates a significant interest or discussion around it in the Reddit community.

GPT is also highly mentioned, with 114 posts. It reflects its prominence in discussions related to AI and machine learning, likely due to its widespread use and familiarity.

Models like Claude, Gemini, BERT, T5 and Mixtral have fewer mentions (24, 18, 11, 9 AND 7 respectively), showing a very niche adoption.

‍

Sentiment of the LLMs

Most LLMs, including GPT, Claude, Gemini, Llama, Mixtral, BERT, and T5, show a majority of discussions that are positive, which means that the conversation around these models is generally favorable. Users and developers, to a certain level, are happy about them.

Claude has minimal negative sentiments, indicating that discussions around this model are less controversial. 

GPT shows a more balanced distribution of sentiments compared to other models, with a notable amount of neutral and negative sentiments alongside the positive. GPT, as the second most frequently mentioned model, while it is highly regarded, it also attracts critiques. 

Models like Llama, while predominantly positive, also have a higher proportion of negative sentiments compared to models like Claude and Gemini. The open-source nature of Llama may lead to both its high visibility and controversial discussions it generates. 

In the next section, I’m going to take a closer look at GPT and Llama, to explore what users and developers appreciate and what concerns they have about these models.

GPT VS Llama

As the top 2 popular models, I looked into these hot Reddit posts’ content, and tried to understand better about users’ love and concerns. 

GPT

GPT has attracted lots of attention for its advanced generative capabilities. Users appreciate GPT's ability to handle complex language tasks and its integration into diverse toolkits. For instance, one user expressed enthusiasm for developing an experimental toolkit that leverages GPT's capabilities, aiming to transform AI engineering into a more scientific endeavor with systematic experimentation.

In the meantime, GPT also has its drawbacks. Users frequently cite issues such as high operational costs and inconsistencies in performance, especially when deployed across different platforms. A user highlighted concerns about performance degradation when using GPT through various APIs compared to centralized platforms, indicating a need for more robust across-the-board performance.

Llama

The open-source aspect of Llama encourages users to integrate it into custom projects like software engineering toolkits. For example, one user shared their positive experience using Llama with the SWEKit, enhancing software engineering processes through improved agent capabilities.

However, users also report difficulties with Llama, such as issues with model reliability and documentation. Problems with "hallucinations" where the model generates nonsensical or irrelevant outputs, and sparse documentation hinder its effective application, particularly in critical or complex scenarios.

Summary

As the AI landscape has blossomed with a myriad of large language models, comparisons between these models have become increasingly varied and detailed. Drawing on my experience as a Product Manager, I've come to appreciate the profound importance of real user feedback. It's these authentic insights that truly illuminate the strengths and weaknesses of technology. That’s how the idea for this article came from.

It comes as no surprise that GPT and Llama are the most frequently mentioned models in our analysis. Representing two sides of the AI spectrum—GPT with its proprietary backbone and Llama with its open-source ethos—both models highlight the importance of diverse development environments. Each environment caters to different needs and fosters unique advancements, proving that both private and open-source models are essential for the growth of AI.

Looking ahead, the trajectory of LLMs is poised to reshape numerous sectors. The evolution from purely technical tools to integrated solutions within everyday applications suggests a future where LLMs are not just assistive technologies but foundational elements in business, creativity, and decision-making processes. The ongoing dialogue, enriched by real user feedback, will be crucial in navigating the ethical, technical, and practical challenges that lie ahead.

‍