The rise of open-source AI language models like DeepSeek challenges the notion that these tools require billions in funding. A new model, S1, further solidifies this trend, developed by Stanford and University of Washington researchers for less than $50 in cloud compute credits. This development poses a direct challenge to OpenAI’s reasoning model, o1.
Reasoning models, like o1 and S1, generate responses by “thinking” through related questions. For example, if asked to calculate the cost of replacing Uber’s fleet with Waymo vehicles, the model would break down the problem into steps, such as determining the number of Ubers and the cost of each Waymo vehicle.
S1, based on an existing language model, learned to reason by studying questions and answers from Google’s Gemini 2.0 Flashing Thinking Experimental model. According to TechCrunch, Gemini’s transparent thinking process allowed the S1 developers to train their model using a small dataset of 1,000 curated questions and answers, mimicking Gemini’s reasoning.
Remarkably, the researchers enhanced S1’s reasoning by simply instructing it to “wait.” Adding this word during the reasoning process improved accuracy, suggesting significant potential for improvement in AI models with relatively simple adjustments. This highlights the nascent nature of current chatbot technology, demonstrating their reliance on specific prompts and their difference from human thought processes. These models are essentially sophisticated next-word prediction engines trained to generate factually accurate responses using specific techniques.
OpenAI has previously criticized the Chinese DeepSeek team for training on its model outputs, a move considered ironic given the widespread data scraping used to train models like ChatGPT, a practice currently under legal scrutiny. While Google also prohibits training on Gemini’s outputs, S1’s impressive performance underscores the potential of leveraging existing models for efficient development. However, it’s important to note that S1’s success relies heavily on Gemini’s prior training, akin to a compressed JPEG image—good, but not a perfect representation. Large language models, even leading ones, still struggle with accuracy, particularly when generating text from web searches, highlighting the need for careful fact-checking. Despite these limitations, models like S1 could prove valuable for on-device processing, such as Apple Intelligence.
The emergence of affordable, open-source models raises questions about the future of the tech industry. While some argue that OpenAI’s models are easily replicable, others contend that the company’s strength lies in building applications on top of these models. ChatGPT’s widespread adoption and its synonymous association with chatbots and search demonstrate the importance of user interfaces and unique datasets. OpenAI’s Operator, which navigates the web, and xAI’s access to X (formerly Twitter) data, exemplify this differentiation.
Furthermore, the cost of inference, the processing of each user query, is expected to remain significant. As AI models become more accessible, increased demand for computing resources is anticipated, justifying investments in server farms like OpenAI’s. However, the long-term viability of this trend depends on the sustained growth and development of the AI field.
