'Stability AI,' the company most known for its Stable Diffusion image-generation and image editing service, recently announced two new Large Language Models (LLMs) named FreeWilly1 and FreeWilly2. What is unique about the FreeWilly LLMs when compared to traditional LLMs is that the FreeWilly models are trained using synthetic data and concentrated datasets.
The name for the models, FreeWilly, comes from the story about the baby whale in the '90s. The relevance of the whale to the LLM is that the FreeWilly LLMs are based on Microsoft's 'Orca' AI training methodology. However, the FreeWilly models only use 600,000 datapoints, or roughly 10% of the Orca method, which means they are essentially baby whales. Stability AI is aiming to show the efficacy of smaller, more focused LLMs, rather than all-encompassing LLMs, both for reducing environmental impact and for ensuring accuracy of results on a smaller scale.
Synthetic Data-Trained LLMs
Stability AI Unveiled Two 'FreeWilly' LLMs Trained on Synthetic Data
Trend Themes
1. Synthetic Data-trained Llms - Using synthetic data and concentrated datasets to train Large Language Models (LLMs) opens up disruptive innovation opportunities in language processing.
2. Freewilly Llms - The development of LLMs like FreeWilly1 and FreeWilly2, which are trained using Microsoft's 'Orca' AI training methodology but utilize a smaller dataset, presents a disruptive innovation opportunity for creating more focused and accurate language models.
3. Reducing Environmental Impact - The focus on smaller, more focused LLMs by Stability AI has the potential to reduce environmental impact by training models with fewer resources.
Industry Implications
1. Artificial Intelligence - The use of synthetic data and concentrated datasets for training LLMs has the potential to revolutionize language processing within the artificial intelligence industry.
2. Data Science - The development and implementation of smaller, more specialized LLMs like FreeWilly1 and FreeWilly2 provides disruptive innovation opportunities within the field of data science.
3. Sustainability - Stability AI's emphasis on smaller, more efficient LLMs aligns with the goal of sustainability, which can impact industries across various sectors.