– Microsoft announces Phi-3 family of open small language models (SLMs), outperforming larger models on language, coding, and math benchmarks
– Phi-3-mini model with 3.8 billion parameters is available, with Phi-3-small and Phi-3-medium to follow
– SLMs offer on-device deployment for low-latency AI experiences, with potential use cases including smart sensors, cameras, and farming equipment
Microsoft has introduced the Phi-3 family of small language models (SLMs) that are said to be highly capable and cost-effective. These models have been developed using an innovative training approach by Microsoft researchers, allowing them to outperform larger models in language, coding, and math benchmarks. The first model, Phi-3-mini, with 3.8 billion parameters, is now available in various platforms such as Azure AI Model Catalog and Hugging Face.
The shift towards a portfolio of models instead of just one singular category is highlighted by Microsoft, allowing customers to choose the best model for their specific needs. The VP of AI at Microsoft, Luis Vargas, emphasizes the importance of combining both small and large models for different use cases. SLMs offer the advantage of on-device deployment for low-latency AI experiences without the need for network connectivity.
Microsoft’s innovative data filtering and generation approach, inspired by bedtime storybooks, led to a quality leap in SLMs. The company has curated high-quality training data such as the ‘TinyStories’ and ‘CodeTextbook’ datasets, which have improved the models’ performance and reasoning abilities significantly. Despite the careful data curation, Microsoft ensures the application of additional safety practices to mitigate AI safety risks associated with the Phi-3 release.
Overall, Microsoft’s Phi-3 family of SLMs offers a compelling alternative to large language models, providing improved performance and reasoning abilities without the massive computational costs. By offering a variety of models and focusing on high-quality training data, Microsoft aims to make AI solutions more accessible to businesses and alleviate adoption barriers.