1. Meta may release the next version of its language model Llama 3 as early as next week.
2. Llama 3 will come in various sizes, including a small version for early release and a full open source model in July.
3. Llama 3 is expected to be multimodal and have a range of sizes, with fewer moderation controls than its predecessor.
Meta is set to release the next version of its large language model, Llama 3, as early as next week, with a full open-source model expected to come out in July. The company is investing heavily in advanced AI systems, including purchasing H100 GPUs from Nvidia to train Llama and other models. Llama 3 will range from small versions to compete with other models like Claude Haiku and Gemini Nano, to larger models that can provide full responses and reasoning similar to GPT-4 and Claude Opus.
Llama 3 is expected to be open source and multimodal, capable of understanding both visual and text inputs. It will come in various sizes ranging from 7 billion parameters to over 100 billion parameters, smaller than the trillion-plus parameters used to train GPT-4. The new model is likely to be less cautious than its predecessor, with fewer moderation controls and guardrails.
Meta’s decision to release a small version of Llama 3 early may be part of a consistent release schedule and to build hype around the upcoming AI model. Smaller models like Anthropic’s Claude 3 Haiku are already showing capabilities similar to larger models like GPT-4. The AI model space is growing quickly and becoming competitive, with new models from companies like DataBricks, Mistral, and StabilityAI. Smaller models are also appealing to businesses for their cost-effectiveness, ease of fine-tuning, and potential for running on local hardware.