1. Gemma is an open weights model family for developers to experiment with, adapt, and productionize on Google Cloud using PyTorch and JAX.
2. Gemma models can run on various platforms including laptops, workstations, Vertex AI, and GKE using Cloud GPUs or Cloud TPUs for training, fine-tuning, and inference.
3. Gemma models, such as Gemma 2B and Gemma 7B, have different architectures and were pre-trained with 2 trillion and 6 trillion tokens respectively, utilizing Rotary Positional Embeddings.
Earlier this year, Google introduced Gemma, an open weights model family designed to allow developers to easily experiment, adapt, and deploy on Google Cloud. Gemma models can be run on various platforms, including laptops, workstations, Vertex AI, and GKE, using Cloud GPUs or TPUs with tools like PyTorch, JAX, vLLM, HuggingFace TGI, and TensorRT LLM.
Benchmark tests have shown up to 3X training efficiency for Gemma models using Cloud TPU v5e compared to the baseline Llama-2 performance. Recently, JetStream was released as a cost-efficient and high-performance inference engine, demonstrating a 3X gain in efficiency for LLM inference when serving Gemma models.
The Gemma family consists of two variants, Gemma 2B and Gemma 7B, with different architectures and pre-training strategies using trillions of tokens. Gemma 2B employs multi-query attention to reduce memory bandwidth requirements, potentially advantageous for on-device inference scenarios.
Training performance for Gemma models is assessed based on Effective Model FLOPs Utilization (EMFU) and relative performance per dollar, with pre-training done using Cloud TPU v5e. The Gemma models were tested on Cloud TPU v5e and Cloud TPU v5p, the most cost-efficient and powerful TPUs available, respectively, showcasing their training efficiency and performance.
Overall, the Gemma models show promising results in training and inference performance on Google Cloud accelerators, with room for further evolution and improvement through community contributions and ongoing development efforts. The architecture details of Gemma models, such as the use of Rotary Positional Embeddings and different attention mechanisms, contribute to their efficiency and effectiveness in various scenarios.