Microsoft has recently released a new small language model called Phi-3 mini, which is optimized for use on modern smartphones. This updated version of the company’s lightweight language model offers performance similar to OpenAI’s GPT-3.5, despite being trained with only 1.4 billion tokens compared to its predecessor, Phi-2, which was trained with 3.3 billion tokens. The new model has 3.8 billion parameters and can fit on a modern smartphone, taking up approximately 1.8GB of memory and can be quantified to 4 bits.
Researchers tested the Phi-3 mini on an iPhone 14 with an A16 Bionic chip and found that it ran natively and offline at a speed of over 12 tokens per second, which is comparable to larger models like Mixtral 8x7B and GPT-3.5.
The transformer decoder architecture used in the Phi-3 mini supports a text length of up to 4K characters and is based on a block structure similar to Meta’s Llama 2, allowing packages developed for Llama 2 to be used with this model.
Phi-3 mini is designed for conversational chat formats and aligns with Microsoft’s values of robustness and security. In addition to the new model, Microsoft has also trained two other models in the same family: Phi-3 medium with 14 billion parameters and Phi-3 small with 7 billion parameters, both trained with 4.8 billion tokens.
Microsoft is focused on providing efficient and secure language models for a variety of applications with these updates to its language model family.
Overall, the Phi-3 mini offers impressive performance despite being smaller than some of its larger counterparts, making it an attractive option for developers looking for a lightweight yet powerful language model that can run natively on modern smartphones.