iPhone 17 Pro Successfully Showcases Ability to Run a 400B Large Language Model, Demanding at Least 200GB of Memory Even in Compressed Form #gadget

iPhone 17 Pro can run a 400B language model, but only via streaming from storage and a MoE approach, not fully in memory. Expect about 0.6 tokens per second and notable battery drain. Practicality aside, this proves on-device LLMs are closer than ever. https://ift.tt/2A67Us0

Source: https://ift.tt/2A67Us0 | Image: https://ift.tt/HOPhcp5