Glad to read it's possible. This must have been overlooked, before releasing 26B+ models in the wild.. #memoryshortage #Intellectualproprety
"In Phison testing, aiDAPTIV enabled a 26B-parameter model to run on a system with 16 GB of DRAM, compared with 32 GB of DRAM required without aiDAPTIV in the same test environment."



