ZSE โ€“ Open-source LLM inference engine with 3.9s cold starts

https://github.com/Zyora-Dev/zse

#HackerNews #ZSE #OpenSource #LLM #Inference #ColdStarts #ZyoraDev #AIEngineering

GitHub - Zyora-Dev/zse: Zyora Server Inference Engine for LLM .

Zyora Server Inference Engine for LLM . . Contribute to Zyora-Dev/zse development by creating an account on GitHub.

GitHub
Oh wow, you eliminated cold starts for 72M page views with *edge caching*? ๐ŸŽ‰๐Ÿš€ Next you'll tell us you've discovered fire with friction! But hey, at least your "Wall of Love" isn't just a list of bugs ๐Ÿ›.
https://www.mintlify.com/blog/page-speed-improvements #edgecaching #coldstarts #pageviews #WallOfLove #techhumor #HackerNews #ngated
How we eliminated cold starts for 72M monthly page views with edge caching

How we improved page speed performance across Mintlify documentation.

Mintlify
How we eliminated cold starts for 72M monthly page views with edge caching

How we improved page speed performance across Mintlify documentation.

Mintlify
๐Ÿ‘จโ€๐Ÿ’ปโœจ Welcome to the latest episode of "How to Overcomplicate Simple Concepts!" This time, #Cloudflare takes us on a thrilling 15-minute saga of technological jargon to 'shard and conquer' cold starts. Spoiler: no actual ice was conquered in the making of this content. ๐ŸงŠ๐Ÿ’ค
https://blog.cloudflare.com/eliminating-cold-starts-2-shard-and-conquer/ #Overcomplication #TechJargon #ColdStarts #ShardAndConquer #HackerNews #ngated
Eliminating Cold Starts 2: shard and conquer

We reduced Cloudflare Workers cold starts by 10x by optimistically routing to servers with already-loaded Workers. Learn how we did it here.

The Cloudflare Blog
AWS Lambda wants to banish cold starts like an exorcist with a magic wand named #Porffor ๐Ÿช„๐Ÿ”ฎ. Apparently, it's a JS engine that turns your beloved #JavaScript into a tiny "1MB, fast millisecond-level binary" โ€“ because compiling JS is now as easy as waving a ๐Ÿง™โ€โ™‚๏ธ wand. Meanwhile, Node and Bun continue to offer their "bundle up and deal with it" solutions, proving that not all magic tricks are created equal ๐ŸŽฉ๐Ÿ‡.
https://goose.icu/lambda/ #AWSLambda #ColdStarts #NodeBun #HackerNews #ngated
Eliminating JavaScript cold starts on AWS Lambda

Porffor can run on Lambda now!

goose.icu
Eliminating JavaScript cold starts on AWS Lambda

Porffor can run on Lambda now!

goose.icu