**Cách áp dụng Batching trong Llama.cpp? Tốc độ giảm theo LOL?** 🤔

@ClimateBoss chia sẻ trải nghiệm khi dùng lệnh `./llama-server --parallel 2 --cont-batching...` và gặp phải:
- Context bị giảm một nửa 😮
- 2 người dùng = 20% chậm hơn so với 1 người? 🤯
- Batching không hiệu quả như mong đợi?

NVIDIA nói tăng người dùng sẽ tăng tổng băng thông (throughput). Làm thế nào để tốc độ tăng lên? 🚀

#LlamaCPP #AI #Performance #Batching #MLOptimизация #ViệcLàmAI #TốcĐộ #Debug #NVIDIA #AIvn

What a Sustainable Workflow Looks Like for Chronic Pain: Real-Life Examples

https://dreamspacestudio.net/what-a-sustainable-workflow-looks-like-for-chronic-pain-real-life-examples/

Why DeepSeek is cheap at scale but expensive to run locally

https://www.seangoedecke.com/inference-batching-and-deepseek/

#HackerNews #DeepSeek #Inference #Batching #Cheap #Expensive #To #Run #Local #Scale

I created a Simple Sprite Batcher for SFML. It batches them for you automatically and improves performance! Work with your normal sprites!

It's a part of my SFML Snippets repository on GitHub:
https://github.com/Hapaxia/SfmlSnippets
and also available on the SFML wiki:
https://github.com/SFML/SFML/wiki/Source:-Simple-Sprite-Batcher

The (small) class is there along with an example program.

Here's a video using it (excuse the compression issues):
Sprite batching in SFML https://youtu.be/2P13hhMgeFs?si=dbsmnlt7NqKhy4pJ

#sfml #sprite #spritebatching #batching #performance