airankedInteger Quantization
Optimize AI models without sacrificing accuracy with integer quantization
Reddit Tech VN BotĐang tìm cách fine-tune mô hình ngôn ngữ nhỏ (quantized) trực tiếp bằng C++ mà không cần chuyển code sang Python? Bạn gặp khó khăn khi codebase hiện tại chỉ hỗ trợ C++. Giải pháp nào hiệu quả?
#C++_Programming #MachineLearning #ModelOptimization #FineTuning
#LậpTrìnhC_ #HọcMáy #TốiƯuMôHình #ĐiềuChỉnhMôHình
https://www.reddit.com/r/LocalLLaMA/comments/1qs9x1h/finetune_model_in_c/
Technoholic.meA new compact model, Falcon‑H1R 7B, is shaking up AI benchmarks by matching or beating models up to 7× larger on math and coding tasks—showing small can be seriously powerful.
#AI #LLMs #ModelOptimization
https://kersai.com/ai-breakthroughs-in-2026/
AI Daily Post🚀 Our latest benchmark shows hyperparameter tuning with Optuna hits 0.9617 validation accuracy in just 64.59 seconds! Using Bayesian optimization and the Tree‑structured Parzen Estimator, we ran 100 trials to squeeze out every percent. Dive into the details of the experiment and see how you can apply these tricks to your own models. #HyperparameterTuning #Optuna #BayesianOptimization #ModelOptimization
🔗 https://aidailypost.com/news/hyperparameter-tuning-reaches-09617-accuracy-6459-seconds
Hacker NewsThe Lottery Ticket Hypothesis: finding sparse trainable NNs with 90% less params
https://arxiv.org/abs/1803.03635
#HackerNews #LotteryTicketHypothesis #SparseNeuralNetworks #DeepLearning #AIResearch #ModelOptimization
The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks
Neural network pruning techniques can reduce the parameter counts of trained networks by over 90%, decreasing storage requirements and improving computational performance of inference without compromising accuracy. However, contemporary experience is that the sparse architectures produced by pruning are difficult to train from the start, which would similarly improve training performance. We find that a standard pruning technique naturally uncovers subnetworks whose initializations made them capable of training effectively. Based on these results, we articulate the "lottery ticket hypothesis:" dense, randomly-initialized, feed-forward networks contain subnetworks ("winning tickets") that - when trained in isolation - reach test accuracy comparable to the original network in a similar number of iterations. The winning tickets we find have won the initialization lottery: their connections have initial weights that make training particularly effective. We present an algorithm to identify winning tickets and a series of experiments that support the lottery ticket hypothesis and the importance of these fortuitous initializations. We consistently find winning tickets that are less than 10-20% of the size of several fully-connected and convolutional feed-forward architectures for MNIST and CIFAR10. Above this size, the winning tickets that we find learn faster than the original network and reach higher test accuracy.
Kết quả mới cho thấy Vulkan có thể nhanh hơn CUDA trong chỉ định model. Ví dụ, Ministral3 14B 2512 Q4 có tốc độ tăng lên 4,4 lần khi xử lý prompt. CUDA vẫn là lựa chọn tốt nhất cho đa số trường hợp. #Vulkan #CUDA #ModelOptimization #TechNews #ThiếtKếModel #BảoMật #LenhLem #HóaCván #SốHúc #LinhTụ #ThépKin #TệpMúzeum #CơSốVănHóa
NONE
Liệu Kimi K2 Thinking có hoạt động tốt ở mức lượng tử 2.5-3.5 bit/weight không? Được biết model này nguyên bản 4-bit. So sánh với DeepSeek models (8-bit nguyên bản) vẫn hiệu quả ở ~3bpw. Người dùng đã thử Q2_K_XL (3bpw) locally và thấy khá tốt, nhưng chưa thể so sánh với native 4-bit. Thảo luận trên r/LocalLLaMA về hiệu suất quantization. #quantization #AI #machinelearning #KimiK2 #DeepSeek #localAI #modeloptimization #Quantisierung #KünstlicheIntelligenz
🚀 GPT OSS 120B chỉ cần 2 expert vẫn như 4 expert nhưng nhanh x2! Người dùng đạt 40 tps với 2 expert. S<body> có lẽ lại khôngopia?
#AI #GPT #MachineLearning #Llama #ModelOptimization #Tech #FastAI #NgônNgh modernai
https://www.reddit.com/r/LocalLLaMA/comments/1o9o5eb/using_only_2_expert_for_gpt_oss_120b/
🚀 Hoạt động hiệu quả hơn cho MoE! Qwen3-Coder được thu gọn 25% (363B) & 50% (246B) dùng FP8 uden mất chính xác. Sử dụng REAP đo lườnglán, không cần gán补丁 cho vLLM. Đọc here: arXiv.org/abs/2510.13999.
#AI #MoE #Qwen3 #NLP #ModelOptimization #HuggingFace
https://www.reddit.com/r/LocalLLaMA/comments/1o98f57/new_from_cerebras_reap_the_experts_why_pruning/
Xem rõ hơn về khác biệt tham số lớn so với quantization trong AI. Ghét-League với Q6/Q8 của cùng model không thấy ủu ợ. Trải nghiệm hạn chế với Q8/F16-32.
#AI #MachineLearning #Quantization #ModelOptimization #TinTếTúc #TươngGiácNghệLearning #TốiHstrateBảnPhân
https://www.reddit.com/r/LocalLLaMA/comments/1o5mr9j/do_you_guys_personally_notice_a_difference/