sayzard3d ago

Content-defined chunking added to Bazel

BuildBuddy가 Bazel 원격 캐시에 Content-Defined Chunking(CDC)을 도입해 대형 빌드 출력물의 중복 데이터를 효율적으로 재사용한다. CDC는 파일 내용을 기준으로 청크를 나누어 변경된 부분만 전송하고 저장하여 네트워크 사용량과 디스크 캐시 크기를 약 40% 줄였다. 이 기술은 특히 GoLink 같은 트랜지티브 액션에서 작은 소스 변경이 대용량 출력물 전체를 다시 업로드하는 문제를 해결하며, 수백 테라바이트의 중복 데이터 전송을 방지해 빌드 효율성을 크게 향상시킨다. Bazel 8.7 이상에서 실험적 기능으로 활성화 가능하다.

https://www.buildbuddy.io/blog/content-defined-chunking/

#bazel #buildcache #contentdefinedchunking #cdc #buildbuddy

Remote Cache CDC: Reusing Bytes | BuildBuddy

BuildBuddy provides enterprise features for Bazel — the open source build system that allows you to build and test software 10x faster.

GripNewsApr 13
🌖 如何將 Firefox 建置速度提升 17%
➤ 透過 Lua 外掛釋放 buildcache 潛力，加速程式碼生成任務
✤ https://blog.farre.se/posts/2026/04/10/caching-webidl-codegen/
本文介紹如何利用 buildcache 的 Lua 外掛系統，針對 Firefox 建置過程中的 WebIDL 程式碼生成步驟進行快取最佳化。作者透過編寫自訂的 Lua 封裝程式，讓原本無法被傳統編譯器快取（如 ccache 或 sccache）處理的 Python 生成任務，也能納入快取機制。實驗數據顯示，這項調整能將冷啟動後的「熱建置」時間縮短 15 秒，進一步優化了開發者的編譯測試週期。此技術未來可推廣至 Firefox 建置流程中其他確定性的自動生成步驟。
+ 這種能針對特定非編譯器任務撰寫快取規則的彈性，確實是 buildcache 相比於 ccache 的巨大優勢。
+ 將 WebIDL 生成步驟納入快取對於經常執行 clobbe
#Mozilla #Firefox #建置最佳化 #buildcache #WebIDL
How to make Firefox builds1 17% faster2

In the previous post, I mentioned that buildcache has some unique properties compared to ccache and sccache. One of them is its Lua plugin system, which lets you write custom wrappers for programs that aren’t compilers in the traditional sense. With Bug 2027655 now merged, we can use this to cache Firefox’s WebIDL binding code generation.

farre’s blog
Jan  Apr 23, 2025

#NativeLink is an open source high-performance build cache and remote execution server, compatible with Bazel, Pants, Buck2, Reclient, and other RBE-compatible build systems. It offers drastically faster builds, reduced test flakiness, and support for specialized hardware.

https://github.com/TraceMachina/nativelink

#Cache #BuildCache #CI

GitHub - TraceMachina/nativelink: NativeLink is a Nix-powered, open source, high-performance build cache and remote execution server, compatible with Bazel, Soong, Pants, Buck2, Reclient, and other RE-compatible build systems. It offers drastically faster builds, reduced test flakiness, and support for specialized hardware.

NativeLink is a Nix-powered, open source, high-performance build cache and remote execution server, compatible with Bazel, Soong, Pants, Buck2, Reclient, and other RE-compatible build systems. It o...

GitHub
GradleMar 22, 2023
#Gradle Build Tool has a fantastic plugin ecosystem, but finding a #plugin that works well can be a challenge. Join our talk with Gradle’s Gary Hale for plugin authors to learn how to write plugins that consumers will love. We'll cover #pluginarchitecture, #taskconfiguration avoidance, configuration #cache compatibility, #buildcache planning and #testing, and backwards compatibility. Keep your plugin healthy and well-behaved as Gradle evolves - https://devnexus.com/presentations/writing-gradle-plugins-that-users-will-love
Writing Gradle plugins that users will love

GradleFeb 16, 2023
Find out how developers can easily observe #BuildCache performance continuously, especially if using the remote cache for #LocalBuilds. Check out our latest blog post and learn how the #CashApp team observes remote build cache performance and tracks avoidance savings per build type -
GradleDec 12, 2022
Parallelizing builds and reducing overhead with the #Gradle #BuildCache: Real-world experience from Elastic's @[email protected] and Google's @[email protected] - https://www.youtube.com/watch?t=1467&v=ltVD87kVpEM&feature=youtu.be
Elastic & Google Cloud Platform | Dev Cloud Acceleration with Gradle Enterprise | 2021

YouTube
Show threadcitizenkahnOct 16, 2019
@aeveltstra have you experimented with the #gradle #buildCache? I recently tried it and cut a 4m build to 35s. I was pretty happy with that. Getting #intellij to use the cache has been a challenge