RE: https://fosstodon.org/@Blosc2/116481891984444131

The beast is out of his cage! 🧸

Look at the bump in performance that we will see with the next Python-Blosc2 release.

Matrix multiplication has been speeded up by using blocks and Blosc2 prefilters and its own efficient and multithreaded engine.

Expect between 5x and 6x better speed for matrices with no padding.

Huge shoutout to @luke_shaw_ironarray Shaw for making this happen!

We designed the DSL kernels in Blosc2 so that they could be used in the main platforms, and nowadays this necessarily includes Web Assembly (WASM) in the browser.

We also implemented a new JIT compiler for WASM specially meant for Blosc2 DSL kernels.

Look at this online notebook: https://cat2.cloud/demo/roots/@public/examples/mandel-jit-vs-nojit.ipynb
where we run of a Mandelbrot DSL kernel, with and without JIT. You can even run it yourself by just clicking in the "Run" button.

Enjoy!

#WASM, #Pyodide #HPC

Blosc2 4.1 Release!

We've packed this minor release: optimised compression and funcs for unicode arrays; cumulative reductions; memory-map support for store containers like `DictStore` ; and a DSL kernel functionality for faster, compiled, user-defined funcs!👇

Notebook here - https://github.com/Blosc/python-blosc2/blob/main/examples/ndarray/mandelbrot-dsl.ipynb

Super-efficient DSL kernels are coming with forthcoming Python-Blosc 4.1.0.

A DSL kernel is a function that takes ndarray objects as input and returns a ndarray object as output.

These allow for full vectorized/tensorized operations on Blosc2 NDArray objects (or NumPy/PyTorch arrays). They can be JIT compiled too!

The performance of those with the classic Mandelbrot example is kind of mind-blowing.

See https://github.com/Blosc/python-blosc2/blob/main/examples/ndarray/mandelbrot-dsl.ipynb for more details.

Just took the new engine in Python-Blosc2 4.0 for a spin, and the performance results are quite awesome. Processing a 400 MB array:

🔹 NumPy baseline: 146 ms
🔹 Blosc2 (on NumPy arrays): 73.1 ms (2x faster)
🔹 Blosc2 (on native Blosc2 arrays): 15.1 ms (**9.6x faster!**) 🤯

The best part? It fully supports NumPy's array and ufunc interfaces. High performance with zero friction! 🏎️💨

More info: https://ironarray.io/blog/miniexpr-powered-blosc2

#Python #DataScience #NumPy #HighPerformanceComputing #Blosc2 #OpenSource

📢 OSS Synergy : Blosc2 🤝 OpenZL 🔁

Exciting plugin🔌announcement - you can now use the new OpenZL compression 🗜️ library from Blosc2: https://github.com/Blosc/blosc2-openzl!

It's as simple as

𝚙̲𝚒̲𝚙̲ ̲𝚒̲𝚗̲𝚜̲𝚝̲𝚊̲𝚕̲𝚕̲ ̲𝚋̲𝚕̲𝚘̲𝚜̲𝚌̲𝟸̲–̲𝚘̲𝚙̲𝚎̲𝚗̲𝚣̲𝚕̲

and just like that, OpenZL compression + Blosc2 compute engine!

Thanks to Yann Collet, his team and all contributors to the OpenZL project - check it out here https://openzl.org/.

Find out more about how the Blosc team implemented this plugin here ⏩ https://blosc.org/posts/openzl-plugin/