"I’m experiencing significant slow-downs when converting data from #LMDB to HDF5 format."

Solution: don't do that. #HDF5 is a pig.

https://stackoverflow.com/questions/79706522/hdf5-write-performance-degrades-over-time-when-converting-from-lmdb-3-7m-entri

I've surprised myself how fun it is to write this pure #haskell #hdf5 parser. It's, in a way, gruesome, fiddling with raw bytes, guessing some details, getting new files with features still unsupported, and I have no clue how exciting it will be to implement all the common filters. But the language is delightful and I'm making constant glacial progress. That accounts for a lot I suppose...

🚀 Excited to share more about Caterva2, your ultimate gateway to Blosc2/HDF5 repositories! 🚀

Caterva2 is designed to redefine how you interact with large datasets.

Want to see it in action? 🤔 We've just released a new introductory video showcasing Caterva2's main functionalities! 🎬

👉 https://ironarray.io/caterva2

#Caterva2 #Blosc2 #HDF5 #BigData #DataManagement #FreeSoftware #Python #DataScience #Tech

✨ Ever wondered how to effortlessly apply lazy expressions to massive tomographic HDF5 datasets and visualize them instantly in Cat2Cloud? 🤯

🎬 Check out this fantastic new video from our very own Luke Shaw! Discover how a bit of Python 🐍 magic and our user-friendly web interface simplify the process.

👇 Watch the full video to see the magic unfold: https://ironarray.io/cat2cloud

#HDF5 #Tomography #LazyEvaluation #DataVisualization #Cat2Cloud

Now we have THomas from #EuXFEL with “down to the bytes: can we simplify alternative access to #HDF5?“

– h5glance

#HDF5 User Group meeting, preparing for day two …

Zdenek from MAX IV #Synchrotron in Lund shows their “innovative data acquisition solutions with HSDS“.

Now it's Gerd from the #HDF5 group showing the history and future of the “most versatile container for sharing scientific and engineering data”.
Now Elena from the #HDF5 group / lifeboat is revisiting SWMR (single writer, multiple readers) and file versioning features.
MIchael from PureStorage continues on #HDF5 scalability and consistency for #HPC High Performance Computing-Storage.
Now it's @FrancescAlted to introduce the #Blosc2 #compression algorithm to reduce #HDF5 file size.