I published a new post, this time trying to make the most of the little RAM I had to create a #TileDB #soma - a #singlecell format and database - but for the complete #LINCS 2020 dataset. A measly 140GB+ of data.
I didn't want to re-invent the wheel, so ended converting between multiple file formats to avoid as many in-memory operations as possible.
https://amjdomingues.com/posts/2025-11-02-cmap-tile-db-build/
There are probably less evolved ways of doing this, but this is the one my technical skills allowed:)
