naturepokerMar 19
Looking at some plasmidsaurus generated pod5 files - bro, 15gb archive decompresses into a 15gb pod5 file. Why are they just adding on processing time for the user? 🤔
Show threadMihail Zdravkov
@naturepoker it may be a requirement by a database. I recently uploaded pod5s for a publication to ENI and they required a tar archive.
Mar 19 at 7:50amWeb
Show threadnaturepokerMar 19
@mzdravkov I think so too. Though being a commercial service people pay for (they charge extra for pod5 download) I'd appreciate some extra compression!
Show threadLeszekMar 19
@naturepoker @mzdravkov pod5 are very efficiently compressed internally so adding anything in top would add very little
https://github.com/nanoporetech/vbz_compression
Note, you can mount tar archives
https://github.com/mxmlnkn/ratarmount
GitHub - nanoporetech/vbz_compression: VBZ compression plugin for nanopore signal data

VBZ compression plugin for nanopore signal data. Contribute to nanoporetech/vbz_compression development by creating an account on GitHub.

GitHub
Show threadMihail ZdravkovMar 19
@lpryszcz @naturepoker yeah, as far as I know, if you need extra compression you probably have to go to lossy-compression. You may find this paper interesting, if you haven't seen it: https://genome.cshlp.org/content/35/7/1574.short
A new compression strategy to reduce the size of nanopore sequencing data

An international, peer-reviewed genome sciences journal featuring outstanding original research that offers novel insights into the biology of all organisms

Show threadLeszekMar 19
@mzdravkov @naturepoker yes, I've seen it. Impressive work by @hasindu2008 et al :)
Show threadnaturepokerMar 19
@lpryszcz @mzdravkov @hasindu2008 this is all very good to know, thank you for the heads up!
Show threadHasindu GamaarachchiMar 28

@mzdravkov @naturepoker
For ENA when tar.gz is a requirement, I just pack the BLOW5 to a tar.gz with 0 compression to avoid this kind of unnecessary compression/decompression.

See Q7 in https://hasindu2008.github.io/slow5tools/faq.html for the commands.

Frequently Asked Questions

Slow5tools is a toolkit for converting (FAST5 <-> SLOW5), compressing, viewing, indexing and manipulating data in SLOW5 format.

slow5tools
Show threadMihail ZdravkovMar 28
@hasindu2008 Thanks, that's a really useful tip.