Mastodawn

R.L. Dane

🍵

Nov 26

For text with a lot of repetition, #bzip3 still blows my mind. 😆

rld@Intrepid:Documents$ for x in cat "gzip -9" "zstd --ultra -22" "xz -9e" "bzip2 -9" bzip3; do $x < weatherlog-2024.txt |wc -c |tr "\n" "\t"; echo "$x"; done
 1735300	cat
   80423	gzip -9
   63275	zstd --ultra -22
   53516	xz -9e
   52374	bzip2 -9
   40645	bzip3
rld@Intrepid:Documents$ echo 1735300/40645 |bc -l
42.69405830975519744125

#Lossless #Compression #LosslessCompression

P.S. times:

real 1.49 zstd --ultra -22
real 0.94 xz -9e
real 0.23 bzip2 -9
real 0.07 gzip -9
real 0.06 bzip3
real 0.00 cat

DANG. 😂

Hacker News Sep 30

How Does Lossless Compression in Fuji RAF Files Work?

https://capnfabs.net/posts/fuji-raf-compression-algorithm/

#HackerNews #LosslessCompression #FujiRAF #Files #ImageProcessing #DataCompression #PhotographyTech

How Does Lossless Compression in Fuji RAF Files work?

I spent 3 weeks earlier this year digging in to and reimplementing the lossless compression algorithm in Fuji’s RAF Camera RAW files. This post discusses the fundamentals of how it works.

fabian writes.

WetHat💦Jul 4, 2024

AlexBuz/llama-zip: LLM-powered lossless compression tool

Leverages a user-provided LLM (large language model) as the probabilistic model for an arithmetic coder. This achieves high compression ratios on structured or natural language text, since few bits are needed to encode tokens that the model predicts with high confidence.

#LLM #LosslessCompression #Python #AI #ArtificialIntelligence #Github #PythonProgramming

https://github.com/AlexBuz/llama-zip

GitHub - AlexBuz/llama-zip: LLM-powered lossless compression tool

LLM-powered lossless compression tool. Contribute to AlexBuz/llama-zip development by creating an account on GitHub.

GitHub

GeekProjects News Aug 27, 2023

Text Compression Gets Weirdly Efficient With LLMs https://hackaday.com/2023/08/27/text-compression-gets-weirdly-efficient-with-llms/ #ArtificialIntelligence #losslesscompression #lossycompression #textcompression #SoftwareHacks #neuralnetwork #compression #winzip #LLM

Text Compression Gets Weirdly Efficient With LLMs

It used to be that memory and storage space were so precious and so limited of a resource that handling nontrivial amounts of text was a serious problem. Text compression was a highly practical app…

Hackaday

Show thread

Alessio Buccino Jun 5, 2023

We started with #LosslessCompression. Across a range of general-purpose (GP) compressors, we found that #Zstandard with
@Blosc2 achieves the best compromise between compression ratio and decompression speed!

NP1: compressed size ~36%
NP2: compressed size ~52%

(3/n)