Devine Lu LinvegaNov 16, 2023

So, for a little project I needed to compress a bit of data, and @cancel and I made up a spec that should be possible to implemented on small systems without too much headaches.

I wrote a bit of documentation for it, and I was wondering if anyone wanted to put the docs to the test and see if they could write a toy implementation in a language of their choice. It'd help me see if there's things missing in the docs.

http://wiki.xxiivv.com/site/ulz_format

So, think of it as a little puzzle.

XXIIVV — ulz format

Show threadRenaud BédardNov 16, 2023
@neauoire @cancel trying my hand at a C# version of this!
Show threadDevine Lu Linvega
@renaudbedard @cancel Keep me posted :>
Nov 16, 2023 at 2:12amWeb
Show threadRenaud BédardNov 16, 2023
@neauoire @cancel A couple of things that are tripping me :
- CPY uses a "negative offset plus 1", but it's not clear if it's "-offset + 1" or "-(offset + 1)"
- It's implied that the dictionary pointer is always pointing to the end of the dictionary buffer after the last copy/append operation, right?
Show threadDevine Lu LinvegaNov 16, 2023

@renaudbedard @cancel

Example: an offset of 0 means go back by 1 bytes into the history.

And the pointer is the at the end of the dictionary buffer yep.

I'll add a bit about the negative increment to the offset to the docs! thanks for pointing it out

Show threadRenaud BédardNov 16, 2023
@neauoire @cancel also I know HTML tables are a nightmare, but if you could make the LIT length box appear to take all 7 bits instead of 6 like CPY1... 
Show threadRenaud BédardNov 16, 2023
@neauoire @cancel wait a minute... if the length of a CPY is greater than its negative offset, what's the expected behavior?
Show threadcancelNov 16, 2023
@renaudbedard @neauoire 
Show threadRenaud BédardNov 16, 2023
@cancel @neauoire that part was definitely unclear to me, but if I loop over the offset until length is hit, it does decode the test case! 🎉
Show threadcancelNov 16, 2023
@renaudbedard @neauoire nice :D here's my original version of the document from like 9 months ago https://gist.github.com/randrew/b43c011ccb44cb8cc6fa9d009ac58a7f the one you read was written by devine after reading mine, so the game of telephone appears to work
uxn lz explanation.md

GitHub Gist: instantly share code, notes, and snippets.

Gist
Show threadDevine Lu LinvegaNov 16, 2023

@renaudbedard @cancel I didn't even implement the loop around myself 🤦‍♀️

Well done! the documentation and telephone implementation works! :D

Show threadcancelNov 16, 2023
@neauoire @renaudbedard the fun part is you don't actually need to explicitly implement looping over the dictionary when the length is longer than the dictionary. you just plow forward and "overflow" into the data you already wrote and keep re-appending it.
Show threadcancelNov 16, 2023
@neauoire @renaudbedard (ok, you do need to explicitly loop over the dictionary if you're doing a streaming/resumable implementation, since you have to store the dictionary separately from the output instead of just treating it as the last 256 bytes)
Show threadDevine Lu LinvegaNov 16, 2023
@renaudbedard @cancel mhm I thought a fixed that, could you reload?
Show threadRenaud BédardNov 16, 2023
@neauoire @cancel yep it’s good now!