Alice Averlong🏳️⚧️okay so the chunk IDs seem to be related to different types of chunk handlers
chunk IDs 0-31 use a 3-parameter callback, and 32-47 use a 4-parameter callback
you could have just made them all take 4 parameters and just have some of them ignore the 4th parameter but NO we gotta make everything complicated so that foone's little brain can't handle it
you'd think the programmers of an Azumanga Daioh, of all games, would realize that the eventual reverse engineer hacking their game might be an Osaka, and would not over-complicated it
oh hello. Someone left the output of a tool on the disc!
Data Pack2 by OOTUKA, Technosoft Co LTD, eh?
Data Pack2 by OOTUKA, Technosoft Co LTD, eh?
that's very interesting. Technosoft had nothing to do with this game... they didn't even exist anymore when it came out.
but given the 1996-1998 dates, I'm guessing they made this tool for one of their PS1 games they released in that period, and it later got used by Ganbarion for Azumanga Donjara Daioh and the One Piece games
Shuji Yoshida is credited as "Library Program" on all three games I know that use PAC files.
It's possible he's OOTUKA.
It's possible he's OOTUKA.
or it might mean he made the APF files
okay so the output of that tool is kinda handy.
because while it's not 100% correct (they changed shit after this file was made), it's still partially correct: azending.pac DOES include endto.pac, in it's entirety
because while it's not 100% correct (they changed shit after this file was made), it's still partially correct: azending.pac DOES include endto.pac, in it's entirety
and it looks like there's a 32 or 36 byte header before the file. So maybe the PAC files are concatenated subfiles with headers right before them
okay it's a 54-byte header.
so PAC is a lazy TAR clone
so PAC is a lazy TAR clone
I just need to write a script to decode it. but my brain isn't working now
the weird thing is that the text file suggests the PAC files contain filenames, but I don't see them. Now, there IS a stretch of bytes that could be a filename, but I can't seem to decode it as anything sensible:
B3 A5 A3 B2 A5 B4 6E B0 A1 A3
B3 A5 A3 B2 A5 B4 6E B0 A1 A3
it does decode as shift-jis (which the text file was encoded as) but turns into:
ｳ･｣ｲ･ｴｰ｡｣
which I don't think makes any sense
and if you decode it as utf-16, the most reasonable encoding for windows computers at the time, you end up with ꖳ늣뒥끮ꎡ, which makes even less sense.
I'm pretty sure they didn't name the files in their Azumanga Daioh game in a mix of Mande, Korean, and Sino-Tibetan scripts
but by matching up the filenames with the text file (azmem.txt) and what subfiles are definitely inside azending.pac, that pile of gibberish is supposed to mean "secret.pac"
maybe this means something.
the "C" in "SECRET" is encoded the same as the "C" in "PAC"
And note that the A in PAC is encoded as A1, which is only 2 less than the A3 which C is encoded as.
the "C" in "SECRET" is encoded the same as the "C" in "PAC"
And note that the A in PAC is encoded as A1, which is only 2 less than the A3 which C is encoded as.
what encoding puts ABCDEF at A1 and up, though?
answer: nothing python 3.11 can encode to.
Maybe this isn't an encoding. Maybe this is encryption.
it's just the ascii value + 64
B3 A5 A3 B2 A5 B4 6E B0 A1 A3
subtract 64 from each letter
>>> ''.join(chr(x-64) for x in [0xB3,0xA5,0xA3,0xB2,0xA5,0xB4,0x6E,0xB0,0xA1,0xA3])
'secret.pac'
also the 54-byte header thing was wrong. it's variable length, because of course it is!
okay so, PAC:
the header for the file itself is 16 bytes.
Then each chunk starts with a null-terminated string, encoded with that silly +64 ASCII mode.
Then there's another NUL byte, then 32 bytes of per-chunk header, then the raw chunk data.
the header for the file itself is 16 bytes.
Then each chunk starts with a null-terminated string, encoded with that silly +64 ASCII mode.
Then there's another NUL byte, then 32 bytes of per-chunk header, then the raw chunk data.
ugh.
the +64 ascii string thing doesn't work for all files. some of them end up negative
the +64 ascii string thing doesn't work for all files. some of them end up negative
34 B6?
THAT DOESN'T MAKE ANY SENSE
way too short to be a filename and it's also -12, 118 after decoding
HOW DO YOU HAVE NEGATIVE ASCII INDEXES
if we assume it loops around and thus this should be F4 76, it's not valid shift-jis, but in utf-16 it'd be 直, which... makes little sense.
I changed my code to ignore that sometimes the filenames make no sense, but then it errors after that: apparently the filenames not decoding ALSO breaks the variable-length headers. Interesting.
interesting: logo.pac goes "40 3F 00 00 A7 AC AF A7 AF 9F 70 71 6E B4 A9 AD 60 D4"
so my code was stopping after 40 3F.
but A7 AC AF A7 ... looks more like a filename
and it encodes as "glogo_01.tim \x94"
so I must be missing something, like some out-of-band file length indicator
got it. the first 2-4? bytes of the PAC are a list of how many 4-byte words come before the filename.
the 40 3f 00 00 before the filename in LOGO.PAC isn't part of the filename, it's part of the header.
the 40 3f 00 00 before the filename in LOGO.PAC isn't part of the filename, it's part of the header.
I can't figure out how it's determining when filenames end, though.
Maybe it's assuming they all have extensions and all extensions are 3 letters long?
that makes some of the files make sense and some of the others not make sense!
oh god there is compression
not all files are compressed. but some are
found the code where it parses the PAC headers.
It's terrible as expected.
The pre-pac header stuff gives you a pointer into each header, but then the fun part is that the pointer is not to the beginning, it's to the middle. So it looks things up by indexing forward AND backward
so the filename starts at the offset of, uh, negative 28
and here's how it determines the ending: it's until it hits a 0, OR the filename ends up being 12 characters long.
FUCK
someday I'm gonna reverse engineer a game and not want to timetravel back to its creation and ask them WHAT THE FUCK at gunpoint
sometimes I won't even ask, I'll just start shooting
so I'm just gonna take all my current PAC parsing code and throw it out and replace it with the nonsense of the actual code.
that was my fatal mistake: I was writing parsing code assuming this shit made any fucking sense
also I think there's a mistake in this code OR ghidra is decoding it incorrectly.
it seems to be trying to ensure all filenames are uppercase, but because it's wrong, it is corrupting all non-lowercase characters.
it seems to be trying to ensure all filenames are uppercase, but because it's wrong, it is corrupting all non-lowercase characters.
they might not have noticed if they apply the same "uppercase" transformation when trying to load filenames, because both would be corrupted in the same way
okay so now I've got working filenames, offsets, lengths, and compressed lengths. So I can find out what files are where and if they're compressed. I can't uncompress them yet.
I have located the decompression routine.
now to try to figure out what the fuck it does
now to try to figure out what the fuck it does
this decompression routine is big-endian.
on a little-endian system.
WHERE DID THEY GET THIS
it seems it's loading 16bit lengths, then using the top 15 bits? with the lowest bit as a flag?
I don't recognize this. I don't think it's DEFLATE
so looking at this code, it doesn't seem to involve huffman encoding. there's no tables, just some look-back with a sliding (I think?) window.
So this is just a slightly fancy RLE, I think?
Matteꙮ Italia@foone some LZ77 variant maybe?
I'm gonna try to bypass figuring out the compression right now by just stuffing the ghidra code into a C program and calling it from python
Bingo. it works!
Mostly. my output file is always 64mb but that's because I don't have a good way to tell how big it should be
there's some weird-ass shit going on here. like, the datafiles have some PAC chunks with type 36.
As far as I can tell, there's no code that handles chunk-36.
So the only way that makes sense is if part of the game dynamically loads code which then registers a chunk-36 parser
As far as I can tell, there's no code that handles chunk-36.
So the only way that makes sense is if part of the game dynamically loads code which then registers a chunk-36 parser
so each character is stored in a a PAC file under viewer\ (inside AZU.APF)
So like, the first Chiyo-chan is chi_v.pac
That PAC file contains 91 texture images and a 95 kilobyte GMD file, which seems to contain all the geometry AND animations.
So like, the first Chiyo-chan is chi_v.pac
That PAC file contains 91 texture images and a 95 kilobyte GMD file, which seems to contain all the geometry AND animations.
so the next step is to figure out how GMD works.
Fortunately I know where the function that parses it starts
Fortunately I know where the function that parses it starts
the code also seems to handle decoding two versions of the GMD format, but I can only find one in use in the datafiles.
Maybe they used the other in the One Piece games, and just never dropped support?
oh hello. I took a quick look at the second of the One Piece games, and it turns out they did the same thing as Azumanga and included the executable inside the APF file... but they did it twice, and the two don't match!
and it looks like that for the second One Piece game, it uses .TMD files, which have a completely different header than the two supported by azumanga
ahh. it looks like the GMD format gets loaded recursively, and I bet that's why there are a total of 3 (not 2, as I suspected) different versions of it.
I bet versions 1 and 0 appear inside version 2
I bet versions 1 and 0 appear inside version 2
h100gfldIs this encryption dot gif
The Doctor@foone That sounds like a BOFH excuse.
Studio 8502 
HarJIT@foone Gives me flashbacks to https://ccsids.net/3-3220-050/sc354.html
endrift 🏳️⚧️
FozzTexx@foone sounds like so,ethung they copied from CP/M
Paul Lalonde@foone DOS filenames - 8.3, so 12 chars. Ugly.
Sekoia@foone isn't there a C function that does exactly that? fnread or smth? Technically that saves a byte, but... it's a singular byte. Just null-terminate it.