mothcomputereally scared to run tools like sort on this file
im verifying that it parses correctly by just running jq on it and its taking minutes. actual whole minutes. several of them. and this is on a very powerful server, with an nvme drive, a lot of cache, and some very fast memory
86 gigabytes of json
Graham Sutherland / Polynomial@mothcompute still kinda cool that jq can handle it at all. nice when a tool actually works at scale.
@mothcompute this is one area where I love jsonl for array datasets. yeet that at 50 or so cores and it fliiiieees