Hal Pomeranzjq is super useful, once somebody explains the basics to you. Here I am explaining the basics in a way that's applicable for all you DFIR types.
Nullstring 🏴☠️@hal_pomeranz jq is my boyfriend
@0x00string jq is in my "parsing hall of fame" along with awk and tshark.
Mattias Wadman@hal_pomeranz @0x00string tried fq? i know some ppl have used it for forensic related things
@wader @0x00string Interesting. I’ve not had a use case for that, but it looks cool.
@hal_pomeranz @0x00string btw one of my favorite things in jq that might be useful for you is construct/destruct shorthands, ex:
.. | {a, $b} is same as {a: .a, b: $b}
.. | . as {$a} is same as {a: $a}
destruct also works with arrays/nested:
.. | . as [$first, {b: $second_inside_object}]
practical example:
$ jq -c '{_HOSTNAME, MESSAGE}' journal.json
.. | {a, $b} is same as {a: .a, b: $b}
.. | . as {$a} is same as {a: $a}
destruct also works with arrays/nested:
.. | . as [$first, {b: $second_inside_object}]
practical example:
$ jq -c '{_HOSTNAME, MESSAGE}' journal.json
viq@wader @hal_pomeranz @0x00string could I ask for a link? I'm failing to find it.
Edit: "fq jq" finds https://github.com/wader/fq
Edit: "fq jq" finds https://github.com/wader/fq
Sam@0x00string @hal_pomeranz okay we are now nemeses
@0f4d0335 @hal_pomeranz jq is enough for both of us
cR0w 
@hal_pomeranz I don't use jq a lot, but when I do, it's absolutely the right tool. Such a cool tool.
Fennix@fennix @hal_pomeranz Does that work with json?
@fennix @hal_pomeranz Ah. I get it now. I should check it out though. I still use sed, awk, tr, cut, grep, etc. for csv files. And the stupid tsv files some tools export to with no other format options.
If you do have to work with CSVs it's quite good.
@hal_pomeranz @fennix Absolutely. But for some reason I prefer fighting that to fighting tabs. I blame hardheadedness and inertia.
@cR0w I’ve been dealing with a lot of JSON data lately and learning a lot about jq. It’s a different mindset than other tools I use and it took a while to get comfortable with it.
@hal_pomeranz I don't use it enough to be comfortable with it, but I have used it enough to have enough examples in my notes to no longer need the help or man page most of the time.
Jess👾@hal_pomeranz Also gotta call out
yq. For when you need to parse yaml@hal_pomeranz and xml and csv and hcl and ...
maaneeack@hal_pomeranz I used jq to fix an issue at my previous job, after seeing errors about it and reinstalling jq.
@maaneeack That’s very meta. Well done!