@thebaldgeek @Fallenstedt in this format: "paths", but presumably (oh I hope) they were keys at some point (key, with nested data, that has keys, that has nested d...).

Any chance to take a look at the original source of this list of paths? It's wild on it's own, but I can't be of much help with JUST the paths. Are you getting data from a public API or data dump?

@rockerest @Fallenstedt
Thanks so much for the feedback, it really helps having some external input, I am so in the forest, I cant see the birds!
https://github.com/szpajder/dumpvdl2
Im running dumpvdl2 and just keeping track of all the aircraft messages I hear from it.

You have got me thinking that the ACARS library it uses might be the 'bad guy':
https://github.com/szpajder/libacars

dumpvdl, dumphfdl and Jaero all use the same libacars and all have those crazy paths.
I am logging all JSON of all of them asap.

@thebaldgeek @Fallenstedt ‼️this is an area of software/data that I've never even glanced at!

That library looks extremely complex!

It seems like maybe (??) it reads data from an implicit source (like hardware, maybe?) and then outputs it into JSON.

Do you have a sample of the output JSON like from 1 minute or 1 hour of data? I'm not sure how to phrase this question since I'm not familiar with aviation data at all!

@rockerest @Fallenstedt
You got it! The antenna picks up the RF from the aircraft and feeds it to the #SDR (30 dollar software defined radio ie, USB dongle) and then the #raspberrypi etc runs #dumpvdl and you get #JSON out.
The dumpvdl config contains the USB device number, the 4-6 or so frequencies to listen on etc.
Here is a nice short sample of the JSON direct from the decoder (ie, it has both name:value).

https://github.com/thebaldgeek/thebaldgeek.github.io/blob/main/json/dumpvdlJSON.json

Thanks again for sharing your thoughts!

@thebaldgeek @Fallenstedt This is interesting!

I massaged the data a bit to be able to render it more easily: https://codepen.io/rockerest/pen/ZERxEQL?editors=0010

The data source is over here: https://codepen.io/rockerest/pen/LYrdYGm?editors=0010 where I wrapped the entries in an array, added commas between them, etc.

I can see why this data is so frustrating!

From my (totally inexperienced) perspective, it looks like each entry may be one of THREE different data sources: ACARS, XID, or X25.

Even within these sources, there isn't really consistency

@thebaldgeek @Fallenstedt

I rendered "everything" (just a sample of data points) in the first table, and then filtered down to just ACARS for the second, just XID for the third, and just X25 for the fourth.

If you're only interested in ACARS data, it might help a lot to do something similar, since the XID and X25 entries are both very different.

High chance I don't know what I'm talking about here, though.

@thebaldgeek @Fallenstedt Notably, the XID entries are annoying, because they contain pub_params and vdl_params, which are "just" normal dictionaries, but with the key:value pairs as individual objects in an array.

In the first Codepen I linked above, I mapped these back to dictionaries, but obviously we can't be certain about any keys even existing, since they're just delivered as lists of potential pairs.

@thebaldgeek @Fallenstedt

I'm not even sure how to help you here!

My first approach would be to try to slim down what I even ingest.

e.g. if I ONLY want ACARS data, I'd get the XID and X25 data filtered out immediately, since it's just muddying the water. But if you need more than one of the different types, things are certainly going to get tricky.