rwxDec 19, 2022
Jason Scott

So, I have been turning my attention to the idea of buffering 24 hour grabs of all radio spectrum, using Web SDR. I'm just reaching out in case people are sitting on sets of these from times past. DM or [email protected].

Am I really to the point of archiving radiation? Yep.

The big concern is disk space. Under current approaches, a full spectrum grab of 24 hours of signal in one geographical location is ten terabyes! But one day a year might be worth it. For history.

Dec 18, 2022 at 3:55pmWeb
Show threadJason ScottDec 18, 2022
If people don't know, we already grab 11,000 hours of radio a day worldwide, but that's not the same as radio spectrum.
Show threadEamonnMRDec 18, 2022
@textfiles ohh, where do I find that/is there a way to search by station-year? I've been (passively) trying to track down a particular rendition of Here Comes The Sun that was used frequently as a bumper on WNPR around 2007-2010.
Show threadzpercussionDec 21, 2022
@textfiles
Is this all under a specific collection?
Show threadJason ScottDec 21, 2022
@zpercussion it's all over
Show threadChris LDec 18, 2022
@textfiles a logical extension to internet archives mission. And really 10T / day is peanuts for such unique data. Maybe get some grant for it.
Show threadAlexandre DulaunoyDec 18, 2022
@textfiles I love the idea. Maybe recording samples of one hour every month and storing the complete speech2text would be more useful (especially allowing indexing and searching).
Show threadJason ScottDec 18, 2022
@a I've begun discussing things with people who know better to see if compression is possible.
Show threadkepstinDec 18, 2022
@textfiles @a it would be pretty hilarious if it ends up that you can just use, say, FLAC and get ok compression.
Show threadJason Nabein (Old Account!!!)Dec 18, 2022

@kepstin @textfiles @a You actually can use FLAC to some extent. I highly recommend reaching out to the people working with ld-decode and vhs-decode, as that's their primary domain of expertise; RF capture + post-capture compression

https://github.com/happycube/ld-decode
https://github.com/oyvindln/vhs-decode

They have a discord here: https://discord.gg/pVVrrxd

Or join #domesday86 on irc://irc.libera.chat

#preservation #rf #domesday86 #DomesdayDuplicator #archival #flac #datahoarder

GitHub - happycube/ld-decode: Software defined LaserDisc decoder

Software defined LaserDisc decoder. Contribute to happycube/ld-decode development by creating an account on GitHub.

GitHub
Show threadwas B (this is my old account)Dec 18, 2022
@textfiles image description: visualization of radio patterns with options to modify frequency, bandwidth, and other settings.
Show threaddr 🛠️🛰️📡🎧Dec 18, 2022
@textfiles That sounds high. Can you do an adaptive sample rate thing? You don't really need 8Gsps or whatever for LF.
Show threaddr 🛠️🛰️📡🎧Dec 18, 2022
@textfiles You could also save the 10TB annually for full IQ decoding, but do waterfall plots all the time to show traffic.
Show threadplasticbiker  Dec 18, 2022
@textfiles what frequency range and sample rate are you capturing?
Show threadJason ScottDec 18, 2022
@plasticbiker ideally, all
Show threadPolygon Pusher📻Dec 18, 2022

@textfiles How relevant is whitespace? Maybe some automated post process to clip whitespace?

Source -> ML(ID/Clip space) -> disk?

If you could specify KPI of the wave, record only those and specify everything else as non relevant or non eventful. Maybe add a buffer of the head and tail for each event.

Show threadJason ScottDec 18, 2022
@polygonpusher i have not gotten that far, I'm a stranger in these parts, recognizing some potential
Show threadPolygon Pusher📻Dec 18, 2022
@textfiles Understanding your signal to noise I think would be a great starting point. That will at least help you understand what can be ignored and what is of interest. With that you can optimize.
Show threadJames HareDec 18, 2022
@textfiles What opportunities are there, if any, to compress this data?
Show threadRyuno-KiDec 18, 2022
@textfiles
I would imagine that there is a significant amount of white noise in it, isn't it?
Show threadCh00fDec 18, 2022
@textfiles this kind of project could have some astronomical applications too if you can pick up the right frequencies.
Show threadJason ScottDec 18, 2022
@ch00f oh, there's a thousand uses
Show threadChristian Pietsch (old acct.)Dec 18, 2022
@textfiles It might be a good idea to use the #WebSDR tag. Not sure if I like your idea tho. You you know if the NSA is doing this?
Show threadEvan 🫠 🔙 MAGFest 🦔💨🥏Dec 18, 2022
@textfiles What format would this be stored in? Sounds like an interesting signal processing challenge to find a still useful but more compressed format
Show threadTom NardiDec 18, 2022

@textfiles There was a very interesting talk about this concept at HOPE XII:

https://hackaday.com/2018/07/23/hope-xiii-time-travel-with-software-defined-radio/

https://youtu.be/Y8ZVroIIhcw

HOPE XII: Time Travel With Software Defined Radio

It’s easy to dismiss radio as little more than background noise while we drive.  At worst you might even think it’s just another method for advertisers to peddle their wares. But in rea…

Hackaday
Show threadLutruloDec 18, 2022
@textfiles That sounds awesome, but be careful. Some parts are illegal to record in the US, namely cell phone bands, because it is considered wiretapping. Militaries would probably have something to say about it too.
Show threadJason ScottDec 18, 2022
@Lutrulo that's me, careful
Show threadSave The Day NickDec 18, 2022
@textfiles are you looking for cheap DVB-T TV dongles?
Show threadbobosvDec 18, 2022

@textfiles Would be fun if we could have some special events... but... having that feature will make it hard to not save more... and more!!
I like the idea!!
I would like to have a day when there are some contests, i.e CQ WW cw/ssb https://www.cqww.com/
Maybe a day when the Grimeton transmitter is active
I would have loved to see the spectrum on 24/2 2022.. Or the days before/after

Hmm.. what do you mean by spectrum btw??

CQ World Wide DX Contest - Home

Show threadJigme DatseDec 18, 2022
@textfiles It sounds like a great idea, but as you say, storage is massive. I'm assuming that you can't really reduce that size without losing data?
Show threadJason ScottDec 19, 2022
@JigmeDatse I appear to have woken up some pretty smart people so we are gonna see what we can do.
Show threadJoebloDec 21, 2022
@textfiles It might be worth looking into ZFS file system. It offers high levels of compression for certain file types, and can also do deduplication. But I’m no expert. Only enthusiast. I highly recommend https://arstechnica.com/information-technology/2020/05/zfs-101-understanding-zfs-storage-and-performance/ as a starter point for anyone interested.
ZFS 101—Understanding ZFS storage and performance

Learn to get the most out of your ZFS filesystem in our new series on storage fundamentals.

Ars Technica
Show threadAndrea Franceschini 🏴‍☠️Dec 24, 2022
@textfiles Ok, I'm intrigued, tell me more. Specifically: what, and why. But also other things. I want to know.
Show threadEric PuryearDec 25, 2022

@textfiles That is a very cool idea!

On the plus side, 10TB stored on AWS Glacier deep archive is about $10/month, so a day a year is doable for sure.

Show threadalphabet soupDec 28, 2022
@textfiles Jason, you may have a look at https://github.com/oe2lsp/SDR-Buffer and check out the SDR Buffer instance located in Graz, Styria at the technical university: https://websdr.iks.tugraz.at/sdrbuffer/
GitHub - oe2lsp/SDR-Buffer: spectrum buffering over time, visualisation and playback

spectrum buffering over time, visualisation and playback - GitHub - oe2lsp/SDR-Buffer: spectrum buffering over time, visualisation and playback

GitHub
Show threadbobosvDec 30, 2022
@textfiles
Worth looking in to?? https://www.youtube.com/watch?v=Y8ZVroIIhcw&list=WL&index=13
The Circle of HOPE (2018): Creating a Radio Time Machine: Software Defined Radios

YouTube