Show threadJohn GoerzenOct 18, 2023
@liiwi I already know and am a fan of #Rust! My #Filespooler program, for instance, is written in Rust: https://www.complete.org/filespooler/
Filespooler

What is Filespooler? Filespooler lets you request the remote execution of programs, including stdin and environment. It can use tools such as S3, Dropbox, Syncthing, NNCP, ssh, UUCP, USB drives, CDs, etc. as transport; basically, a filesystem is the network for Filespooler. Filespooler is particularly suited to distributed and Asynchronous Communication. Filespooler is a tool in the Unix tradition of “do one thing and do it well.” It is designed to integrate nicely with decoders (to handle compressed or Encrypted packets, for instance).

www.complete.org
Show threadJohn GoerzenOct 13, 2023

@minoru Thanks for sharing that info!

It's practical because Syncthing's very nice distributed nature lets it be an underlying communications layer for a lot of other things. For instance, I talk about using #Filespooler over #Syncthing here: https://www.complete.org/using-filespooler-over-syncthing/ , and #NNCP over Syncthing here: https://changelog.complete.org/archives/10219-a-simple-delay-tolerant-offline-capable-mesh-network-with-syncthing-optional-nncp

That you can have a "roaming" device (eg, a phone) nicely sync between two locations that otherwise don't have connectivity to each other is pretty slick.

Using Filespooler over Syncthing

Filespooler is a way to execute commands in strict order on a remote machine, and its communication method is by files. This is a perfect mix for Syncthing (and others, but this page is about Filespooler and Syncthing). This page also functions as a tutorial for Filespooler. While I talk about Syncthing in particular here, the instructions here apply to pretty much any directory-synchronization tool, such as Dropbox, Box, etc. All that’s required is that it meet the requirements laid out in Guidelines for Writing To Filespooler Queues Without Using Filespooler.

www.complete.org
John GoerzenSep 30, 2023

@alienghic @schlink Yes! This is a perfect use case for #NNCP. As @craigmaloney mentioned, #Syncthing can also do the job; if you name your files according to a set pattern, you can just add them to a directory on the sending side, and mv/rm them out of there on the receiving side when processed.

My #Filespooler tool https://www.complete.org/filespooler/ can provide some more queue-processing tools, esp. if you need the queue to be processed in order. It can run atop NNCP, Syncthing, or others.

Filespooler

What is Filespooler? Filespooler lets you request the remote execution of programs, including stdin and environment. It can use tools such as S3, Dropbox, Syncthing, NNCP, ssh, UUCP, USB drives, CDs, etc. as transport; basically, a filesystem is the network for Filespooler. Filespooler is particularly suited to distributed and Asynchronous Communication. Filespooler is a tool in the Unix tradition of “do one thing and do it well.” It is designed to integrate nicely with decoders (to handle compressed or Encrypted packets, for instance).

www.complete.org
John GoerzenDec 8, 2022

I wrote "Building an #Asynchronous, Internet-Optional Instant Messaging System" today. It suggests using #DeltaChat (@delta) across an #email system that can run on non-IP networks to achieve #IM coverage without having to have an Internet backbone, using things like #NNCP, #Syncthing, #Filespooler, and such. I love the ways we can combine interesting tools like this!

https://changelog.complete.org/archives/10435-building-an-asynchronous-internet-optional-instant-messaging-system

Building an Asynchronous, Internet-Optional Instant Messaging System

I loaded up this title with buzzwords. The basic idea is that IM systems shouldn’t have to only use the Internet. Why not let them be carried across LoRa radios, USB sticks, local Wifi networ…

The Changelog
John GoerzenNov 23, 2022

@downey I am on board with the movement to #GiveUpGithub! I have already transitioned many (though not all of my older, semi-abandoned) repos to #Debian's #salsa service, which is based on #FLOSS #Gitlab and not limited to Debian projects. @Codeberg is another common choice and I would heartily recommend both.

My most recent #FreeSoftware project, #filespooler, has never had a Github presence of any kind. https://salsa.debian.org/jgoerzen/filespooler

John Goerzen / Filespooler · GitLab

CLI & Library for Sequential, Distributed, POSIX-style job queue processing

GitLab
Show threadJohn GoerzenNov 1, 2022

3/

Introducing Fast, Ordered Unixy #Queues over #NNCP and #Syncthing with #Filespooler: https://floss.social/@jgoerzen/108387639329271399

"#Modem World" and modern communities that span interests: https://floss.social/@jgoerzen/108546787833118936

Why do I feel drawn to work from my #DEC #vt510 #terminal? Does it have advantages over my 27" LCD GUI? https://floss.social/@jgoerzen/108898543787831520

Growing up through the #PC and #Internet revolution in #rural America: https://floss.social/@jgoerzen/108909211718813036

John Goerzen (@[email protected])

I am happy to finally introduce Fast, Ordered Unixy Queues over NNCP and Syncthing with Filespooler - see my blog post here: https://changelog.complete.org/archives/10368-fast-ordered-unixy-queues-over-nncp-and-syncthing-with-filespooler #asynchronous #queue handling over #Syncthing, #NNCP, #S3, or many others. Written in #Rust.

FLOSS.social
John GoerzenOct 31, 2022
@deadparrot Hello! I'm a #rust/#rustlang person. Been using it at work for several years, and also have several crates out there. #Filespooler https://www.complete.org/filespooler/ is my most recent project. I'm also behind #xbnet and #lorapipe, which can tunnel TCP/IP or other data down #XBee SX or #LoRA long-distance radios.
Filespooler

What is Filespooler? Filespooler lets you request the remote execution of programs, including stdin and environment. It can use tools such as S3, Dropbox, Syncthing, NNCP, ssh, UUCP, USB drives, CDs, etc. as transport; basically, a filesystem is the network for Filespooler. Filespooler is particularly suited to distributed and Asynchronous Communication. Filespooler is a tool in the Unix tradition of “do one thing and do it well.” It is designed to integrate nicely with decoders (to handle compressed or Encrypted packets, for instance).

www.complete.org
John GoerzenJun 20, 2022
So that weird #pipe issue I saw with #Filespooler and used #strace to diagnose? Probably a #kernel #bug. https://changelog.complete.org/archives/10390-pipe-issue-likely-a-kernel-bug
Pipe Issue Likely a Kernel Bug

Saturday, I wrote in Pipes, deadlocks, and strace annoyingly fixing them about an issue where a certain pipeline seems to have a deadlock. I described tracing it into kernel code. Indeed, it appear…

The Changelog
Show threadJohn GoerzenJun 10, 2022

@gwil Welcome, gwil! Glad to see you here. It looks like your interests intersect mine quite a bit; I've been interested in #NNCP and #Syncthing as decentralized, offline-first or offline-capable systems, and recently wrote #Filespooler in that space.

I hadn't heard of #Earthstar. After reading the website, it sounds kind of like Syncthing, but with a more-public way of joining a share? (Sharing a link without having the approve the request) I'd be interested in other comparisons.

John GoerzenMay 31, 2022
Ever wanted to pass things like environment variables to commands in #ssh? #Filespooler without queues can help! Its metadata is perfect for this. https://www.complete.org/using-filespooler-without-queues-to-pass-more-metadata/
Using Filespooler Without Queues to Pass More Metadata

One frustration people sometimes have with ssh or NNCP is that they’d like to pass along a lot of metadata to the receiving end. Both ssh and nncp-exec allow you to pass along command-line parameters, but neither of them permit passing along more than that. What if you have a whole host of data to pass? Maybe a dozen things, some of them optional? It would be very nice if you could pass along the environment.

www.complete.org