Jeff KaufmanNov 9, 2024

In my bioinformatics work I often stream files between linux hosts and Amazon S3. This could look like:

$ scp host:/path/to/file /dev/stdout | \
aws s3 cp - s3://bucket/path/to/file
This recently stopped working after upgrading:

ftruncate "/dev/stdout": Invalid argument
Couldn't write to "/dev/stdout": Illegal seek

Looks like this is scp switching from the SCP protocol to the SFTP one: https://www.jefftk.com/p/force-sequential-output-with-scp

Force Sequential Output with SCP?

In my bioinformatics work I often stream files between linux hosts and Amazon S3. This could look like: $ scp host:/path/to/file /dev/stdout | \ aws s3 cp - s3://bucket/path/to/file This recently stopped working after upgrading: ftruncate "/dev/stdout": Invalid argument Couldn't write to "/dev/stdout": Illegal seek I think I figured out why this is happening: New versions of scp us

Show threadJoachim Schipper
@jefftk FWIW, out of idle interest: I assume you've already considered workarounds like "ssh host cat /path/to/file | <...>"? Changing workflow is always annoying, but is there anything in particular that didn't work for you?
Nov 9, 2024 at 3:41pmWeb
Show threadJeff KaufmanNov 9, 2024
@JoachimSchipper good question! In this case it's a host that allows scp and sftp but not ssh.