Mastodawn

Simon Tatham May 15

A habit I'm thinking of adopting for my shell scripting:

Write scripts in the form of a sequence of function definitions, with no commands at the top level that actually do anything. Then, at the very end, write a compound statement that calls one of the functions and ends with an exit. Like this:

#!/bin/bash
subroutine() { ... }
main() { ... }
{ main "$@"; exit $?; }

The idea is that the shell doesn't actually _do_ anything while reading this script until it reads the braced compound statement at the end. And then it's committed to exiting during the execution of that statement. So it won't ever read from the script file again.

With a normal shell script, it's dangerous to edit the script file while an instance of the shell is still running it, because the shell will read from the modified version of the file starting at the file position it had got to in the old one, perhaps reading a partial command or the wrong command and doing something you didn't want. But with a script in this style, the shell finishes reading the script before it does anything, so it's safe to edit.

(Of course, if your editor saves the new script file to a different file name and renames it over the top, you're safe anyway. But not all editors do: emacs, in particular, reopens the existing file and overwrites its contents.)

@simontatham I would never use this... It requires that you avoid ctrl-c. Why? And if you are testing a script why is it in the background anyway?

If you are so scared of your program crashing and running in a type of zombie state, why use nested functions at all, that almost guarantees bad behaviour. Furthermore, don't normal people use 'set -e'?

Idk maybe I'm just overthinking it or just don't understand..

Simon Tatham May 15

@ponies I'm completely confused by everything you just said!

What's the problem you've spotted with Ctrl-C?

Who said the script was running in the background? More usually I'm running the script in one terminal window, and editing it in another, or in a GUI editor.

What nested functions are you talking about? I didn't show any.

How does 'set -e' help? That's solving a completely different problem.

@simontatham well my point is that stopping and starting the program again makes your issue irrelevant, right?

Simon Tatham May 15

@ponies yes, certainly. If you always stop the script running, then edit it, then start it again, you don't need this protection.

But it's _useful_ to be able to edit a program so that the next run will work better, while the previous run is still going. It means you can fix each problem as you see it, instead of having to remember what they all were once the program finishes running, maybe hours later.

In almost any other language – interpreted _or_ compiled – you can do this safely. It's only in shell where editing a running script causes a disaster. Why _wouldn't_ someone want to eliminate that exception, and make shell scripts behave the same as everything else?

@simontatham This is dumb, even with no burden of implementing it.

So yes bash may have this problem, find me an editor that actually behaves this way. Basically, you have to contrive some crackpot scenario to have this eventuate.

Not even with sed -i "s/code/replacement/" can I trigger this.

Yes I note your emphasis on "useful". Indeed, it's not useful at all and furthermore I have to add functions which simply do not need to be there in the first place

Simon Tatham May 15

@ponies I already mentioned an editor that behaves this way – emacs.

There's no weird edge case involved. You have a script wrapping some subcommand that takes a long time and you don't want to ^C it, like a backup. You run the script. You notice it could do something better. You edit the script, in emacs, now, while you can still remember what it was you wanted to change.

Then the backup (or whatever) finishes, bash reads more data from the script file, and gets confused by the fact that you edited it.

@simontatham no, I cannot do this in Emacs either.

Can you actually reproduce this at all? Please tell me how you encounter this

And if you can't.. why change your code style for a non issue 🧐

Simon Tatham May 15

@ponies certainly I can reproduce it – I checked it this morning before making the original post. And I've reproduced it again just now. Here's a demo video.

If you can't reproduce this in emacs, that's very interesting. Perhaps if you stopped trying to insult me we might investigate *why* your emacs is behaving differently from mine? Does your emacs write a new file and rename it into place?

(You could check by examining the inode number of the file before and after saving over it, or by watching in strace what it's actually doing during the save action. In mine, strace shows that it opens the existing file name with O_CREAT|O_TRUNC|O_WRONLY, and before and after saving, 'ls -li' reports the same inode number.)

@simontatham so to make the above example work you have to specifically tell your editor to keep the same inode. I can see the Emacs setting which by default is set to nil (gnu manual). On my distro it is also nil.

If you are changing settings and then crying wolf.. it's kind of a lie isn't it? I can break stuff too you know

Simon Tatham May 15

@ponies I didn't deliberately configure emacs to keep the same inode. I don't know why you think I did.

I did configure 'make-backup-files' to nil, to avoid the clutter of all those extra foo~ files. It looks as if that's what causes it: emacs in its default configuration renames the old file to the backup name and then makes a fresh
inode for the new file, but if you turn off make-backup-files, it just saves over the original file with no renaming.

But it's not unreasonable to turn off backup files. And emacs _could_ still make a new inode and atomically replace the old one, even if it's not making a backup file. It just doesn't.

What setting are you talking about, which defaults to nil? I haven't been able to find it. You apparently know its name but preferred to insult me again rather than actually saying what the name of the setting is?

@simontatham look I'm following and bookmarking you so 👍

I have also figured out how to make bash do this without Emacs. Ok it's real.

I'm just saying the idea of somebody running code and editing it at the same time (despite your use case) is for the most part a fiction, it requires

1. Editing configuration away from defaults, or
2. Overwriting the file from within the script whilst it's running

Edit: some possible setting, maybe
http://www.gnu.org/s/emacs/manual/html_node/emacs/Backup-Copying.html

Backup Copying (GNU Emacs Manual)

Backup Copying (GNU Emacs Manual)

Alfred M. Szmidt May 15

@ponies >I'm just saying the idea of somebody running code and editing it at the same time (despite your use case) is for the most part a fiction, it requires [...]

It is not fiction, and a very common (and known case) case. You have a semi-long running shell script, you run it, and then see something to improve -- edit, and boom. This can lead to real crap too, depending on how far bash has executed the file (not parsed!).

I'm not sure that @simontatham way is solves the problem.

@amszmidt @simontatham The whole discussion was about the real possibility that this will NEVER HAPPEN if somebody did what you just said, @amszmidt.

Editors don't behave this way unless you make them behave that way.

Alfred M. Szmidt May 15

@ponies Sorry, bullshit. This has happened to me during this single year multiple times. And this is with default settings on editors, and in this case #GNU Emacs.

@amszmidt @simontatham gnu documentation says otherwise.

All I ask is - what distro and which Emacs

Alfred M. Szmidt May 15

@ponies Nonsense. It is unrelated to GNU/Linux system, or even Unix. It is also unrelated to #GNU #Emacs. It is all about how the standard input/output/error streams buffer data.

Alfred M. Szmidt May 15

@ponies The solution @simontatham is/was presenting, could possibly (untested) be simplified by putting { and } around all the code. It forces #GNU #bash to read the whole block (storing it in memory, so buffered reading of the file won't be an issue -- in theory) before executing it.

@amszmidt @simontatham the only nonsense here is you ignoring what I'm actually saying -- 👌😆

@amszmidt @ponies @simontatham
Speaking as someone who's been dealing with bash for a *very long time* in occasionally weird situations, I have to weigh in here: your particular use case is not common.
It's *your* use case, though, and it looks to me like you could save yourself a lot of trouble by simply getting in the habit of not editing bash scripts while you run them.

That's an interesting workaround you have come up with, though I don't know how you're planning to exit main with an error code to pass to $? without actually exiting the script.

I'm not meaning to pry much, but what on earth are you doing that does not benefit from a restart with correct code?

Simon Tatham May 15

@http "simply get in the habit of not editing scripts while you run them" sounds to me a lot like "simply get in the habit of never having an accident". If that were easy, then a great many safety precautions in this world wouldn't be needed. And yet, here we are.

How to exit main with an error code in $?: if you've been doing bash for that long then I'd expect that would be obvious. First, the result of the last subprocess or other command is left in $?, so whatever you ran last _inside_ main will be the value of $? when main returns to the calling block. Second, you can also use 'return' in a shell function to set the $? seen by its caller. (But of course if you do that in main, you might as well run 'exit' directly; _that_ isn't the case where the fallback exit outside main is needed.)

What doesn't benefit from being restarted: something you don't want to interrupt! For example, a shell script wrapping a long-running thing which you'd have to start all over again if you interrupt it. If the thing you want to correct isn't critical, like "oops it would have been better to run the subcommand in verbose mode, but at least it'll do the right thing this time round even if it does it quietly", then it makes perfect sense to want to change the script for _next_ time, without interrupting _this_ run.

Yes, of course you _could_ wait until it finishes, and then remember to come back and make your change. But doing things now is a good way to avoid forgetting to do them later. In any other language you can just do it. And with this precaution, you can do it in shell too.

@simontatham @http

The result of $? Is never bloody obvious and I agree that you would have no reliable way of knowing what result is being returned there as your code becomes more complex

@simontatham @http the important question is: when do you watch YouTube if you can't stop coding while a long test suite is running?
XKCD 303!

@http @amszmidt @simontatham
🤩🙏

Alfred M. Szmidt May 16

@http > Speaking as someone who's been dealing with bash for a *very long time* in occasionally weird situations, I have to weigh in here: your particular use case is not common.

And speaking as soneone who has used bash since it existed, since before bash, yeah it is bloody common. Maybe stop telling people what their experience is. It is also unrelated to bash and applies to all shells that do buffered reading, go troll soneone else.

@ponies @simontatham

@ponies @simontatham It doesn't require changing editor configuration. vim.tiny, for example, writes into the original file by default (and I'd guess the full version of vim does the same, but I don't have that installed). I've been bitten by this problem enough times that I came up with a similar solution to Simon's independently. If the way you use your machines doesn't cause the problem, maybe you didn't need this solution - but that doesn't mean it's never a problem for others.

Adam Sjøgren May 15

It's not fiction for me, it happens regularly and I have to think about it when working on long running shell scripts.

It is as simple as automating something that processes a lot of files in a loop, and then, while running it, thinking "ah, I botched that output, it would be nicer if it also echo'ed this other thing the next time I run the script", adding a line, saving, and then boom the script goes haywire after the loop.

Plenty of editors overwrite the file when saving:

$ ls -li test.sh 
636 -rw-rw-r-- 1 asjo asjo 34 2025-05-15 21:28:51 test.sh
$ vi test.sh
$ ls -li test.sh 
636 -rw-rw-r-- 1 asjo asjo 24 2025-05-15 21:29:30 test.sh
$ jove test.sh
$ ls -li test.sh 
636 -rw-rw-r-- 1 asjo asjo 25 2025-05-15 21:30:02 test.sh
$ nano test.sh
$ ls -li test.sh 
636 -rw-rw-r-- 1 asjo asjo 28 2025-05-15 21:30:42 test.sh

Even vanilla, unconfigured Emacs:

$ emacs -Q -nw test.sh
$ ls -li test.sh 
636 -rwxrwxr-x 1 asjo asjo 30 2025-05-15 21:31:57 test.sh