bash parameter expansion

Permalink: https://wizardzines.com/comics/parameter-expansion/

wizard zines

wizard zines
@b0rk_reruns One of my favorite idioms is:
: ${VAR:=default-value}
which sets VAR to default-value if it's not already set, and then does nothing (":").
#sh #hacks #unix #Linux

@arensb @b0rk_reruns This reminds me of the plan9 shell rc’s default prompt:
;
which nicely nops when copy pasting multiple lines. And I think recently I stole a variant from @cross:
: user@hostname/cwd;

Instead of the common:
user@hostname:cwd$

@josephholsten But then doesn't that mean you don't find zero-length files called "ls" and "pwd" all over your directory? 🙂
@b0rk_reruns Something I found not entirely obvious from the #bash manual: In ${var:-...} and ${var:?...}, the colon is not required. Its presence indicates that empty variables (i.e. variables containing an empty string) are to be treated like unset variables. If you write ${var-...} or ${var?...}, the default value or error message only kicks in if the variable is genuinely unset, not just empty.

For the prefix/suffix removal, you can choose between removing the shortest or longest matching pattern by using %/# vs. %%/##:

$ var="Hello World"
$ echo ${var%l*}
Hello Wor
$ echo ${var%%l*}
He
$ echo ${var#*l}
lo World
$ echo ${var##*l}
d

@b0rk_reruns

@b0rk_reruns the bash hackers wiki had (has) a great page about it but it seems to be gone for some reason, though it's still on IA: https://web.archive.org/web/20230408142504/https://wiki.bash-hackers.org/syntax/pe
Parameter expansion [Bash Hackers Wiki]

@b0rk_reruns This is awesome. I knew almost none of that and have been using various grep/sed commands in scripts for this sort of thing for years
@b0rk_reruns I feel like day I will be able to use these stuff without looking on internet is the day I will really understand what being adult means...
@b0rk_reruns I started running shellcheck on my scripts and frens are like why? my reply is it shows me new things in bash I was not aware of. Win! thanks Julia

@b0rk_reruns @b0rk Does anyone have any good way of remembering what each of these syntaxes do? Each time I use them, I have to look it up in the docs to make sure I'm using the right thing.

How do you remember which of ${var#pattern} or ${var%pattern} removes a prefix or a suffix?

I have such a hard time remembering I look it up each time. Anyone have some tricks for remembering?

@unlambda That always trips me up too. The way I remember it is that percent is the modulo division operator in C-style languages. So 10%3 gives the "remainder" of 1 and somehow in my head, that translated as chipping off the end of the string.🤷 Now that I am writing this down, it makes no sense, though.
@unlambda @b0rk_reruns @b0rk I don't try to remember, I keep https://devhints.io/bash on speed dial
Bash scripting cheatsheet

Variables · Functions · Interpolation · Brace expansions · Loops · Conditional execution · Command substitution · One-page guide to Bash scripting

Devhints.io cheatsheets
@alex @unlambda @b0rk_reruns @b0rk I also do not even try to remember. I have been using this for about a decade, but it's not a daily thing, so I look it up every time. I don't think we should feel bad about that.
@unlambda @b0rk_reruns @b0rk I remember it because # is before % on the top row of my keyboard (US QWERTY, # is shift+3, % is shift+5). This reminds me that # is prefix (before) and % is suffix (after).
@unlambda @b0rk_reruns @b0rk I feel like removing a suffix is so much more useful in practice that you can just remember that one

@unlambda @b0rk_reruns @b0rk If it helps I work with a lot of these regularly and have to look it up every time.

Only when I’ve used it that day but not with too many in between can I get away with going from memory.

@Jackson @b0rk_reruns Glad to see I'm not the only one!
@unlambda @b0rk_reruns @b0rk
I try '#', find that it doesn't work, try '%' instead, realize that it's not matching what I need it to, try '%%' to make a more aggressive match, realize that I just removed the part I was trying to keep, finally settle on '##', test it, see that it takes too much off and then in the end use '#'.

@unlambda @b0rk_reruns @b0rk my trick is to switch to another language when i start needing this kind of things 😆. You can do a lot in bash, but the syntax is just too cryptic to do anything complex. the default value is an exception, it often comes handy in simple scripts.

about docs:
«[I do not] carry such information in my mind since it is readily available in books. ...The value of a college education is not the learning of many facts but the training of the mind to think.» Albert Einstein.

@unlambda @b0rk @b0rk_reruns I assume the intended mnemonic is that # goes in front of a number (e.g., #1, #2, …) and % goes behind a number (e.g., 50%).

@unlambda @b0rk_reruns @b0rk # is before $ on a keyboard, so it’s for prefix. % is after — so suffix

At least that’s how I remember

@unlambda on a US keyboard, the # is over the 3 and the % is over the 5, which means # is on the left (toward the beginning of the word) and % is on the right (toward the end of the word)
@unlambda @b0rk_reruns @b0rk On a standard US English keyboard, the symbols appear on 3,4,5 in the order #,$,% - # comes before the $ so it cuts off the start, while % comes after the $ so it cuts off the end.
@b0rk_reruns I like that you can do basename and dirname and even build something like find to walk the directory tree. If you have a directory on a NetApp filer that has too many files in one directory, everything will be slow. What you do not want is that your tree walk will do a couple of operations like lstat on every file (hello find/basename/dirname).
basename: ${path##*/}
dirname: ${path%/*}
#TIL about ${somevar:?error message}. Thanks @b0rk !
@b0rk_reruns sometimes #bash feels like magic. The syntax is imo very impenetrable but it is just so powerful

@b0rk_reruns

this says “null” to mean “empty”?

by contrast, Python’s os.environ.get(“name”) distinguishes “” for empty and None for unset

ditto TypeScript, except “null” is their spelling of None

p.s. thanks especially for so strongly surfacing the x// vs x/ distinction for replacing all or just one, i had forgotten that syntax till you came to remind us here

@b0rk_reruns @b0rk daaaang this is so awesome. I thought I was the bee's knees once I knew about ${varname:-otherValueIfBlank}, I'm gonna have to play around with all these others

@b0rk_reruns

I LOVE THIS

just used to glob my latest generated files and create a directory from it

```
TRAINING_DATA=$(ls -lt training_data* | rev | head -n1 | cut -d' ' -f 1 | rev) \
OUTPUT_DIR="${TRAINING_DATA%.*}" \

mkdir models_$OUTPUT_DIR
```