Ran into two compatibility issues with my Shell scripts between different Ubuntu versions: dpkg-deb changed its default compression and pigz changed where it sends --version info (used to be stderr, then stdout).

All this happened years ago, but OS version upgrades at the enterprise level are hard, so it's good to be aware of differences (like that the non-GNU AWK in Ubuntu Xenial is a mawk version that requires quoting a slash in bracket expressions.)

#shell #linux #ubuntu #debian #awk

40+ years later, and AWK is still a powerhouse! 💻 Brian Kernighan, the co-creator of AWK, explains why it remains essential for text processing.

Original Video: AWK Is Still Very Useful | Brian Kernighan and Lex Fridman

#linux #awk

A good day is when I learn something new. Today I learned a small thing in AWK for IF statements. It makes the life of the team easier now as I can allow for multiple year checks on an input file. This coming December I can set it up to handle 2026/2027 dates without causing an error message like we received today. I knew how to do it in APL, BASIC, C, dBase, PASCAL, PL/1, COBOL, FORTRAN, REXX, RUST, SQL and BASH. Never did it in AWK.

#awk

printf "%s\n" 'e !sort -u %' wq | ed -s skip; w3m -dump 'https://cdn.media.ccc.de/congress/2025/webm-hd/' | awk '/\[[0-9]+\] https:.*webm-hd.*\.webm$/{print $2}' | sort | comm -23 - skip | tee urls | while read -r u; do echo $u; grep -Fq "$u" skip && continue; wget -c "$u" && echo $u >> skip; done

Downloading recorded talks from #39c3 using #ed #w3m #awk and other #unix tooling <3

Thought about replacing #wget with #curl but old dogs and such...

Dumb #awk one-liner:

I wanted to see which general documentation man-pages (section 7) my FreeBSD system had in the base, but there were a number of them that were identical/hard-linked so I didn't want to see those duplicates. Easy enough:

$ ls -i /usr/share/man/man7/ | awk '!a[$1]++{print $2}' | column

(piped though column(1) so they wouldn't scroll off the screen)

As a Linux user and tech professional I have to confess I've never learnt to use #awk... Whenever needed, I just find a #bash workaround using #sed (etc.) commands.

I know I'm not perfect, but I am looking for a bit of external validation (on this matter) - surely I'm not the only one like this..?

Oh, and I also believe I compensate for this with my advanced RegEx skills.

OK here we go, #AdventOfCode day 2 with a one-liner in a different cursed language from yesterday (that was Matlab).

My plan for part 1 is to split on dashes, loop through the range, and add up numbers that match the pattern. #Awk is extremely DWIM about a value being both a number and a string so it works fine:

```
tr , '\n' <input2.txt | awk -F- 'BEGIN { sum=0 } { for(i=$1;i<=$2;i++) if(substr(i,0,length(i)/2) == substr(i,length(i)/2+1)) sum+=i } END { print sum }'
```

Part 2 is quite a bit harder, we need to allow for any number of repetitions within each ID. Awk is definitely the wrong language, but you could have told me that before. The loops nest quite a bit higher; inside of looping through the ID range, we have to loop through the possible substring lengths, and if it evenly divides the ID, go through the subsequences and make sure they are all the same. It is, *technically*, still one line:

```
tr , '\n' <input2.txt | awk -F- 'BEGIN { sum=0 } { for(id=$1;id<=$2;id++) for(sl=1;sl<length(id);sl++) if(length(id)%sl==0 && length(id)/sl>1) { ok=1; for(i=1;i<length(id)/sl;++i) ok = ok && substr(id,0,sl) == substr(id,i*sl+1,sl); if(ok) { sum+=id; break } } } END { print sum }'
```

You know when you create a dumb little tool because the grunt-work annoys you when doing a particular task, then that task drops in your lap, you whip out that tool, and suddenly it's fun again because you don't have to do the boring grunt work? Today was one of those days, using this little #awk script I threw together:

https://www.reddit.com/r/SQL/comments/1lpjcp9/dumb_awk1_script_for_making_create_table_and/

"Here's some tabular data with headers, how would I make a SQL query against it?" To do that I need to turn that tabular data into CREATE TABLE and INSERT statements to set up a mock version of the data so I can write queries against it. It's tedious even with a powerful $EDITOR. But turn it into an awk script and, with the data on the clipboard, I can just

$ xsel | create_table.awk | xsel -ib

and paste into something like db-fiddle.com, and it's mostly ready to go.