As it turns out, that's not quite perfect, but that's a slightly different story...

The story that I want to tell is that for many of you, this is an indecipherable mess. To me right now it's pretty straight forward. To me in a week it will be odd. To me next year it's probably a mostly indecipherable mess!

But also it's ridiculously straightforward to explain what it does.

Go through my file and tell me all of the entries that I've reference but don't have a topic entry.

That's it.

I'm gonna meander on a sidetrack here off of my already sidetrack.

While I was writing entries for all of the topics that were missing entries, I realized that my knowledge is fairly fractal.

Most of the entries I ended out creating new references that didn't have topics! So I ended out just doing my best to leave myself indications that I might want to make some future links or something. So Rather Than Full References I just left the words capitalized like that.

The realization just how fractal my knowledgebase is was pretty interesting. And now back to our regularly scheduled program.

:4^M:r!comm <(sed -E '/- ([A-Z][a-z]+){2,}/,/^$/\!d;s/^- //;/^$/q' journal.txt | sort | uniq) <(egrep '\b([A-Z][a-z]+){2,}\b' -o journal.txt | sort | uniq) -1 -3 | shuf -n1^MI- ^[w*Nvip:sort^Mnyy+w*P2xyypVr=

Is incredibly basic and simple and straight forward but to grok it requires... not necessarily a fractal of knowledge, but knowledge of a lot of things!

Let's start with something that isn't exactly in this script, but is pretty essential to know. And that's the Unix philosophy that we should have programs that do one thing, and do it well, and (hopefully) are well documented.

There's a bit of a related idea that you should be liberal in what you accept, conservative in what you emit, but in our case that's not too terribly relevant.

If you're aware of those two things, then the next thing to know is #regex or Regular Expressions.

I mean, you don't even need to be particular familiar with them, just that they exist. And our regular expression is actually very simple!

egrep -o '\b([A-Z][a-z]+){2,}\b'

If you man egrep you'll know that's the grep that gives you extended regular expressions which honestly are only slightly helpful here. And the `-o` only emits the match. I'm guessing that most of you could even puzzle out what this does, knowing my original description (and if you've read up on my approach to Wiki).

[A-Z] matches any character [A-Z]. [a-z] gives you any lowercase ASCII letter. And the + means that you need one or more. The () groups, and the {2,} means match that two or more times.

So:

FooBar matches
Foo2Bar does not
FOOBar does not
FBar does not

Feel free to vote in this poll:

That's actually all it takes to get all of the WikiWords/PascalCase/CamelCase words out of my journal file.

The next part actually took me quite a while and honestly was pretty shocking, since this entire thing took me like 26 lines including blank lines to do it in Python.

I wanted to find my TableOfContents at the top of the file

---

This doesn't matter

- ButThisIsTheStart
- AndGetThis
- ThisToo

But not this
OrThis
- OrThis
- OrAnyOfThese

---

Food time, and then another sidetrack!

#ed is the standard editor. I'm https://www.gnu.org/fun/jokes/ed-msg.en.html was the actual man page for ed years old.

From what I understand, grep is from the ed `g/re/p` pattern - g for global, re being your regex, and p being print. Fun times!

Well, ed only operated on actual files, but there was a need to do the same thing but for streams. And wouldn't you know it, the #UnixPhilosophy pops up again: let's call the program `sed` for Stream ed!

In my case, I only want to display the first block of matches.

This is a bit of a challenge, since #sed only operates on *lines*. So I can't just say "give me line A to line B". But I *can* say `sed -n '/match A/,/match B/p'` and sed will give me everything from match A to match B and then quit. Sort of.

It will actually give me everything from *every* match A to match B. Which I didn't quite understand, so trying to do `sed -En '/- ([A-Z][a-z]+){2,}/,/^$/p'` ended out giving me every first line that started with `- WikiText` to every blank line. Oops!

The Internet finally helped out here but reading `man sed`, well.. it wouldn't have helped much.

"d Delete pattern space. Start next cycle"

So what I actually am doing with #sed is deleting everything *not* in the pattern space. That leaves me with only the table of contents. But cool enough, I can also go ahead and strip off the `- `, since I'm here. Finally, I just want until `match B` and then q(uit).

Breaking down my #sed command:

sed -E # give me extended regex
/- ([A-Z][a-z]+){2,}/,/^$/!d; # find everything starting at a '- WikiCase' ending at a blank line. Delete everything else
s/^- //; # remove the '- ' from the start of line
/^$/q # quit at the first blank line

Oh! I just remembered another sidetrack. sed uses the same "keybindings" as ed. So `s/^- //` would remove the starting hyphen from the current line in #ed

Now remember - so far all of this is just to get two lists - my TableOfContents and all of the WikiWords that exist in my file.

If you use #Linux or #Unix for any period of time (or #GnuLinux - where my pedants at?), you will probably have come across `sort`.

Now we get to talk about streams and files!

Everything in Linux is a file. Which is shockingly powerful. Input from the keyboard? A file you can read from. Output to the screen? A file you can write to. #stdin and #stdout are their names