Mastodawn

Scenario:
Two machines: Desktop Debian/KDE, and Laptop (System76 PopOS based on debian)

I want to search all files on my filesystem for a fragment of an email address or mastodon domain name, including inside POP mail archives, gzip files, binary files, etc.

What I've tried so far:

KFind: eventually searched files, but took almost as long as grep, and the KFind window crashed when I tried to scroll results

Tracker3: It found a textfile with the relevant string. Didn't find anything else, and didn't update it's cache when I edited the file.

Grep: Slow, as expected, but my usual tool.

I've also tried:
Baloosearch: This seems to work for text files, but not binary. I think this is used by KFind / Krunner in the background, so after a re-index maybe it can find more.

https://news.ycombinator.com/item?id=28950947

There was a discussion on Ycombinator. Several suggestions, and an author of one tool saying he didn't support it anymore personally.

Recoll: A desktop full-text search tool | Hacker News

John (he/him)

Nov 24, 2022

@lizakowski last time I was poking around I used DeskBar, but I can’t seem to find them around anymore 😔

BTW, If I'm doing mastodon wrong, please let me know :)

418 Nov 24, 2022

@lizakowski you might checkout mlocate - it runs on the command line and is fast and efficient. For mail, I use notmuch and mutt. notmuch is amazing.

@admin mlocate is a good tool, but only searches for filenames. I'm looking for contents of files. Grep is too slow.

418 Nov 24, 2022

@lizakowski gotcha 👍

@lizakowski The 'locate' command usually works pretty well. It builds a database and maintains it with cron.
https://linuxize.com/post/locate-command-in-linux/

Locate Command in Linux

The locate command is the quickest and simplest way to search for files and directories by their names.

@bradbrownjr I really like the locate command, and it's super fast, but it only searches filenames. I want the equivalent for file contents. Grep is wasteful and slow.

@lizakowski I see. My apologies, I didn't see the follow-up posts with your use case or the replies until after I replied. You're right, find | grep is sloooow. I'm curious to hear the solution when you find it, as I struggle with the same problem.

https://www.linuxjournal.com/article/1164

@lizakowski I think I think I found what you're looking for!: 'glimpse'.

I apt install'ed it,
ran 'glimpse index ~'
and then glimpse "HELLO WORLD"

it found a script I had been playing with in my home directory.

Indexing with Glimpse | Linux Journal

@bradbrownjr That looks interesting. I just installed it and will try it. It would make sense if this problem was solved in 1995 or earlier.

@bradbrownjr Update: "glimpse" is the right idea, but limited. It is really designed for text files, and has some limits due to it's age:

+ It is fast
+ works well on text
+ supports manhattan distance query -E
+ it can print byte offset within a file

- It doesn't like binary files (-z option might help).
- It objects to long filenames
- It only extracts metadata from html files (not media files, pdf, etc).
- It has a bug: "NUMBERS occur in 68226% of 1963 words..."

But an interesting find!

@lizakowski Bummer, the search continues... Pun intended. 😁