Julia EvansNov 24, 2023

has anyone made a read-only FUSE filesystem for a git repository where every commit is a folder and the folder contains all the files in that commit?

the idea is that you could just run `cd COMMIT_ID` and poke around instead of checking out the commit

and maybe the branches could be symbolic links to the commit folders?

Show threadJulia Evans

guys this is such a fun idea I cannot believe people are in the replies trying to explain to me why they think it is impractical

the whole point of computers is to do impractical things and see what happens

Nov 24, 2023 at 1:03amWeb
Show threadJulia EvansNov 24, 2023

seems like the answer is yes!

- https://github.com/fanzeyi/giblefs (fuse implementation in rust where every commit is a folder)
- https://orib.dev/git9.html (for plan 9)

would love to hear about any others

GitHub - fanzeyi/giblefs: Mapping a Git repository as a virtual filesystem

Mapping a Git repository as a virtual filesystem. Contribute to fanzeyi/giblefs development by creating an account on GitHub.

GitHub
Show threadJulia EvansNov 28, 2023

ok I got extremely nerd sniped and made my own version of this “filesystem where every git commit is a folder" thing https://github.com/jvns/git-commit-folders

I wrote a FUSE version and an NFS version that I think will be easier for mac users to use. it definitely does not work on windows.

it probably has about 5 million bugs but it seems to kind of work

GitHub - jvns/git-commit-folders

Contribute to jvns/git-commit-folders development by creating an account on GitHub.

GitHub
Show threadJulia EvansNov 29, 2023

the main reason I've found this "all your git commits are folders" feature to be useful is for searching past code: I can run `grep someFunction branch_histories/main/*/commit.go` to find the old version of someFunction I deleted. There are other ways to do that in git but this is easier.

or if I want to just quickly look at a file on another branch to copy a line from it, I can run `vim branches/other-branch/go.mod`

curious about other use cases

(from https://github.com/jvns/git-commit-folders)

GitHub - jvns/git-commit-folders

Contribute to jvns/git-commit-folders development by creating an account on GitHub.

GitHub
Show threadhrbrmstr 🇺🇦 🇬🇱 🇨🇦Nov 29, 2023
@b0rk this is super cool also for a use-case I have where I sometimes need to access older versions of a data file that gets overwritten daily. Current process uses a hacky temp-checkout method, but your work makes it possible to just use `fd`/`find`!
Show threadPetru RatiuNov 29, 2023
@hrbrmstr @b0rk it can be done with git show <revision>:full/path/to/file , but I always have to google the syntax and it's annoying that tab completion doesn't work (since you need to specify the full path from root instead of current folder)
Show threadhrbrmstr 🇺🇦 🇬🇱 🇨🇦Nov 29, 2023
@rpetre aye that’s the hack I’ve been using in an R function wrapper. This is much cleaner
Show threadSumurai8Nov 29, 2023
@b0rk how performant is that. Does it search all files in all commits, or only changed files?
Show threadRhizomatic ArcadeNov 28, 2023
@b0rk if you wanna do windows, try webdav :>
Show threadJulia EvansNov 28, 2023
@arcade yea i wrote a webdav implementation too actually but it has a lot of problems
Show threadRhizomatic ArcadeNov 28, 2023
@b0rk oh interesting!
Show threadRichard BarrellNov 28, 2023
@b0rk I asked a friend at MS who helpfully pointed me to the fact that Windows since Win 10 build 1903 ships with a 9P client (because WSL2 uses it) so in theory if someone did want to make this idea work on Windows too then it might be possible.
Show threadJulia EvansNov 28, 2023
@0x2ba22e11 ooh cool
Show threadjrjsmrtnDec 18, 2023
@b0rk @0x2ba22e11 There is a FUSE equivalent for Windows, btw: WinFSP (https://winfsp.dev). The excellent SSHFS for Windows (https://github.com/winfsp/sshfs-win) is implemented on top of it.
WinFsp

Show threadRichard BarrellDec 18, 2023
@jrjsmrtn thanks!
Show threadBrian McCallisterNov 28, 2023

@b0rk Brilliant!

Exposing things as filesystems rocks! It becomes ridiculously useful to be able to do things like grep and write scripts against stuff like service discovery, ldap/ad, or weird metrics exporting (jmx, snmp, etc).

"Everything SHALL be a file” :-)

Show threadDaniel SchildtNov 28, 2023

@brianm @b0rk That looks useful in combination with Git scraping.

"track changes over time by [collecting changes] to a Git repository"

https://simonwillison.net/2020/Oct/9/git-scraping/

Git scraping: track changes over time by scraping to a Git repository

Git scraping is the name I’ve given a scraping technique that I’ve been experimenting with for a few years now. It’s really effective, and more people should use it. Update …

Simon Willison’s Weblog
Show threadPeter BinkleyNov 28, 2023
@b0rk this might make it easier to make tools like the Obsidian-Git plugin https://github.com/denolehov/obsidian-git , which lets you autocommit changes in your Obsidian vault to a Github repo for syncing between machines
GitHub - denolehov/obsidian-git: Backup your Obsidian.md vault with git

Backup your Obsidian.md vault with git. Contribute to denolehov/obsidian-git development by creating an account on GitHub.

GitHub
Show threadJohn MitchellNov 28, 2023
@b0rk Hm... Are you recreating ClearCase? That thing is/was horrid to use.
Show threadJohnathan CorganNov 28, 2023
@b0rk It is extremely satisfying to browse a homemade filesystem made with FUSE, which is good, because it counterbalances the extreme frustration of trying to understand its documentation 😆
Show threadBen CardoenNov 29, 2023
@b0rk Interesting, thanks for making this available! It seems a hybrid of the timeshift capability of BTRFs (where you can browse snapshots without restoring) and git checkout if I understand it correctly?
Show threadJulia EvansNov 29, 2023
@bencardoen yeah!
Show threadAnthony SoraceNov 24, 2023

@b0rk Glad you found git9. The idea there comes from all the “main” Plan 9 file servers implementing history this way (although with timestamped snapshots rather than commits). Having used it there for many years, I’d say your intuition is absolutely correct. It’s really wonderful to be able to ‘cd’ around and use all your normal tools.

(It’s *extra* nice on Plan 9 where you can, say, bind yesterday’s version of some library over today’s and recompile to bisect a bug.)

Show threadalexwlchanNov 24, 2023
@b0rk I haven’t done this with Gur/FUSE, but I did do terrible things with WebDAV a few years ago to make virtual filesystems that I remember not being too hard? Might be a fun experiment to revisit.
Show threadPetr ViktorinNov 24, 2023
@b0rk Not FUSE, but I made one for Python Paths: https://github.com/encukou/gitpathlib
GitHub - encukou/gitpathlib: Object-oriented paths in Git repositories

Object-oriented paths in Git repositories. Contribute to encukou/gitpathlib development by creating an account on GitHub.

GitHub
Show threadJakub NarębskiNov 24, 2023

@encukou @b0rk There is also `fsspec`: Filesystem interfaces for Python, which among others provide filesystem-like interface to Git repositories, in Python

https://filesystem-spec.readthedocs.io/

fsspec: Filesystem interfaces for Python — fsspec 2024.6.0.post1+g8be9763.d20240613 documentation

Show threadJakub NarębskiNov 24, 2023

@b0rk I thought there were some FUSE for git repo tools mentioned in Git Rev News https://git.github.io/rev_news/archive/ but I could not find any (sources are in git, on GitHub, so it is easy to search the whole archive).

I found list of links on archived old Git SCM Wiki, but I wonder how many of those are still alive:

https://archive.kernel.org/oldwiki/git.wiki.kernel.org/index.php/InterfacesFrontendsAndTools.html#Filesystem_interfaces

Git Rev News Archive

Show threadKonrad HinsenNov 24, 2023

@b0rk Software Heritage has large-scale version: not for one Git repo, but for all Git repos in their archive (meaning all of GitHub plus all of Gitlab plus many more).

https://docs.softwareheritage.org/devel/swh-fuse/index.html

Software Heritage Filesystem (SwhFS) — Software Heritage documentation

Show threadRaineer  🦆Nov 24, 2023
@b0rk I have learned so much trying to do wrong and impractical things
Show threadFrancis 🏴‍☠️ GulottaNov 24, 2023
@b0rk 10000%
Show threadRicardo Signes Nov 24, 2023

@b0rk It's American Thanksgiving, so the turkeys are out in force.

I think it sounds super fun, and hope it exists now or soon. 🙂

Show threadJoe ❌👑Nov 24, 2023
@b0rk Sorry. Go for it.
Show threadKaran JNov 24, 2023
@b0rk 100%, this sounds great way to visualise
Show threadFugue State Machine MotoNov 24, 2023

@b0rk Why not write a shell function which overrides cd() and does a quick ‘git stash -m “cdtemp” && git checkout’ when the destination appears to be a commit hash? You could even autocomplete commit hashes into your cd commands …

Sure, you *could* wrap all that up into a read-only FUSE module, but the shell script is how this old sysadmin thinks about things.

Show threadNasadoNov 24, 2023
@cmdrmoto @b0rk If all you care about is your shell's `cd`, then sure. But off the top of my head, you might want to browse commits in your code editor of choice, search multiple commits' trees simultaneously, or copy files from some old commit to the working tree, none of which can be done easily without treating it as a real filesystem. Not to mention the mental model of `cd` no longer fits, since you can do things like change commits and stay in the same real working directory, or change commits while in a directory that doesn't exist in the destination, or...
Show thread: j@fabrica:~/src;  Nov 24, 2023
@cmdrmoto @b0rk And now you have me wondering what the minimal tooling required to effectively enable writing fuse modules in shell (or anything else capable of execvpe(3))…
Show threadAthena L.M.Nov 24, 2023
@b0rk absolute truth, can't believe I keep having to justify my silly projects to people
Show threadRobNov 24, 2023
@b0rk at recurse last week someone made a fuse filesystem on top of SQLite and then demoed using SQLite in their filesystem, this is like a billion times more practical than that and also that demo was awesome
Show threadOlivier MenguéNov 24, 2023
@simrob @b0rk If you can get a link, I'm interested.
https://mamot.fr/@dolmen/111464201879695372
Olivier Mengué (@[email protected])

@[email protected] I'm now thinking about writing in Go a FUSE fs that exposes any io/fs.FS. I recently wrote an io/fs.FS that exposes an SQLite Archive. https://github.com/dolmen-go/sqlar

La Quadrature du Net - Mastodon - Media Fédéré
Show threadMarcos DioneNov 25, 2023
@simrob @b0rk we want names!... errrr, links to code!
Show threadAndrew DunhamNov 24, 2023
@b0rk I want to take the last sentence there and get it printed out and framed 😂 💯
Show threadPieter de Bruin 🌍🌈☮️Nov 24, 2023
@andrew @b0rk 🙂
Show threadPaul CantrellNov 24, 2023
@b0rk
A FUSE filesystem where every folder is a playful idea and its contents are all of the joyless rebuttal replies
Show threadEdwin YoungNov 24, 2023

@b0rk Be nice if we could treat social media more like improv, where "Yes, and..." is the golden rule.

I have never used it but I think ClearCase works a bit like your description, so it's a sufficiently cromulent idea to have a $$$$ product based on it...

Show threadNicholas WeaverNov 24, 2023
@b0rk I wonder if you keep it with a timing based snapshot mechanism so the timing-based snapshot ALSO forms a GIT commit?
Show threadNoam RossNov 24, 2023
@b0rk Not the same thing but this reminds me of https://mgree.github.io/ffs/
ffs: the file fileystem

mount semi-structured data (like JSON) as a Unix filesystem

ffs
Show threadPeter RalphNov 24, 2023
@noamross woah, neat
Show threadDavid ZaslavskyNov 24, 2023

@noamross @b0rk ooh that's awesome

Though the coolness of the idea is kind of overshadowed by the brilliance of the name 😂

Show threadThe DoctorNov 24, 2023
@b0rk Can I quote you on that?
Show threadcratermoonNov 24, 2023
@b0rk The metaphor of a versioned filesystem is far more intuitive than.. whatever metaphor people have for git.
Show threadPeter RalphNov 24, 2023
@b0rk I love it.
Show threadEoinNov 24, 2023
@b0rk it’s a /great/ idea!
Show threadDan SugalskiNov 24, 2023
@b0rk seems neat, tbh, and I’d love to see it. (And if someone’s going to do that, a fuse file system where the root directory is the commit id and the entire repository existed underneath it to allow easy shell tool twiddling between different commit states would be neat too)
Show threadJes WolfeNov 24, 2023
@b0rk it reminds me of back when the vision for GNU HURD was: no really *everything* is a file, unix philosophy to its most extreme
Show threadmike sugarbakerNov 24, 2023
@jes5199 @b0rk I was gonna say Plan 9!
Show threadJes WolfeNov 24, 2023
@misuba @b0rk yeah it’s never been clear to me whether that idea was directly copied from one to the other or if it was just in the air in the 90s
Show threadOblomovNov 24, 2023
@jes5199 @b0rk wasn't that Plan 9?
Show threadJes WolfeNov 24, 2023
@oblomov @b0rk both!
Show threadRandom Damage 🌻Nov 24, 2023
@b0rk People said PSDoom was impractical and madness as well, but it was so *fun*
Show threadElias MårtensonNov 24, 2023

@b0rk Why would anyone say that? I need this feature right now.

In fact, isn't this kind of how Clearcase worked?