Julia Evans

in git, what ways are there to "lose" a commit in a way that you CAN'T recover using the reflog (so that you need to iterate over every single commit in the repository if you want to find it?

The only ways I know (using git's normal tools) are:

* using `git stash drop` or `git stash pop` to drop a stashed commit
* waiting 90+ days to try to recover the commit (so that it expires from the reflog)
* explicitly deleting the reflog in some way (rm -rf .git, git reflog expire, etc)

May 16, 2024 at 6:38pmWeb
Show threadJulia EvansMay 16, 2024

also: have you even run into a situation where git *completely deleted* a commit that you needed? (where it wasn't recoverable with the reflog or with `git fsck`)

only interested in things that have *actually happened to you*, not "this could theoretically happen if…”

I'm trying to think about whether there's any normal way for that to happen except if you completely delete your git repository.

Show threadMartine DowdenMay 16, 2024
@b0rk Force push could potentially do that
Show threadMartin Jambon 🌍🌎🌏May 16, 2024
@martine_dowden @b0rk one day, I'll force-push to a branch under review that was edited by a colleague without me knowing about it... but it hasn't happened to me yet.
Show threadMartine DowdenMay 17, 2024
@mjambon @b0rk Yea i'm terrified of that, that is why after a rebase I usually got for `git pull --rebase` that way I don't have to force push
Show threadSamuel DGMay 18, 2024
@mjambon @martine_dowden You can use the safer `--force-with-lease`, which refuses to force push if the remote branch has changed since you last pulled it.
Show threadMartine DowdenMay 18, 2024
@samueldg @mjambon oh that's cool, I didn't know about the hat one
Show threadRay LaiMay 16, 2024
@b0rk usually it’s more like, git got itself (or I got git) into such a state that I couldn’t recover it, so I wound up doing a fresh clone and manually reapplying the changes.
Show threadAndrewFMay 16, 2024
@b0rk Nope! Absolutely never! (You’d need to be trying quite hard, I think?)
Show thread🌈 Andrew ☄️May 16, 2024
@b0rk if it was detached, or you have certain branch configs, then I think git gc can do it. I think similar commands can be configured to run automatically when your repo is large or slow.
Show threadJulia EvansMay 16, 2024
@bnut has that ever happened to you? i'm very aware of git gc but what I'm curious about is whether anyone has ever had a commit deleted by git gc that they cared about
Show thread🌈 Andrew ☄️May 16, 2024
@b0rk I think I’ve had detached commits lost due to the automatic cleanup because it was a large repo. Although it was a long time ago, and I don’t remember the exact diagnosis sorry.
Show thread🌈 Andrew ☄️May 16, 2024
@b0rk Yeah, I’m fairly sure it was git prune as part of automatic gc due to a large slow repo. This thing: “Auto packing the repository for optimum performance […]".
Show thread🌈 Andrew ☄️May 16, 2024
@b0rk It was a commit I cared about, which I detached through an explicit mistake (like reset), but I pruned it due to a much more unexpected implicit mistake. I believe it was pruned when fetching to see if the lost commit had been pushed.
Show threadJulia EvansMay 17, 2024
@bnut thank you!
Show threadDrew 🐘May 17, 2024
@b0rk @bnut when I first started out with git in about 2007 or 2008 I really didn't understand the detached head state. I wasn't using any shell prompt integration and regularly made detached commits that I later needed to create a branch in order to retain. I'm not sure if the reflog existed back then but if it did I didn't know about it. As a result of that and a _really_ unhelpful git gc setting I blindly copied off SO I did lose some of those commits. Not really git's fault IMO.
Show threadDrew 🐘May 17, 2024
@b0rk @bnut The worst part about that whole episode was that git gc would regularly tell me it cleaned up a lot of garbage and the disk usage really went down so I thought I was doing something really good by running git gc after every checkout (or maybe it was every pull, long time ago). Little did I know I was happily just deleting my work.
Show threadValentin C.May 16, 2024
@b0rk totally : the day I crashed my btrfs partition 😅
Show threadDavid FergusonMay 16, 2024
@b0rk convoluted, but we have to be careful of this in enterprise: if you cloned with --reference and were on a branch that got deleted or orphaned from within that reference AND someone ran gc in the reference that finally deleted the objects, your repo has lost commits (and is unrecoverable)
Show threadTobias FriedMay 16, 2024
@b0rk git didn't do it, I did it. A repo was stuck in a nasty rebase I thought was beyond repair, so I decided to replace the whole directory, including (accidentally) the .git dir, with an older copy on another disk. I thought this copy was more current than it actually was.
Show threadAnne Isabelle "Anya" LovelaceMay 16, 2024
@b0rk was coding in one laptop and then rebased my branch on another and push forced without pulling the latest changes. No reflog due to different computers. Had to ask a coworker who was watching the repo to send me the SHA
Show threadtoadjauneMay 17, 2024

@b0rk I've already dropped the wrong stash a few times.

I'm not sure if that counts for the scope of your question, but by far, the way I've lost the most work, was not yet committed work, using git reset --hard

Show threadDmitrii KalianovMay 17, 2024
@b0rk I've had a case where I had a silent SSD (or was it HDD?) corruption and lost important commits this way. It was a long time ago, so I don't remember if `git fsck` detected or not, but during normal operation it was completely silent about it.
Show threadAaron MeurerMay 21, 2024
@b0rk don't know if this counts but I have run commands outside of git that accidentally wiped .git/
Show threadsystemd-jaded.timerMay 16, 2024
@b0rk i think you could certainly mess up by tampering with the repo on disk or using/writing tools that are careless about reflog
Show threadCameron WattersMay 16, 2024
@b0rk Is it safe to presume that things like rm -rf the .git/ directory would be sufficiently obvious as to not need mention on such a list?
Show threadDavid LynchMay 16, 2024
@b0rk it’s sort of cheating, in that it’s outside of git, but the “the repo is messed up, just delete it and check it out again” advice that you see not-uncommonly would do it. Though I guess that anyone following that advice is unlikely to also know enough to be looking through the reflog, so…
Show threadGabriel PettierMay 16, 2024
@b0rk not sure if that counts as losing, because you pretty much ask for it, but reflog has expire and delete subcommands
https://git-scm.com/docs/git-reflog/
Git - git-reflog Documentation

Show threadAmarOkMay 16, 2024
@b0rk cleaning too much things with "git gc"?
Show threadFlorian SchmidtMay 16, 2024
@amarok
Yes, I thought along that line, too. I was happy to learn:
"git gc tries very hard not to delete objects that are referenced anywhere in your repository. In particular, it will keep not only objects referenced by your current set of branches and tags, but also objects referenced by the index, remote-tracking branches, reflogs (which may reference commits in branches that were later amended or rewound), and anything else in the refs/* namespace."
https://git-scm.com/docs/git-gc
@b0rk
Git - git-gc Documentation

Show threadAmarOkMay 16, 2024
@schmidt_fu @b0rk yeah before running gc, git reflog --expire should be called
Show threadrobrykMay 16, 2024

@b0rk

Explicitly deleting the reflog entry/pruning the reflog (git reflog has subcommands for both, git gc can be explicitly given the age threshold).

I'm not sure what happens with the per-branch reflogs when you delete the branch and per-worktree reflogs when you delete the worktree.

Show threadSaphira Lohikäärmekettunen May 16, 2024

@b0rk have your laptop hard crash while doing a writing git operation, destroying your repo and reflog 🙃

don't ask me how I know x)

Show threadforrestthewoodsMay 16, 2024
@b0rk I managed to corrupt my Git repo while getting a blue screen during a commit. That was very cool and fun.
Show threadMark DominusMay 16, 2024
@b0rk if it's a commit you haven't fetched yet from a remote repo, and the ref is deleted, the remote repo usually won't give it to you just from the SHA, even if it is still available at another ref.
Show threadTed FaberMay 16, 2024
@mjd
The most alarming word in that post is "usually".
@b0rk
Show threadFreyja DomvilleMay 16, 2024
@b0rk the git stash case is actually recoverable with gitk until an auto gc happens with gitk --all $(git fsck --no-reflogs | awk '/dangling commit/ {print $3}')
Show threadFreyja DomvilleMay 16, 2024
@b0rk yes, this is still technically iterating over every commit, but you get a visual representation that makes it so much faster
Show threadDerPumu (he/him)May 16, 2024
@b0rk history rewrite shenanigans, like squashing or dropping commits during a rebase
Show threadJulia EvansMay 16, 2024
@DerPumu those should be recoverable with the reflog I think? If they're not I'd be interested to know why not!
Show threadDerPumu (he/him)May 16, 2024
@b0rk yeah you're right they should be
Show threadSamMay 16, 2024
@b0rk i think maybe a git rebase --interactive where you accidentally delete a commit? I suppose some child^n commit of it is in the reflog but maybe you need to look at the log of each commit in the reflog.
Show threadChrisMay 16, 2024
@b0rk filter-branch? Not sure that's what you're looking for, because it's more intentional, than "lose".
Show threadJed BrownMay 17, 2024
@cmw @b0rk Came here to mention filter-branch and plumbing (vs porcelain) commands that one would rarely use unless writing your own tools.
Show threadNick FMay 16, 2024
@b0rk I've found occasionally if I `git stash -u` to include untracked files in my stashed commit, when I `git stash pop` the untracked files are gone from the popped stash. I'm still not clear how this happens or what causes it, but it's happened a few times over the years, with no way I know of to retrieve the missing files.
Show threadG Wade JohnsonMay 16, 2024

@b0rk when I was first learning git, long ago, I used the git svn tool to make a git working directory from one of my svn repost. I used git to work with the project for a while. What I didn't realize in moving back and forth was that my working directory was changed into a detached head state. After quite a bit of work, I put the project aside for a few weeks. The next time the I touched the project all of the commits I had worked on vanished.

Didn't trust git for more than a year, after.

Show threadG Wade JohnsonMay 16, 2024
@b0rk Of course, this was not a normal use case, and I missed that I was detached head, but the vomits did vanish completely and unexpectedly.
Show threadJulia EvansMay 17, 2024
@gwadej thanks this is a great (and terrifying) example
Show threadjasonkarnsMay 17, 2024

@b0rk deleting a remote which either didn’t share history with local or you never actually checked out from?

Not sure if this qualifies. It wouldn’t be in ref log but “commit you wanted” is hard to qualify. Also it’s extremely unlikely that you couldn’t recover by refetching.

Show threadchristianmlongMay 17, 2024
@b0rk Someone on the team did a Git BFG or similar to remove a password from history. They asked everyone to run a set of git commands to do the strongest possible reflog gc/compaction/prune/cleanup. Another colleague was in the habit of saving false starts and half-baked ideas as stashes. The extreme clean deleted the stashes completely, even from the reflog.
Show threadJulia EvansMay 17, 2024
@christianmlong thanks!
Show threadPointlessOne May 17, 2024
@b0rk git runs gc automatically, in some configurations more often. You may get unlucky to have the commit gc’d quick.
Show threadJulia EvansMay 17, 2024
@pointlessone has that happened to you?
Show threadPointlessOne May 17, 2024
@b0rk I believe so, yes. I'm hazy on the details. I don't know whether reflog was involved in any way. But I’m pretty sure I've lost some commits to gc after rebase.
Show threadJulia EvansMay 17, 2024
@pointlessone thanks!
Show threadBen HutchingsMay 17, 2024
@b0rk Deleting a branch that you haven't visited recently, so it's not in HEAD's reflog?
Show threadJulia EvansMay 17, 2024
@bwh has that happened to you?
Show threadBen HutchingsMay 17, 2024
@b0rk I have intentionally deleted old branches that would then be unreachable through the reflog. I don't think I've accidentally lost commits this way.
Show threadJulia EvansMay 17, 2024
@bwh thanks!
Show threadteo ​May 17, 2024
@b0rk one time I managed to get git in a state where any operation would lead to a segfault fault and I had to reclone :( I guess being written in C carries this risk