Mastodawn

So, I had a kernel panic recently, and on restart, it would get to the "Unlock Disk" screen (first password prompt on a FileVaulted system), progress about 1/16th of the way, get stuck there for a while, and watchdog reboot.

Tried that a few times with no progress, so into Recovery.

Trying to repair with Disk Utility in Recovery got to the "unlocking disk" point, put up an indeterminate progress bar and a spinner, and looked like it was doing nothing.

This is already one step further than a "normal" user would go. On a plain kernel panic. Bad.

Worried that the encryption was corrupt, I force powered down (nothing was repaired), back into Recovery, then Terminal, then used diskutil to unlock without mount...no problem. Phew.

Next - tried to repair, but you can't, beacause *part* of the volume (system) is mounted. Have to force unmount the drive (disk3), then repair that.

Which is happening, and in progress. But how are "normals" supposed to deal with this?

Truly terrible UX.

Dave Nanian Jun 3

In process.

Dave Nanian Jun 3

And we're two hours in, and it's starting its second pass on "deferred repairs".

old man and the cbowns Jun 3

@dnanian this is so stressful to read but fingers crossed that it finds all the bits. spinning or solid-state media?

(you have it on a UPS? summer storm weather always stresses me out with stuff like this)

Dave Nanian Jun 3

@cbowns Oh yes. Many redundancies. :-)

Dave Nanian Jun 3

Oh ffs, hours later, drive "fixed", still won't boot.

MacOS, when and where did you go wrong? This is absurd.

Music from the 80ies and more Jun 3

@dnanian Sad to hear. As a Windows user, this has happened to me at least once. No recovery mode worked. But let's still hope the best for you!

Dave Nanian Jun 3

@Music_From_The_80ies It'll be fine in the end. It's just a day wasted. I had things to do. A lot of things to do.

Dave Nanian Jun 3

You used to be able to do a verbose boot and see what was happening but I guess that's déclassé.

Inability to boot is the new hotness.

Dave Nanian Jun 3

OK, and now after a full OS reinstall, it rebooted a bunch of times, including one obvious watchdog boot near the end.

And, now, we're back exactly where we were, at the drive unlock pseudo-login, 1/16th of the way in with a progress bar, and about to watchdog-boot again.

macOS is great when it works, which is most of the time. The APIs are similar: great when they work, most o fthe time.

But Apple has a tendency to create black boxes—don't worry about how this stuff does what it does, it just works!—that, when they don't work, are just mysterious, time-sucking failures...either when you're a developer, or, in this case, a user just trying to get his Mac to boot after a kernel panic he did not cause.

It's absurd...and will clearly be an entire, wasted day.

Sebastian Hagedorn

@dnanian I've never tried it, but perhaps a recovery diagnostics file might shed light on things?
https://support.apple.com/guide/mac-help/macos-recovery-a-mac-apple-silicon-mchl82829c17/mac

Use macOS Recovery on a Mac with Apple silicon

Learn how to use macOS Recovery on a Mac with Apple silicon.

Apple Support

Dave Nanian Jun 3

@hagedose68 Hm. I'll see there's anything there after this very slow target disk transfer finishes.

Dave Nanian Jun 3

@hagedose68 Alas, nothing at all useful there.

Dave Nanian Jun 3

I think it's time to admit defeat.

I'm going to use target disk mode to grab the latest versions of the important folders, just in case, and wipe.

Unreal. One mystery kernel panic. Complely mysterious inability to fully boot. No reason, no explanation.

Dave Nanian Jun 3

Target Disk Mode / Drive Sharing works from recovery, so it's not like the drive is somehow unreadable. Everything's there and seems fine.

Little things to be thankful for, even with lots of backups...beacuse there's always a gap, and it's not like I wasn't working.

Matt Godden Jun 3

@dnanian makes the case for *only* having your data on a NAS, and thinking of your Mac as a thin client terminal, if the local OS is so disposable and breakable...

do (smb) network home directories still work? I used to love messing with that, and netboot etc under Tiger / Tiger Server.

Dave Nanian Jun 3

@metaning Heh, no it doesn't. There's nothing magic about a NAS - it's running an OS, like anything else, and it's even MORE of a black box, sometimes quite literally.

Matt Godden Jun 3

@dnanian Yeah, I guess I'd expect NAS vendors to care more / be more competent at stability & data integrity than Apple these days. 🤷

Dave Nanian Jun 3

@metaning I wouldn't.

Apple really does care about this stuff. Things don't usually go wrong. I think the last time I had something like this happen was in...10.4?

Whereas the last time my NAS was unable to self-heal and required direct intervention from the vendor was...last year?

Dave Nanian Jun 3

Honestly, the worst part about this is not having a clue about what's wrong (I mean, beyond the obvious).

The amusing "Recovery Assistant" in Recovery says "nothing is wrong"!

Well, OK, then YOU boot the machine. :-P

Dave Nanian Jun 3

Because I have a lot of redundant backups, I wanted to take inventory before proceeding to nuke-and-pave + restore, so I know what's got the most recent stuff.

I started with my Diskstation TM backups. We've been here for an hour.

Yes, I should just use my super duper backup. I've got everything that I need from the drive. But, I'm trying to see how Apple expects normal people to do this.

Dave Nanian Jun 3

After far too long, and reviewing the I/O speed at the switch, I gave up on that.

Restoring an on disc time machine snapshot from 2 hours before the failure still hangs. This leads me to believe that the problem is probably the snapshots themselves, so I decided to go into disk utility in recovery and try to remove them.

For some reason, the delete option for the snapshot is disabled on every snapshot. There is no way to get rid of them.

It's looking more and more likely that this is just a lost cause and I'm going to have to whack the drive.

Dave Nanian Jun 3

Comedy of wtf continues. Erase the drive and this:

Mieszko Ślusarczyk Jun 3

@dnanian any chance it’s a hardware issue? Not necessarily storage.

Dave Nanian Jun 3

@spitfire There was no indication of SSD failure looking at the SMART data. And hardware tests show no issues.

Mieszko Ślusarczyk Jun 3

@dnanian have you been able to run any equivalent of a real memtest? Like one that doesn’t finish in <3 minutes?

Dave Nanian Jun 3

@spitfire I did whatever the built-in non-quick HW tests are. But it doesn't feel like memory - it's not locking up as such.

Mieszko Ślusarczyk Jun 3

@dnanian it can still be flipping bits. And it might not be memory itself - it can be CPU, memory controller, etc. I don’t know if equivalent of memtest86+ exists for Apple Silicon macs though. Another thing to test (regarding storage ) would be the simple f3 (f3write, f3read).

Mieszko Ślusarczyk Jun 3

@dnanian or it may be bugginess of OS 26 - I wouldn’t know because I haven’t tourchednit except from putting it on an external drive, booting into it to update it, seeing how it works/looks like, and going back to 15.

Dave Nanian Jun 3

@spitfire It must during the HW tests--pretty basic stuff.

@dnanian This is getting ridiculous. Have you considered sorcery? Or … calling AppleCare?

Dave Nanian Jun 4

@WTL I'm recovered at this point. Just documenting the path here so that people can follow along a situation where, basically, everything *but* my backups went wrong.

@dnanian It’s a good example of why having good, robust backups is critical to small business.

Dave Nanian Jun 4

@WTL Hardly just for small businesses.

With people's lives and information on their computers more and more, backups that *work* are important.

@dnanian Oh, larger businesses usually have an IT department. Small shops almost never to, so backups can fall to the side.

Dave Nanian Jun 3

To get by that, you have to go into Terminal, use ""resetpassword", don't reset, and select erase mac from the menus

And then, if you're on a beta, you're rolled back to the regular release. No doubt the next challenge is coming.

But that's right, Apple. The ability to restore everything from a bootable backup is unnecessary.

Because this process is GREAT.

8 hours later. 8 HOURS.

@dnanian the older we get the bigger percentage of our remaining lives that 8 hours is. I’d recommend finding a copy of If Summer Had Its Ghosts and listening to it, having seen Bill Bruford play again recently.

Dave Nanian Jun 3

I think, at this point, I deserve a beer.

Krishna Draws✏️Jun 3

Dave Nanian Jun 3

@krishnadraws That may be too many beers. ;-)

Krishna Draws✏️Jun 3

@dnanian We'll share. :D

Jack Brewster Jun 3

@dnanian You also deserve to know which deities you must have pissed off, and how.

Dave Nanian Jun 3

lol - as expected.

Chancerubbage Jun 3

Updates and bugs and mysterious black boxes of internal function.

I certainly hope the next WWDC addresses some of these long recurring seemingly culturally systemic issues.

@dnanian This is way out of hand. 😳

Dave Nanian Jun 3

Updating to the beta so I can re-wipe and restore.

Mixing with Saison Dupont and Kenny Burrell's Midnight Blue.

Dave Nanian Jun 4

And, given this, off to DFU.

Dave Nanian Jun 4

I have plenty of redundancies. I'm not going to lose any data. What you can't plan for is the time it may take when Apple's process goes wrong.

13 hours in. Restoring from the backup. Probably another 12 hours left

Well, in the meantime, we pushed an update with Tahoe fixes, and if I ever get back into my Mac, I'll post to the blog about it

Stephen Darlington Jun 3

@dnanian Hope you have a good backup...

Dave Nanian Jun 3

@sdarlington So many...

Stephen Darlington Jun 3

@dnanian Perk of the job I suppose!

Dave Nanian Jun 3

@sdarlington better to have them and not need them then need them and not have them. I'm pretty sure I won't need them for this, it's just taking absolutely forever to do the repair.

The point was more how bad and experience this would be for a user who doesn't have the skills I do. It's a terrible terrible design.

Stephen Darlington Jun 3

@dnanian Absolutely.

CM Harrington Jun 3

@dnanian SuperDiskFixer! when? 🤣

Dave Nanian Jun 3

@octothorpe let me check my schedule... Looking like, never? Does that work for you?

CM Harrington Jun 3

@dnanian I mean, I don’t need one right now ;-)

(Monkey paw finger closes)

Rob Fahrni Jun 3

@dnanian Not a good way to spend your day. I do hope you get things recovered. I know of this software company that makes a really great backup software you should try. 😁

Dave Nanian Jun 3

@fahrni You'll have to pass along their name. ;-)

(I do have backups. I was just hoping not to have to use them in anger.)

@dnanian shared family laptop suddenly decided photos would download all original photos in all our accounts and promptly ran out of disk space. (There’s a separate Mac with big external storage that already does that).

No clue or notice that the default changed on the shared laptop. Two hours lost freeing up disk space.

Dave Nanian Jun 4

@cpragman Weird. But hey, could have been much worse: it could have deleted them all.