zdwApple randomly closes bug reports unless you "verify" the bug remains unfixed
I recognize that this is annoying from a user perspective, but I do understand it. Not all bugs are easily reproducible (and even if they are 100% reproducible for the user, it's not always so easy for the developers). Also sometimes you make a change to the code that you think might be in a related area, and so sometimes the most "efficient" thing is just to ask the user to re-test.
When I close an old bug that is not actionable, I do feel bad about it. But keeping the bug open when realistically I can't really do anything with it might be worse.
A company like Apple should have complex enough tools to perfectly capture system state at the time of the bug so that they can reproduce it
That’s easy enough. The hard part is doing so without capturing a bunch of email, messages, and other private data that happens to be in memory at the time.
Ignorant question, if privacy didn’t matter and they had an atomically identical machine, would there still be plenty of edge cases where it was the printer or the Wi-Fi causing the issue?
In any case I would have said it sounds difficult on every front
I don't work at Apple, so I can't comment on that. But that doesn't always help. There's been plenty of times where I have a full HAR file from the user and I can clearly see that something went wrong, but that doesn't always mean I can reproduce the issue. (I recognize a HAR file doesn't represent the complete state of the world, but it's often one of the best things a backend developer can get)
Reminds me of this Raymond Chen Microsoft blog post:
https://devblogs.microsoft.com/oldnewthing/20241108-00/?p=11...
> keeping the bug open when realistically I can't really do anything with it might be worse
I've heard this from others before but I really don't understand the mindset.
What's the harm in keeping the bug open?
I used to think that there is no harm in keeping the bug open. I think if you honestly feel that you have the time and resources to go back to the bug and fix it, then by all means keep it open.
But I find that sometimes I can tell from experience that the IR is not actionable and that it will never be fixed. Some examples:
* There's not enough info to reproduce the issue and the user either can't or won't be able to reproduce it themselves. Intermittent bugs generally fall into this category.
* The bug was filed against some version of the software that's no longer in production (think of the cloud context where the backend service has been upgraded to a newer version).
Sometimes the cost to investigate a bug is so high relative to the pain caused that it just closed as a WONTFIX. These sometimes suck the most because they are often legitimate bugs with possible fixes, but they will never be prioritized high enough to get fixed.
Or sometimes the bug is only reproducible using some proprietary data that I don't have access to and so you sometimes have no choice but to ask the bug filer "can you still reproduce this?".
Computer systems are complicated. And real-world systems consisting of multiple computer systems are even more complicated.
I think asking someone if they can still reproduce an issue is valid. Especially if it was trivially reproducible for them, and now it isn't, that seems like a fine resolution, and the bug should be closed.
But in the other cases, closing the bug seems to me to be a way to perturb metrics. It might be true that you'll never fix a given bug, but shouldn't there be a record of the "known defects", or "errata" as some call them?
For your specific scenarios:
- lack of information on how to reproduce or resolve a bug doesn't mean it doesn't exist, just that it's not well understood.
- For the "new version" claim, I've seen literal complete rewrites contain the same defects as the previous version. IMHO the author of the new version needs to confirm that the bug is fixed (and how/why it was fixed)
- I agree there are high cost bugs that nobody has resources to fix, but again, that doesn't mean they don't exist (important for errata)
- Similarly with proprietary data, if you aren't allowed to access it, but it still triggers the bug, then the defect exists
In general my philosophy is to treat the existence of open bugs as the authoritative record of known issues. Yes, some of them will never be solved. But having them in the record is important in and of itself.
Surely a few years worth of open but unverified bugs would cause some issues with reporting and such.
What is the use in keeping it open when no one will ever look at it again after it goes stale? It still exists in the system if you ever wanted to find it again or if someone reports the same issue again. But after a certain time without reconfirming the bug exists, there is no point investigating because you will never know if you just haven't found it yet or if it was fixed already.
See my reply to eminence32 - bug tracking serves as a list of known defects, not as a list of work the engineers are going to do this [day/month/year].
How is that worse? Leaving it open signals to anyone searching about it that's it's still an issue of concern. It will show up in filters for active bugs, etc. Closing it without fixing it just obfuscates the situation. It costs nothing (except pride?) to leave "Issues (1)" if there is indeed an Issue.
Back in another part of my career I worked a lot with putting Macs on ActiveDirectory. And there was a common refrain from Apple about bugs in that implementation: "works on 17!".
The joke is that Apple owns the 17.x.x.x class-A range on the Internet (they got in early, the also have a second class-B and used to have a second class-B that they gave back), and what engineers were really saying is that they could not reproduce on the AD systems that Apple had setup (lots of times it was because AD had been setup with a .local domain, a real no-no, but it was in Microsoft's training materials as an example at the time...).
> Not all bugs are easily reproducible
Apple did not say they couldn't reproduce it. Neither did they say that they thought they fixed it. They refused to say anything except "Verify with macOS 26.4 beta 4".
> and even if they are 100% reproducible for the user, it's not always so easy for the developers
It's not easy for the user! Like I said in the blog post, I don't usually run the betas, so it would have been an ordeal to install macOS 26.4 beta 4 just to test this one bug. If anything, it's easier for Apple to test when they're developing the beta.
> the most "efficient" thing is just to ask the user to re-test.
Efficient from Apple's perspective, but grossly inefficient from the bug reporter's perspective.
> realistically I can't really do anything with it
In this case, I provided Apple with a sample Xcode project and explicit steps to reproduce. So realistically, they could have tried that.
I suspect that your underlying assumption is incorrect: I don't think Apple did anything with my bug report. This is not the first time Apple has asked me to "verify" an unfixed bug in a beta version. This seems to be a perfunctory thing they do before certain significant OS releases, clear out some older bug reports. Maybe they want to focus now on macOS 27 for WWDC and pretend that there are no outstanding issues remaining. I don't know exactly what's going through their corporate minds, but what spurred me to blog about it is that they keep doing this same shit.
Author must not have worked in enterprise software before.
That's a classic trick where the developer will push back on the bug author and say "I can't reproduce this, can you verify it with the latest version?" without actually doing anything. And if it doesn't get confirmed then they can close it as User Error or Not Reproducible.
Of course, the only way to counter this is by saying "Yes I verified it" without actually verifying it.
From experience with Microsoft (paid) support (after doing 5 tickets because it's never the right team and apparently moving tickets internally is for losers), they will ask for proof of the reproduction. And they will take every opportunity to shift the blame ("Oh I can see in the log you're running an antivirus, open a ticket with them. Closed").
It'd kind of sad, how the market went. I suppose there are pluses too.
But back in the 80s and 90s, margins were significantly higher. If you look at hardware, I recall selling hardware with 30% margin, if not more... even 80% on some items.
Yet what came with that was support, support, support. And when you sell 5 computers a month, instead of 500, well.. you need that margin to even have a store. Which you need, because no wide-scale internet.
On the software side, it was sort of the same. I remember paying $80 for some pieces of software, which would be like $200 today. You'd pay $1 on an app store for such software, but I'd also call the author if there was a bug. He'd send an update in the mail.
I guess my point is, in those days, it was fun to fix issues. The focus was more specific, there was time to ply the trade, to enjoy it, to have performant, elegant fixes.
Now, it's all "my boss is hassling me and another bug will somehow mean I have to work harder", which is .. well, sad.
I recompiled OpenSSL to make s_server -www return the correct, static XML blob for a .NET application that was buggy to make a reproducer for them that didn't rely on our product at all and which could be self-contained on a very barren windows VM they could play with to their heart's content and which didn't even care about the network because everything was connecting via loopback, so they couldn't blame that, eitehr.
Turns out there was a known bug in Microsoft schannel that had yet to be patched and they'd wasted weeks of our effort by not searching their own bug tracker properly.
My favourite variant of this merrygoround is when they ask you to demonstrate the issue live in a Teams session, you do so, and there's this moment of silence followed by an "Oh... I see".
Then you assume, naively, that this means that they've recognised that there really is a product problem and will go off and fix it. However, then in turn the support tech needs to reproduce the the issue to the development team.
They invariably fail to do so for any number of reasons, such as: This only happens in my region, not others. Or the support tech's lab environment doesn't actually allow them to spin up the high-spec thing that's broken. Or whatever.
Then the ticket gets reject with "can't reproduce" after you've reproduced the issue, with a recorded video and everything as evidence.
If you then navigate that gauntlet, the ticket is most typically rejected with "It is broken like that by design, closed."
> Author must not have worked in enterprise software before.
Or with open source projects. Fucking stalebot.
All my homies hate stalebot
Take a look at Anthropic's repo. They auto-close issues after just a few weeks.
I don't think I've seen an issue of theirs that wasn't auto-closed.
As an open source maintainer, I feel that statement is really unfair. Yes, we do sometimes close bug reports without evidence they are fixed. But:
- We owe you nothing! And the fact that people still expect maintainers to work for them is really sad, IMHO.
- Unlike corporate workers, nobody is measuring our productivity therefore we have no incentive to close issues if we believe they are unfixed. That means that when we close the issue, we believe it has a high chance of being fixed, and also we weigh the cost of having many maybe-fixed open issues against maybe closing a standing issue, and (try to) choose what's best for the project.
Why do you close the issue then?
> That means that when we close the issue, we believe it has a high chance of being fixed
I agree with this iff it's being done manually after reading the issue. stalebot is indiscriminate and as far as "owing" the user, that's fair, but I'd assume that the person reporting the bug is also doing you a favor by helping you make things more stable and contributing to your repo/tool's community.
Or even non-software tickets at large corporations. I reported a water dispenser filling too slowly at my office because it took me a few tries just to fill my 1L water bottle. They said it was fixed and closed it.
It was not fixed. So I took a video of myself refilling my water bottle, attached it to the ticket, and re-opened it. They actually fixed it after that. The video was 2m12s long (and I spent god knows how long making the video file small enough to attach to the ticket lol)
Yep. On the other side of the curtain this often isn't nefarious. It's a simple cost/benefit analysis of spending time on something that one user is complaining about versus a backlog of higher business priorities. I've seen this in my work and it makes me sad for the user, but it often does take a bit of effort to spear these bug reports through.
I’d argue that there should be no higher business priority than shipping a product you already sold. If you sold a product and your customer spends their time documenting exactly why and how you sold them something that’s broken, you should make that a high priority. As a natural progression, you’ll start shipping less buggy / better tested products and that’s how you unlock yourself from the obligation you made to your existing customers to do other work.
Not directed at you of course, just the proverbial “you” from the frustration of a purchaser of software.
Careful saying that too loudly, the “ship new features at all costs” gang will come for your head. They don’t approve of things like “quality software” and “making stuff that works past the demo and cursory inspection” or “actual user utility”.
I totally understand that from the perspective of individual employees: they have little incentive to do more than the bare minimum to close tickets. But this behavior is typically a symptom of broken corporate culture and failure to align internal metrics. For every customer who takes the trouble to submit a formal bug report there are likely many others who just live with it, and badmouth you to other customers. Doing deep investigations of even minor bug reports also tends to expose other, more serious latent bugs. And root cause analysis allows you to create closed-loop solutions to prevent similar future bugs.
Large monopolistic tech companies like Apple and Microsoft can afford to ignore this stuff for years because there are few realistic alternatives. But longer term eventually a disruptive competitor comes along who takes product quality and customer service more seriously.
There's also going to be mountains of bugs resulting from cosmic rays hitting the computer, defective ram chips, weird modifications of the system the reporter hasn't mentioned.
You could sink an infinite amount of time investigating and find nothing. At some point you have to cut off the time investment when only one person has reported it and no devs have been able to reproduce it.
What if no devs even tried to reproduce it, and they have no reason to believe they've fixed the bug with any other changes?
That seems to be the case described in the article. In such a situation, I think it's dishonest to ask the reporter to expend even more effort when you've spent zero. Just close it if you don't want to do it, you don't have to be a jerk to your customers, too, by sending them off on a wild goose chase.
Otherwise, why not ask the reporter to reproduce the issue every single day until you choose to fix it in some unknown point in the future, and if they miss a day, it gets closed? That seems just as arbitrary.
Hi, bigcorp employee getting showered with tickets here.
I don't have enough time in the day to deal with the tickets where the reporter actually tries, let alone the tickets where they don't.
If I tell you to update your shit, it's because it's wildly out of date, to the point that your configuration is impossible for me to reproduce without fucking up my setup to the point that I can't repro 8 other tickets.
Please tell us where you work so we can avoid all of your company’s software. Unless it’s Microsoft, because we’ve already seen the results of that attitude there.
I don't see how it's an unreasonable request. If you demand that I work with some ancient version, I then have to install and uninstall said program every time I work on your ticket specifically. You will be prioritized last, because my effectiveness is measured by how many tickets I close.
This is psychotic. Literally. <https://en.wikipedia.org/wiki/Occupational_psychosis>
> If you demand that I work with some ancient version, I then have to install and uninstall said program every time I work on your ticket specifically.
You completely missed the point of the blog post. Apple was in the process of developing macOS 26.4 beta 4, and they wanted me to install the beta just to "verify" the bug.
Apple could test my bug with 26.4 beta 4 a heck of a lot easier than I could. Nobody was asking Apple to install some ancient version.
> my effectiveness is measured by how many tickets I close.
That was one of the points of the blog post: this is a perverse incentive from management.
Note what you did not say: "my effectiveness is measured by how many bugs I fix." So engineers are incentivized to close tickets even if the bugs they report are unfixed. This is how a company ends up with crappy, buggy software.
I'm with you on the Apple thing, that's asinine.
The parent comment is talking about the broader practice of people telling you to update and then repro again. That's a completely legitimate thing to ask, given both the perverse corporate incentives and the basic reality that version toggling makes a tech far less efficient at solving all tickets, not just yours.
I think that's entirely dependent on the workload the company is placing on their support staff. If Apple decides the techs should be handling 10 tickets at once, then the techs have a choice:
1. Tell everyone to update their shit, and close tickets if they don't.
2. Waste several hours per day uninstalling and reinstalling 10 versions of the same program.
One of these will allow you to close lots of tickets immediately, and handle the remaining ones as efficiently as possible. Yay! Good job, peon! You get a raise!
The other approach will result in a deep backlog, slow turnaround times, and lower apparent output from management's perspective. Boo! Bad job, peon! You're fired!
Back when I worked at Apple I would just try it in whatever I had installed. If it didn't reproduce I'd write "Cannot reproduce in 10.x.x" and close it. Maybe a third were like that, duplicates of some other issue that was resolved long ago.
Anyone that attached a repro file to their issue got attention because it was easy enough to test. Sometimes crash traces got attention, I'd open the code and check out what it was. If it was like a top 15 crash trace then I'd spend a lot longer on it.
If the ticket was long and involved like "make an iMovie and tween it in just such and such a way" then probably I'd fiddle around for 10-15 minutes before downgrading its priority and hope a repro file would come about.
There were a bunch of bug reports for a deprecated codec that I closed and one guy angrily replied that I couldn't just close issues I didn't want to fix!
Guess what buddy, nobody's ever going to fix it.
The oldest bug like that I ever fixed was a QuickDraw bug that was originally written when I was 8 years old but it was just an easy bounds check one liner.
But the mistake OP is making is assuming this one thing that annoyed him somehow applies to the whole Apple org. Most issues were up to engineers and project managers to prioritize, every team had their own process when I was there.
> But the mistake OP is making is assuming this one thing that annoyed him somehow applies to the whole Apple org. Most issues were up to engineers and project managers to prioritize, every team had their own process when I was there.
Except this same shit keeps happening with multiple teams.
Judging from your mention of QuickDraw, which was removed entirely from macOS in 2012, perhaps your Apple experience is now out of date.
Nah, you're just making shit up.
That the ~50000 engineers at Apple are conspiring to close your tickets in the exact same way. It's ridiculous
> perhaps praying that the bug had magically disappeared on its own, with no effort from Apple.
I suspect that this is a common approach. It maybe even works, often enough, to make it standard practice.
For myself, I've stopped submitting bug reports.
It's not the being ignored, that bothers me; it's when they pay attention, they basically insist that I become an unpaid systems engineering QC person, and go through enormous effort to prove the bug exists.
> they basically insist that I become an unpaid systems engineering QC person
Microsoft support is guilty of this, especially for Azure & 365 issues.
Like sorry, but you aren't paying me to debug your software. Here's a report, and here's proof of me reproducing the problem & some logs. That's all I'm going to provide. It's your software, you debug it.