K900
So, this is a right to repair rant, I guess. Or a story? Anyway. Some timelines will be redacted for narrative purposes, but everything did really happen, just not in that exact order.

I have an old-ish LG TV. It's way out of warranty, and honestly holding up surprisingly well despite all the scary stories about OLED burn-in (yes, yes, technically burn-_out_, I don't care) on the internet. I like it, and have no real reason to replace it.

So, a month or two ago I see the TV randomly turn off by itself, then turn back on again. I shrug it off, probably a power spike or something. The next day it happens again. And again. And again. AND AGAIN.

OK, at this point I'm willing to admit the TV is not doing so good. Also, somehow it seems to be correlated to me turning the computer on. My first thought is power delivery, since they are on the same power rail. It's, again, a pretty old TV, and maybe there's a bad cap or something somewhere.

So I do what any engineer in my position would - I go online to look for a schematic. There is no schematic. I look for a service manual. There is no service manual. I look harder. There's a service manual for an adjacent model. It says:

- step 1: replace motherboard. did it help? if no, go to step 2
- step 2: replace power delivery board. did it help? if no, call your LG rep

OK, fuck you too, I guess. I call my LG rep, which in my case is just the customer service hotline. They say they'll send out a tech.

Fast forward two days. The tech comes in, pops open a service menu, shows me the error code. I now know there's a service menu with error codes. The error code is "CPU ABNORMAL". Very helpful. The tech says it happens to these TVs, and I need a motherboard replacement. OK, cool. I look up the error online and find a few Reddit threads that seem to corroborate that info.

Fast forward a day. The tech calls me and says they don't have spare parts, and need to order the part from LG HQ. They put in a request and we wait.

Fast forward three days. LG HQ calls me and says they don't have the part. I call the tech (who was kind enough to leave me his personal phone number). He says I should write a formal request for reimbursement (yay customer protection laws). I do that.

Fast forward a week. Somehow, LG HQ decides they don't want to reimburse me for the entire TV, so they dig up a spare part. It's $200, and the tech will be here next week to install it. OK, screw it, I'll pay, that's still a lot less than a new TV.

Fast forward... The techs are here, they install the part, things seem to work. Next day, TV reboots again. At this point, I'm starting to believe it's cursed. The techs drive out again, look at the TV, see the same error, call a higher level tech, I hijack the call, he basically says "we have no idea how those things actually work, if it breaks, we send it to HQ, but let's try another board replacement".

Fast forward to yesterday, when I accidentally stumble onto a new root exploit for the webOS versions these TVs run. I've got nothing to lose at this point, so let's dig. After a few attempts to root the TV, I have SSH in. itsaunixsystem.jpg. More specifically, it's a Linux 4.4.84 (oof) on a custom LG SoC (oofffff). I poke around a few things and what do I find? A kernel panic log. The traceback points to `stmmac`. I unplug Ethernet. The TV has been working fine since.

I don't really have a nice ending to this, especially since I'm not 100% confident that was the only issue yet, and I have not fully identified what exactly is causing the bug (though I found some suspicious looking patches in Linus' tree that are missing from LG's). However, probably an entire week's worth of person-hours has been wasted on a software bug that would have been trivial to identify if LG just gave the user enough information.

P.S. If someone from LG is reading this, DO BETTER. You lost SO MUCH fucking money on this. I lost SO MUCH fucking money on this. Hiding this information doesn't help you, and it sure as hell doesn't help me. I don't care about your giga proprietary AI picture improvement algorithm, just give me _something_ I can look at next time my computer (and yes it's a computer) shits itself.
Jan 27 at 8:55amWeb
Show threadK900Jan 27
Oh yeah, some more interesting findings from poking around webOS:

- It's a surprisingly normal Linux overall! It runs systemd! There's even a Pulseaudio instance, though I'm not entirely sure if it's actually doing anything
- The base system seems to be a lot more flexible than just a TV platform (which makes sense, given webOS' history, but it's impressive how much of it survived)
- Most of the apps are built with QML or a custom JS framework named "Enyo" running on what seems to be a custom Electron-style runtime
- The app suspend/resume functionality is implemented with CRIU (on kernel 4.4.lol, no less!)
- There's a custom message bus called Luna that's used instead of D-Bus for most of the actual apps, but there's also a D-Bus for systemd and such
- There is A LOT of log collection machinery that presumably phones home whenever the TV breaks, and collects tons of diagnostic data. What happens to said data afterwards, I don't know. Probably goes into a ticket somewhere and gets ignored forever?
Google

Show threadzopieuxJan 28

@K900 It's a pretty normal Linux indeed! Folks made a homebrew framework right after the first root exploits came out, there's some interesting RE bits in there e.g. screen capture for an ambilight clone https://github.com/webosbrew/webos-homebrew-channel

Slightly cursed fact: the QML can only produce 1080p overlays for the UI, then is hw scaled to 4k!

Personally, I immediately rooted my TV and made sure it couldn't update (call home). It has been the most stable computerish thing I have owned since 2018!

GitHub - webosbrew/webos-homebrew-channel: Unofficial webOS TV homebrew store and root-related tooling

Unofficial webOS TV homebrew store and root-related tooling - webosbrew/webos-homebrew-channel

GitHub
Show threadK900Jan 28
@zopieux I am running that, yeah, though so far mostly because the root script installed it for me. I'll have to see what it can actually do once I am more confident I won't have to yeet this board again.
Show threadCyberspiceFeb 19
@K900 A surprising number of devices now run systemd. I use to have to handle open embedded and rip out systemd but now no one requires that so its openembedded with systemd.
Show threadK900Feb 2
Final (probably?) TV update: it's been a week with no Ethernet plugged in, and zero random crashes. At this point I am basically 100% confident that this was the issue. If anyone here knows any firmware engineers at LG, feel free to ping me and I'll get you the crash dumps and everything.
Show threadK900Feb 17
Final (final) (this time for sure) TV update: after some conversations with LG tech support, they're going to refund me for the replacement board, but they'll have to send a guy to PUT MY OLD BOARD BACK IN AND TAKE THE NEW ONE. Even though both fail exactly the same way. At this point, I have given up on understanding anything. Also, the TV hasn't crashed since, which is nice, I guess.
Show threadJulesJan 27
@K900
Show threadK900Jan 27
@jpenuchot Honestly I don't even mind the smarts, but shipping a 4.4.84 kernel in 2025 is ... definitely a choice.
Show threadJulesJan 27
@K900 2 or 3 years ago I logged on a Skylake-based supercomputer that came with Linux 2.6.32. The kernel was 5 years older than the CPU 🙃
Show threadDaniel GibsonJan 27
@K900
But why did it suddenly start randomly crashing? Wasn't it connected to LAN via Ethernet before?
Show threadScott LairdJan 27

@Doomed_Daniel @K900 probably something new on the network, or a config change someplace innocuous.

Early in the pandemic, I had an extra Ethernet cable plugged into a Sonos speaker cause iPads to grind to a halt, plus random weirdness on a lot of other devices.

When you're dealing with bugs in low-level network code, almost anything can be the trigger.

Show threadK900Jan 27
@laird @Doomed_Daniel My guess would actually be a firmware update - there was one somewhere around the time this began, and the replacement board seems to have worked fine until it downloaded the newest firmware as well (though that may as well be coincidence).
Show threadDaniel GibsonJan 27

@K900 @laird
that sounds very plausible!

one wonders how they managed to make such an ancient kernel buggier with an update, but human ingenuity knows no limits

Show threadK900Jan 28
@Doomed_Daniel @laird it'd be interesting to diff the two kernels, but I think they don't keep old files on the website and I don't care enough to do the GPL request dance.
Show threadWolf480plJan 27
@K900 wait, why was LG HQ on the hook for a refund if the TV was long out of warranty?
Like, I get that there are statutory obligations / implied warrantees, but I thought those don't concern the product breaking down in a safe manner after many uears of use...
Show threadK900Jan 27
@wolf480pl Russian consumer protection laws define a separate warranty period and support period. During the warranty period, the vendor is on the hook for basically everything, at their expense; during the support period, they're required to provide replacement components on request, but they're allowed to charge the user for it. If they're unable to provide a replacement part, they have to either replace the entire device or issue a refund. The TV was not under warranty, but still within the support window.
Show threadWolf480plJan 27
@K900 wow, that's a pretty cool law!
Though it sounds like it's not very effective at making things easy to repair...
Show threadK900Jan 27
@wolf480pl It's not really supposed to make things easy to repair for the consumer, it's mostly supposed to make things _possible_ to _get_ repaired for a reasonable amount of time.
Show threadLichoJan 27
@K900 you say it doesn't help them but if people were just fixing their tvs who would be buying the new, even crappier ones? That's what the planned obsolescence is about. They can afford to get a few people through the hell of trying to fix it to make people think they were honestly trying but you just have to buy a new one.
Show threadK900Jan 28
@licho They've definitely spent more than a new TV's worth in costs on this.
Show threadLichoJan 28
@K900 Once. But they are delivering their message in the process.