Powerful people imprisoned by the cluelessness of their own isolation, locked up with their own motivated reasoning: "It's impossible to get a CEO to understand something when his quarterly earnings call depends on him not understanding it."
--
If you'd like an essay-formatted version of this thread to read or share, here's a link to it on pluralistic.net, my surveillance-free, ad-free, tracker-free blog:
https://pluralistic.net/2023/08/02/self-incrimination/#wei-bai-bai
1/
Take #MarkZuckerberg. #Zuckerberg insists that anyone who wanted to use a pseudonym online is "two-faced," engaged in dishonest social behavior. The #ZuckerbergDoctrine claims that forcing people to use their own names is a way to ensure civility. This is an idea so radioactively wrong, it can be spotted from *orbit*.
From the very beginning, social scientists (both inside and outside Facebook) told Zuckerberg that he was wrong.
2/
People have lots of reasons to hide their identities online, both good and bad, but a #RealNamesPolicy affects different people differently:
For marginalized and at-risk people, there are plenty of reasons to want to have more than one online identity - say, because you are a #MeToo whistleblower hoping that #HarveyWeinstein won't sic his ex-Mossad mercenaries on you:
https://www.newyorker.com/news/news-desk/harvey-weinsteins-army-of-spies
3/
Or maybe you're a #Rohingya Muslim hoping to avoid the genocidal attentions of the troll army that used Facebook to organize - under their real, legal names - to rape and murder you and everyone you love:
But even if no one is looking to destroy your life or kill you and your family, there are *plenty* of good reasons to present different facets of your identity to different people.
4/
No one talks to their lover, their boss and their toddler in exactly the same way, or reveals the same facts about their lives to those people. Maintaining different facets to your identity is normal and healthy - and the opposite, presenting the same face to everyone in your life, is a wildly terrible way to live.
None of this is controversial among social scientists, nor is it hard to grasp.
5/
But Zuckerberg stubbornly stuck to this anonymity-breeds-incivility doctrine, even as dictators used the fact that Facebook forced dissidents to use their real names to retain power through the threat (and reality) of arrest and torture:
https://pluralistic.net/2023/01/25/nationalize-moderna/#hun-sen
Why did Zuck cling to this dangerous and obvious fallacy? Because the more he could collapse your identity into one unitary whole, the better he could target you with ads.
6/
Truly, it is impossible to get a billionaire to understand something when his mega-yacht depends on his not understanding it.
This motivated reasoning ripples through all of Silicon Valley's top brass, producing what @anildash calls "#VCQAnon," the collection of conspiratorial, debunked and absurd beliefs embraced by powerful people who hold the digital lives of billions of us in their quivering grasp:
https://www.anildash.com/2023/07/07/vc-qanon/
7/
These fallacy-ridden autocrats like to disguise their demands as observations, as though wanting something to be true was the same as making it true. Think of when #EricSchmidt - then the CEO of Google - dismissed online privacy concerns, stating "If you have something that you don't want anyone to know, maybe you shouldn't be doing it in the first place":
https://www.eff.org/deeplinks/2009/12/google-ceo-eric-schmidt-dismisses-privacy
8/
Yesterday, the web was buzzing with commentary about Google CEO Eric Schmidt's dangerous, dismissive response to concerns about search engine users' privacy. When asked during an interview for CNBC's recent "Inside the Mind of Google" special about whether users should be sharing information with...
Schmidt was echoing the sentiments of his old co-conspirator, Sun Microsystems CEO Scott McNealy: "You have zero privacy anyway. Get over it":
https://www.wired.com/1999/01/sun-on-privacy-get-over-it/
Both men knew better. Schmidt, in particular, is *very* jealous of his own privacy. When #Cnet reporters used Google to uncover and publish public (but intimate and personal) facts about Schmidt, Schmidt ordered Google PR to ignore *all* future requests for comment from Cnet reporters:
https://www.cnet.com/tech/tech-industry/how-cnet-got-banned-by-google/
9/
(Like everything else he does, Elon Musk's policy of responding to media questions about Twitter with a poop emoji is just him copying things other people thought up, making them worse, and taking credit for them:)
https://www.theverge.com/23815634/tesla-elon-musk-origin-founder-twitter-land-of-the-giants
Schmidt's actions do not reflect an attitude of "If you have something that you don't want anyone to know, maybe you shouldn't be doing it in the first place." Rather, they are the normal response that we all have to getting doxed.
10/
Tesla’s origin story is the focus of episode two of Land of the Giants: The Tesla Shock Wave, featuring interviews with the co-founders, early employees, and investors to tell the story of Elon Musk’s takeover.
When Schmidt and McNealy and Zuck tell us that we don't have privacy, or we don't want privacy, or that privacy is bad for us, they're disguising a demand as an observation. "Privacy is dead" actually means, "When privacy is dead, I will be richer than you can imagine, so stop trying to save it, goddamnit."
We are all prone to believing our own bullshit, but when a tech baron gets high on his own supply, his mental contortions have broad implications for *all* of us.
11/
A couple years after Schmidt's anti-privacy manifesto, Google launched #GooglePlus, a social network where everyone was required to use their "real name."
This decision - justified as a means of ensuring civility and a transparent ruse to improve ad targeting - kicked off the #NymWars:
https://epeus.blogspot.com/2011/08/google-plus-must-stop-this-identity.html
12/
One of the best documents to come out of that ugly conflict is "Falsehoods Programmers Believe About Names," a profound and surprising enumeration of all the ways that the experiences of tech bros in Silicon Valley are the *real* edge-cases, unreflective of the reality of billions of their users:
https://www.kalzumeus.com/2010/06/17/falsehoods-programmers-believe-about-names/
13/
This, in turn, spawned a whole *genre* of programmer-fallacy catalogs, falsehoods programmers believe about time, currency, birthdays, timezones, email addresses, national borders, nations, biometrics, gender, language, alphabets, phone numbers, addresses, systems of measurement, and families:
https://github.com/kdeldycke/awesome-falsehood
But humility is in short supply in tech. It's impossible to get a programmer to understand something when their boss requires them not to understand it.
14/
A programmer will happily insist that ordering you to remove your "mask" is for your own good - and not even notice that they're taking your skin off with it.
There are *so many ways* tech executives could improve their profits if only we abandoned our stubborn attachment to being so goddamned *complicated*. Think of #Netflix and its anti-passsword-sharing holy war, which is a demand that we redefine "family" to be legible and profitable for Netflix:
https://pluralistic.net/2023/02/02/nonbinary-families/#red-envelopes
15/
But despite the entreaties of tech companies to collapse our identities, our families, and our online lives into streamlined, computably hard-edged shapes that fit neatly into their database structures, we continue to live fuzzy, complicated lives that only glancingly resemble those of the executives seeking to shape them.
Now, the rich, powerful people making these demands don't plan on being constrained by them.
16/
They are conservatives, in the tradition of #FrankWilhoit, believers in a system of "in-groups whom the law protects but does not bind, alongside out-groups whom the law binds but does not protect":
https://crookedtimber.org/2018/03/21/liberals-against-progressives/#comment-729288
17/
As with Schmidt's desire to spy on you from asshole to appetite for his own personal gain, and his violent aversion to having his own personal life made public, the tech millionaires and billionaires who made their fortune from the flexibility of #GeneralPurposeComputers would like to *end* that flexibility. They insist that the time for general purpose computers has passed, and that today, "consumers" crave the simplicity of *appliances*:
https://memex.craphound.com/2012/01/10/lockdown-the-coming-war-on-general-purpose-computing/
18/
It is in the #WarOnGeneralPurposeComputing that we find the cheapest and flimsiest rhetoric. Companies like Apple - and their apologists - insist that no one *wants* to use third-party app stores, or seek out independent repair depots - and then spend *millions* to make sure that it's illegal to jailbreak your phone or get it fixed outside of their own official channel:
https://doctorow.medium.com/apples-cement-overshoes-329856288d13
19/
The cognitive dissonance of "no one wants this," and "we must make it illegal to get this" is powerful, but motivated reasoning is more powerful. It is impossible to get Tim Cook to understand something when his $49 million paycheck depends on him not understanding it.
The War on General Purpose Computing has raged for decades. Computers, like the people who use them, stubbornly insist on being reality-based, and the reality of computers is that they *are* general purpose.
20/
Every computer is a #TuringComplete, universal #VonNeumannMachine, which means that it can run every valid program. There is no way to get a computer to be *almost* Turing Complete, only capable of running programs that don't upset your shareholders' fragile emotional state.
There is no such thing as a printer that will only run the "reject third-party ink" program. There is no such thing as a phone that will only run the "reject third-party apps" program.
21/
There are only laws, like the #Section1201 of the #DigitalMillenniumCopyrightAct, that make writing and distributing those programs a felony punishable by a five-year prison sentence and a $500,000 fine (for a first offense).
That is to say, the War On General Purpose Computing is only incidentally a *technical* fight: it is primarily a *legal* fight.
22/
When Apple says, "You can't install a third party app store on your phone," what they means is, "it's *illegal* to install that third party app store." It's not a *technical* countermeasure that stands between you and #TechnologicalSelfDetermination, it's a legal doctrine we can call "#FelonyContemptOfBusinessModel":
https://locusmag.com/2020/09/cory-doctorow-ip/
But the mighty US government will not step in to protect a company's business model unless it at least *gestures* towards the technical.
23/
To invoke #DMCA1201, a company must first add the thinnest skin of #DigitalRightsManagement to their product. Since 1201 makes removing #DRM illegal, a company can use this molecule-thick scrim of DRM to felonize any activity that the DRM prevents.
More than 20 years ago, technologists started to tinker with ways to combine the legal and technical to tame the wild general purpose computer.
24/
Starting with #Microsoft's #palladium they theorized a new "#SecureComputing" model for allowing companies to reach into your computer long after you had paid for it and brought it home, in order to discipline you for using it in ways that undermined its shareholders' interest.
Secure Computing began with the idea of shipping every computer with *two* CPUs. The first one was the normal CPU, the one you interacted with when you booted it up, loaded your OS, and ran programs.
25/
The second CPU would be a #TrustedPlatformModule, a brute-simple system-on-a-chip designed to be off-limits to modification, even by its owner (that is, you).
The #TPM would ship with a limited suite of simple programs it could run, each thoroughly audited for bugs, as well as secret cryptographic signing keys that you were not permitted to extract.
26/
The original plan called for some truly exotic physical security measures for that TPM, like an acid-filled cavity that would melt the chip if you tried to decap it or run it through an electron-tunneling microscope:
https://pluralistic.net/2020/12/05/trusting-trust/#thompsons-devil
27/
This second computer represented a crack in the otherwise perfectly smooth wall of a computer's general purposeness; and Trusted Computing proposed to hammer a piton into that crack and use it to anchor a whole superstructure that could observe - and limited - the activity of your computer.
This would start with observation: the TPM would observe every step of your computer's boot sequence, creating cryptographic hashes of each block of code as it loaded and executed.
28/
Each stage of the boot-up could be compared to "known good" versions of those programs. If your computer did something unexpected, the TPM could halt it in its tracks, blocking the boot cycle.
What kind of unexpected things do computers do during their boot cycle? Well, if your computer is infected with malware, it might load poisoned versions of its operating system.
29/
Once your OS is poisoned, it's very hard to detect its malicious conduct, since normal antivirus programs rely on the OS to faithfully report what your computer is doing. When the AV program asks the OS to tell it which programs are running, or which files are on the drive, it has no choice but to trust the OS's response. When the OS is compromised, it can feed a stream of lies to users' programs, assuring these apps that everything is fine.
30/
That's a very beneficial use for a TPM, but there's a sinister flipside: the TPM can also watch your boot sequence to make sure that there aren't *beneficial* modifications present in your operating system. If you modify your OS to let you do things the manufacturer wants to prevent - like loading apps from a third-party app-store - the TPM can spot this and block it.
31/
Now, these beneficial and sinister uses can be teased apart. When the Palladium team first presented its research, my colleague #SethSchoen proposed an "#OwnerOverride": a modification of Trusted Computing that would let the computer's owner override the TPM:
https://web.archive.org/web/20021004125515/http://vitanuova.loyalty.org/2002-07-05.html
This override would introduce its own risks, of course.
32/
A user who was tricked into overriding the TPM might expose themselves to malicious software, which could harm that user, as well as attacking other computers on the user's network and the other users whose data were on the compromised computer's drive.
33/
But an override also provides serious benefits: it rules out the monopolistic abuse of a TPM to force users to run malicious code that the *manufacturer* insisted on - code that prevented the user from doing things that benefited the user, even if it harmed the manufacturer's shareholders. For example, with owner override, Microsoft couldn't force you to use its official MS Office programs rather than third-party compatible programs like #iwork #GoogleDocs or #LibreOffice.
34/
Owner override also completely changed the calculus for another, even more dangerous part of Trusted Computing: #RemoteAttestation.
35/
Remote Attestation is a way for third parties to request a reliable, cryptographically secured assurances about which operating system and programs your computer is running. In Remote Attestation, the TPM in your computer observes every stage of your computer's boot, gathers information about all the programs you're running, and cryptographically signs them, using the signing keys the manufacturer installed during fabrication.
36/
You can send this "attestation" to other people on the internet. If they trust that your computer's TPM is truly secure, then they know that you have sent them a true picture of your computer's working (the actual protocol is a little more complicated and involves the remote party sending you a random number to cryptographically hash with the attestation, to prevent out-of-date attestations).
37/
Now, this is *also* potentially beneficial. If you want to make sure that your technologically unsophisticated friend is running an uncompromised computer before you transmit sensitive data to it, you can ask them for an attestation that will tell you whether they've been infected with malware.
But it's also potentially *very* sinister. Your government can require all the computers in its borders to send attestations to confirm that you're still running mandatory spyware.
38/
Your abusive spouse - or abusive boss - can do the same for their own #DisciplinaryTechnologies. Such a tool could prevent you from connecting to a service using a #VPN, and make it impossible to use #TorBrowser to protect your privacy when interacting with someone who wishes you harm.
The thing is, it's *completely normal and good* for computers to lie to other computers on behalf of their owners.
39/
Like, if your #IoT ebike's manufacturer goes out of business and all their bikes get bricked because they can no longer talk to their servers, you can run an app that tricks the bike into thinking that it's still talking to the mothership:
https://nltimes.nl/2023/07/15/alternative-app-can-unlock-vanmoof-bikes-popular-amid-bankruptcy-fears
40/
A new app to unlock bicycles produced by the financially troubled Amsterdam firm, VanMoof, has reached the top tier of most popular iPhone apps in the Netherlands within a day. VanMoof's Belgian competitor, Cowboy, launched an app called Bikey for iOS on Thursday, and less than a day later it was one of the top three free apps in the Dutch market. Once it became public knowledge that VanMoof needed protection from its creditors, the company's e-bike owners grew more concerned about whether the VanMoof app will still work. The app is capable of unlocking the bike for its rider.
Or if you're connecting to a webserver that tries to track you by fingerprinting you based on your computer's RAM, screen size, fonts, etc, you can order your browser to send random data about this stuff:
https://jshelter.org/fingerprinting/
Or if you're connecting to a site that wants to track you and nonconsensually cram ads into your eyeballs, you can run an #adblocker that doesn't show you the ads, but tells the site that it did:
https://www.eff.org/deeplinks/2019/07/adblocking-how-about-nah
41/
Owner override leaves some of the beneficial uses of remote attestation intact. If you're asking a friend to remotely confirm that your computer is secure, you're not going to use an override to send them bad data about about your computer's configuration.
42/
And owner override also sweeps *all* of the malicious uses of remote attestation off the board. With owner override, you can tell any lie about your computer to a webserver, a site, your boss, your abusive spouse, or your government, and they can't spot the lie.
But owner override also eliminates *some* beneficial uses of remote attestation.
43/
For example, owner override rules out remote attestation as a way for strangers to play multiplayer video games while confirming that none of them are using cheat programs (like #aimhack). It also means that you can't use remote attestation to verify the configuration of a cloud server you're renting in order to assure yourself that it's not stealing your data or serving malware to your users.
44/
This is a tradeoff, and it's a tradeoff that's similar to lots of other tradeoffs we make online, between the freedom to do something good and the freedom to do something bad. Participating anonymously, contributing to free software, distributing penetration testing tools, or providing a speech platform that's open to the public all represent the same tradeoff.
45/
Cory Doctorow
