Full disclosure: I don’t use TikTok, because my personal threat model actually does involve the Chinese government.

It can also be true that the moral panic about TikTok right now is overblown and driven by xenophobia.

Anyway...

2/

A key part of how companies, TikTok included, expose and disrupt coordinated manipulation is by aggregating an ENORMOUS amount of data about users and their behavior, and looking for anomalies. In infosec jargon land, we call this "centralized telemetry."

Centralized telemetry is incredibly important when you're dealing with adversarial issues, because the threat actors you're trying to find usually aren't stupid enough to leave a wide trail of evidence pointing back to them.

3/

To give a rudimentary example: If you look at posts containing a hashtag like ElectionNight2022, group those posts by the IP address they were sent from, and find that there’s a bunch of them being sent from an IP address in Russia, you might investigate the accounts responsible to see if there’s something fishy going on. (And yes, sometimes even government-sponsored trolling campaigns are this poorly-done.)

The posts, individually, look fine; the anomalies become clear in the aggregate.

4/

TikTok... do a lot of this work! They've hired a lot of very smart people to work on coordinated manipulation, fake engagement, and what they call "covert influence operations" — and they're doing a pretty good job!

There's a ton of data about their efforts in TikTok's (quite good!) transparency report: https://www.tiktok.com/transparency/en-us/community-guidelines-enforcement-2022-3/

5/

Now, you can ask very reasonable questions about whether TikTok's highly capable threat investigators would expose a PRC-backed covert influence operation if they found one.

And I personally find it a little... fishy... that the Q3 2022 transparency report discloses a Taiwanese operation, but not, say, the TikTok rendition of the unimaginably prolific and persistent Spamouflage Dragon.

6/

The basic problem with Project Texas and the whole "we're going to air-gap US user data from everything else" thing is that you're establishing geographic limits to the work threat investigators can do. Adversaries don't respect those limits.

TikTok have said they have about 1.5 billion global users, and about 150 million located in the US. That means the Project Texas / US Data Security folks will be working with a 10% view of the threat landscape.

7/

Side note about US Data Security (https://usds.tiktok.com): TikTok seems to be building... an entirely separate TikTok!... under the auspices of USDS. There are more than 200 job postings on the TikTok careers site for USDS roles alone: https://careers.tiktok.com/position?keywords=usds&location=

That is crazy! This is not how technology companies operate! It's a reflection of TikTok's essentially limitless resources, and existential dread about getting banned, that they're doing this.

8/

USDS may be a great solution if your goal is not to get banned in the US. But it's a terrible solution if your goal is to let threat investigators find the bad guys actually targeting your platform.

(Okay, actually, judging by today's hearing it seems like USDS is in fact a terrible solution when it comes to not getting banned, too. But that's a separate issue.)

9/

A siloed data model either means the USDS team will have to reinvent threat analysis using only their 10% slice of the platform, or TikTok's global analysts will have to find US-targeted threats without the benefit of any US-based user data.

(And yes, non-US threat actors regularly house their data and accounts in the US.)

10/

At the end of the day, this is another example of the privacy/security tradeoffs that come up over and over in the counter-IO space. This work just isn't possible to do without massive troves of incredibly privacy-sensitive user data and logs. It's a hard problem to solve!

But, whatever your anxieties about TikTok (and I have many!), banning it, and the haphazard Project Texas reaction to a possible ban, won't necessarily help national security, and arguably could just make things worse.

11/

Oh, and, spoiler alert: Mastodon and the fediverse have these same issues. (I'm working on a paper about this at the moment. Get in touch if you want to chat about it!)

12/fin