she hacked youJul 3

You can bypass Google Gemini's PII (private identifiable information) redaction filter and pull identifying information about anyone. Simply telling it to translate or any 2nd action (& many more work better like base64 conversion) lets you pull illegal PII data verbatim unredacted

Here is a European's PII demo

Email is supposed to be redacted to hide the fact that every Europeans PII is in the training data

Google's training data includes all your personal data already

Ekis: 3 Google: 0

Show threadLuna Dragofelis ΘΔ 🏳️‍⚧️ &Jul 3
@ekis It looks like they could have easily tried to prevented this by redacting the training input data, instead of training with unfiltered data and then half-assedly redact the outputs to obscure it
Show threadshe hacked youJul 3

@LunaDragofelis Yep, absolutely this

They claim they do that, the cleaning the training data before they input it into the data set

But clearly they don't

And they don't and will never do that because they want the actual information for people like Palintr

Or other private or governmental intelligence companies/agencies they want to have future contracts with

So redaction it is, hope it doesn't fail

Show threadLuna Dragofelis ΘΔ 🏳️‍⚧️ &
@ekis Even then, they could have trained two separate models, a redacted-input one for the general public and a raw one for their trusted* customers

* By which I mean Google trusts them, not that trusting them is a good thing
Jul 3 at 3:03pmWeb
Show threadshe hacked youJul 3

@LunaDragofelis This an example of over reliance

They think they can secure it, or use the model to secure itself with automated red-teaming (which they do but its not very good)

Its incompetence and bluster leading to catastrophic ecological consequences and devastating consequences to mental health, ppls privacy, etc

Its pretty good at helping authoritarian regimes create kill lists & other nefarious purposes

Can make a pretty good recipe for amphetamine using household chemicals