g_tresoldi

@g_tresoldi@infosec.exchange
68 Followers
417 Following
53 Posts
Cyber and Infosec🇨🇭
🚑 EMT 🛩️ Glider pilot 🎿 Avid skier
✉️g@tresoldi.me
3ma78ECWK3A

A lot of the current hype around LLMs revolves around one core idea, which I blame on Star Trek:

Wouldn't it be cool if we could use natural language to control things?

The problem is that this is, at the fundamental level, a terrible idea.

There's a reason that mathematics doesn't use English. There's a reason that every professional field comes with its own flavour of jargon. There's a reason that contracts are written in legalese, not plain natural language. Natural language is really bad at being unambiguous.

When I was a small child, I thought that a mature civilisation would evolve two languages. A language of poetry, that was rich in metaphor and delighted in ambiguity, and a language of science that required more detail and actively avoided ambiguity. The latter would have no homophones, no homonyms, unambiguous grammar, and so on.

Programming languages, including the ad-hoc programming languages that we refer to as 'user interfaces' are all attempts to build languages like the latter. They allow the user to unambiguously express intent so that it can be carried out. Natural languages are not designed and end up being examples of the former.

When I interact with a tool, I want it to do what I tell it. If I am willing to restrict my use of natural language to a clear and unambiguous subset, I have defined a language that is easy for deterministic parsers to understand with a fraction of the energy requirement of a language model. If I am not, then I am expressing myself ambiguously and no amount of processing can possibly remove the ambiguity that is intrinsic in the source, except a complete, fully synchronised, model of my own mind that knows what I meant (and not what some other person saying the same thing at the same time might have meant).

The hard part of programming is not writing things in some language's syntax, it's expressing the problem in a way that lacks ambiguity. LLMs don't help here, they pick an arbitrary, nondeterministic, option for the ambiguous cases. In C, compilers do this for undefined behaviour and it is widely regarded as a disaster. LLMs are built entirely out of undefined behaviour.

There are use cases where getting it wrong is fine. Choosing a radio station or album to listen to while driving, for example. It is far better to sometimes listen to the wrong thing than to take your attention away from the road and interact with a richer UI for ten seconds. In situations where your hands are unavailable (for example, controlling non-critical equipment while performing surgery, or cooking), a natural-language interface is better than no interface. It's rarely, if ever, the best.

More ICBM photos and discussion here: https://www.mattblaze.org/blog/titans/
Matt Blaze: Notes from the No Lone Zone

Regarding #Brazil banning #X :

After the disaster of the Weimar Republic, the Federal Republic of #Germany came up with the concept of the "Wehrhafte Demokratie" - a "democracy that can defend itself". Fundamental to this concept is that people, groups, and organizations who want to overthrow the basic democratic order should _not_ get a "level playing field".

So while I can understand people worried about any kind of #censorship , I think by this standard this action is justifiable. Brazil went through an attempted coup, and X - and Elon #Musk - were instrumental in aiding the coupists.

And by now Elon Musk has established a clear pattern of "wanting to overthrow the democratic order" - _any_ democratic order. Thus, it is valid if Brazil wants to restrict his speech.

I see this as part of the "Paradox of Tolerance" - tolerating the intolerant will result in less tolerance. And tolerance, like freedom of speech, are social contracts. If you break the social contract - as Elon Musk has - then there should be consequences.

#BREntscheid Die 🇨🇭 darf neu zertifizierten US-Unternehmen Personendaten auch ohne zusätzliche Garantien übermitteln. Sie setzt die USA auf die Liste der Länder mit angemessenem Datenschutz. Der “Swiss-U.S. Data Privacy Framework" tritt am 15.9. in Kraft: https://admin.ch/gov/de/start/dokumentation/medienmitteilungen.msg-id-102054.html
Swiss-U.S. Data Privacy Framework: Zertifizierte US-Unternehmen bieten einen angemessenen Schutz für Personendaten

Aktuelle Informationen aus der Verwaltung. Alle Medienmitteilungen der Bundesverwaltung, der Departemente und Ämter.

Okay so you know, bunch of shitposting and all that, but a serious interlude:

Someone pushed the button to start this rollout. They are probably having a _really_ bad time right now.

If someone at Crowdstrike knows who that is, please go and check on them, give them a hug, tell them it's not their fault, that it's going to be okay. No matter what the company line is on blameless culture whatever, the lizard brain is in charge right now and needs reassurance.

During the international Locked Shields 2024 cyber defense exercise, the Swiss team showcased innovation by using the Hashtag#SCION network architecture to securely connect to Estonia. This collaboration between the Cyber-Defence Campus, the Swiss Locked Shields Team, and industry partners including Anapaya, CybExer Technologies, and Swisscom exemplifies the power of #Swiss technological innovation in enhancing international cyber defense capabilities.

🔐 SCION provided route control and failure isolation, ensuring that our traffic was secure, even in the face of potential routing and DDoS attacks. This milestone not only highlighted the availability of SCION but also sets the stage for future advancements in the security of wide-area networks (WANs).

🌐 The Cyber-Defence Campus is currently exploring further applications of SCION in the context of WANs, as presented at the NATO Cooperative Cyber Defence Centre of Excellence (CCDCOE) Conference on Cyber Conflict #Cycon2024 by Roland Meier. This technology, developed in collaboration with ETH Zürich, promises to improve the security of wide-area networks over public ISP infrastructure.

Read the full story on our website: https://lnkd.in/dyunA4nA
LinkedIn

This link will take you to a page that’s not on LinkedIn

I have managed to extract a list of encoded strings within the liblzma/xz backdoor payload (5.6.1):

https://gist.github.com/q3k/af3d93b6a1f399de28fe194add452d01

The code has a dictionary of strings that are encoded as a prefix trie, which helps to keep things stealthy. This is eg. then used to look up symbols, eg. bd_elf_lookup_hash(..., 0x2b0, ...) means bd_elf_lookup_hash(..., "__libc_stack_end", ...). This is also why it's slow :).

This should bring us one step closer to knowing what the binary payload does.

liblzma backdoor strings extracted from 5.6.1 (from a built-in trie)

liblzma backdoor strings extracted from 5.6.1 (from a built-in trie) - hashes.txt

Gist

This email could have been a paper. And maybe it will be.

https://groups.google.com/a/list.nist.gov/g/pqc-forum/c/8cNYhg23B9k

ML-KEM is not MAL-BIND-K-CT

For context, the problem of LLM-generated papers is something we anticipated at @cydcampus in early 2023, due to the potential of it interfering with scientific article-based emergent technology monitoring.

To respond to this threat, we hired @alexandersternfeld to work on building monitoring proxies that would be resilient to LLM-generated text while not penalizing legitimate articles that were rewritten with LLMs to improve readability.

A new arXiv preprint from James Zou suggests that quite a few academics are also using ChatGPT to write their peer reviews.

I don't have to explain what an utter shitshow this is going to be, do I?

https://arxiv.org/abs/2403.07183

Monitoring AI-Modified Content at Scale: A Case Study on the Impact of ChatGPT on AI Conference Peer Reviews

We present an approach for estimating the fraction of text in a large corpus which is likely to be substantially modified or produced by a large language model (LLM). Our maximum likelihood model leverages expert-written and AI-generated reference texts to accurately and efficiently examine real-world LLM-use at the corpus level. We apply this approach to a case study of scientific peer review in AI conferences that took place after the release of ChatGPT: ICLR 2024, NeurIPS 2023, CoRL 2023 and EMNLP 2023. Our results suggest that between 6.5% and 16.9% of text submitted as peer reviews to these conferences could have been substantially modified by LLMs, i.e. beyond spell-checking or minor writing updates. The circumstances in which generated text occurs offer insight into user behavior: the estimated fraction of LLM-generated text is higher in reviews which report lower confidence, were submitted close to the deadline, and from reviewers who are less likely to respond to author rebuttals. We also observe corpus-level trends in generated text which may be too subtle to detect at the individual level, and discuss the implications of such trends on peer review. We call for future interdisciplinary work to examine how LLM use is changing our information and knowledge practices.

arXiv.org