The Art of Pivoting - Techniques for Intelligence Analysts to Discover New Relationships in a Complex World

This open source book explores how intelligence and cyber-security analysts can uncover hidden links between threat actor infrastructure and ongoing investigations by pivoting on both classic and unconventional indicators — many of which are often overlooked. The material is grounded in empirical, field-tested strategies used in cyber-security, digital forensics, cyber threat intelligence, and intelligence analysis more broadly.

I released the first version of this book following the @firstdotorg CTI Conference 2025 in Berlin, where the initial idea for the project emerged.

🔗 PDF https://raw.githubusercontent.com/adulau/the-art-of-pivoting/refs/heads/main/output/the-art-of-pivoting.pdf

🔗 Source of the book in Markdown https://github.com/adulau/the-art-of-pivoting (if you want to contribute ;-)

#cti #threatintelligence #pivoting #pivot #intelligence #intelligencecommunity #cybersecurity #book #openbook #investigation #opensource

@misp
@ail_project

🚀 From C to Python to Rust: How We Built a Fast URL Parser in 3 Days ⚡

When our @ail_project (https://github.com/ail-project) started hitting performance walls with URL processing, we knew we needed a change. We were using Faup (https://github.com/stricaud/faup), a capable C library, but faced two growing pains: 🔄 architectural portability issues and cumbersome Python binding installations. Our first attempt was rewriting it in Python - which solved the installation problems but created new performance bottlenecks when processing millions of URLs. 🐢

That's when we decided to try Rust. Three days later, we had faup-rs (https://github.com/ail-project/faup-rs) - a zero-allocation URL parser with full Python bindings (https://pypi.org/project/pyfaup-rs/). 🎉

What made this possible? Two incredible Rust ecosystem tools:
- Pest (https://github.com/pest-parser/pest) - it might look a bit hostile at first, but this parser generator has consistently saved us weeks of development time across multiple projects. You define the grammar, and Pest handles a lot of the parsing magic. ✨
- PyO3 (https://github.com/PyO3) - which made creating Python bindings almost effortless. 🤝

The result speaks for itself:
- ⚡ Fast URL parsing callable from Python
- 🌍 True cross-platform compatibility
- 📦 Simple pip installation
- 🗑️ Zero allocations during parsing

This experience reinforced an important lesson: when you hit fundamental performance limits, sometimes the fastest solution is rewriting your bottleneck in Rust. The language's combination of speed, safety, and growing ecosystem makes these kinds of transformations not just possible, but practical even on tight timelines. 💡

You can try it today:
- Rust crate: https://crates.io/crates/faup-rs
- Python package: https://pypi.org/project/pyfaup-rs

As always, it is open-source and you can check it out: https://github.com/ail-project/faup-rs 🏗️

#Rust #Python #CTI #Programming #OpenSource #Performance

By the way, we’ve never managed to get any hashes or urls from the so-called non-profit organisations that claim to share CSAM material hashes or links for detection. We’ve come to assume we’ll never receive any, so supporting actual detection was out of reach for us.

So we decided to build it ourselves. We now provide a public onion lookup service where you can look up Tor onion addresses to check whether they’re linked to CSAM for filtering and detection.

Lookup example (non CSAM): https://onion.ail-project.org/?lookup=ef42schauidg4vpuvodxxvd5lz7w2romgrym5fwv3bo4okg5lvcx6xyd.onion

If the content is related to CSAM, the following tag is used dark-web:topic="child-sexual-abuse-material"

🤖 API https://onion.ail-project.org/apiman/redoc/

This functionality (starting from version 6.1) is also included in @ail_project to limit crawling unsafe content. https://www.ail-project.org/blog/2025/02/06/AIL-v6.1.released/

#csam #dfir #threatintel

@torproject

A new release of the AIL project is coming soon, featuring a significant improvement in language detection.

A lot of work has been done on LexiLang by @terrtia to clean up dictionaries and improve support for localized languages and slang.

In the example below, you can see a user active in different Telegram channels, using both Russian and Ukrainian.

🔗 https://www.ail-project.org/

If you're interested in the topic, join us at a 2-day hackathon in Luxembourg on April 8–9, 2025, focused on open-source security tools. The developers of the AIL project will be there in person!

🔗 https://hackathon.lu/

#threatintel #threatintelligence #opensource #ail #intelligence

@ail_project
@circl