Shawn WebbAug 29, 2023

Turns out that libcurl rejects lookups of .onion domains. This makes it impossible to use pkg in #HardenedBSD with our #Tor Onion Service endpoint for our package repo.

Good thing we've forked the #FreeBSD package manager, and it includes an import of libcurl.

So let's remove this arbitrary restriction that harms #HumanRights and get the package repo working again for our Tor users:

https://git.hardenedbsd.org/hardenedbsd/pkg/-/commit/d785066a1e71e5437678e837a91508bcb85759bb

https://git.hardenedbsd.org/hardenedbsd/ports/-/commit/9e3a0a6d0396429f2aa6627b1cd3f1d3199a3ad9

HBSD: Enable support for resolving .onion domains in vendored libcurl (d785066a) · Commits · HardenedBSD / pkg · GitLab

libcurl rejects outright any Tor Onion domain. Since the HardenedBSD project provides a Tor Onion Service endpoint for its package repos, the rejection of .onion domains needs to be removed....

GitLab
Show threadShawn WebbAug 29, 2023

I've now applied the patch to the ftp/libcurl port in the #HardenedBSD ports tree: https://git.hardenedbsd.org/hardenedbsd/ports/-/commit/3173d64b4bc01c0c1c258fe5191c65ea0a766181

To developers working in networking code: please just do the right thing: do not treat any TLD as special. Do not violate the principle of least astonishment, especially when it comes to DNS.

Perhaps we can convince the #curl #libcurl community to revert the patch on their end.

#infosec #FreeBSD #HumanRights #HumanRightsTech

HBSD: Restore libcurl's ability to resolve .onion domains (3173d64b) · Commits · HardenedBSD / Ports · GitLab

Earlier this year, the curl project decided to prevent working within a human rights-centric environment. HardenedBSD provides Tor Onion Service endpoints for all parts of its public infrastructure. With the...

GitLab
Show threaddaniel:// stenberg://Aug 30, 2023

@lattera The Tor project itself was involved in the RFC that curl follows by *not leaking* .onion addresses.

curl always worked fine with Tor and still does. Without that patch.

Show threadShawn WebbAug 30, 2023

@bagder .onion resolution is completely broken for me with curl commit 0ae0abbe72514a75c10bfc4108d9f254f594c086. It completely refuses to lookup any .onion address.

Here's my Tor setup: https://git.hardenedbsd.org/shawn.webb/articles/-/blob/master/infosec/tor/2017-01-14_torified_home/article.md

infosec/tor/2017-01-14_torified_home/article.md · master · Shawn Webb / articles · GitLab

Various technical and personal blog articles I've written.

GitLab
Show threaddaniel:// stenberg://Aug 30, 2023
@lattera You know I know this. But it would be cooler if you offered a way forward instead of accusing us of shady practices.
Show threadShawn WebbAug 30, 2023
@bagder I think the way forward is just to remove the offensive code. Don't violate POLA and don't block access to an entire human rights focused network. I would consider a patch like the one curl adopted a form of censorship.
Show threadjordanAug 30, 2023

@lattera @bagder Except that it's something the Tor folks explicitly asked for people to do, to prevent attacks involving tricking people into inadvertently resolving onion domains on the clearnet.

Maybe this behavior could be controlled by something like a "NO_RFC7686" environment variable? I assume Tor and curl work fine together if you use the SOCKS5 proxy and take curl out of the DNS resolution pathway?

Show threadShawn WebbAug 30, 2023

@jordan @bagder I use the transparent proxy feature to prevent information leaks, like DNS issues. Proxying has a lot of issues, including (and especially) DNS leakage issues.

I understand that the Tor project helped with this RFC, and the part that influenced the creation and adoption of the patch in question.

However, I think both the patch and the section in the RFC in question are both a net negative.

Using an environment variable is an attractive solution in some, but not all, cases. I can set it on my HardenedBSD laptop. I'm unsure whether I can control libcurl behavior on my Android phone via environment variables.

Performing any secondary network calls in libcurl itself would introduce a leak, too. So libcurl can't really auto-detect that it's behind a Tor-ified network. Asking every developer to change every networked application to perform Tor auto-detection would introduce the same leak and be a Herculean task, especially for proprietary applications that don't receive updates.

Really, the only solution here that preserves compatibility is for libcurl to trust its operating environment and not treat .onion in any special manner.

Show threadjordanAug 30, 2023
@lattera @bagder I understand how it is a net negative for your "I know exactly what I'm doing" use case. I think the RFC was written with the perspective of "I want to protect J. Random Dissident, who may not have the same level of technical resources as his adversaries," and I think from that perspective, the RFC is valuable. I don't think it's a good idea to change the default behavior to increase convenience for "I know exactly what I'm doing" at the expense of safety for J. Random Dissident.
Show threadShawn WebbAug 30, 2023

@jordan @bagder the default of "allow Tor" was changed to "censor Tor" in March of this year: https://github.com/curl/curl/commit/0ae0abbe72514a75c10bfc4108d9f254f594c086

It's certainly not too late to change direction.

hostip: refuse to resolve the .onion TLD · curl/curl@0ae0abb

RFC 7686 states that: > Applications that do not implement the Tor > protocol SHOULD generate an error upon the use of .onion and > SHOULD NOT perform a DNS lookup. Let's do that. h...

GitHub
Show threaddaniel:// stenberg://Aug 30, 2023
@lattera @jordan and yet it works great with Tor
Show threadShawn Webb

@bagder @jordan No, in fact, it does not. libcurl rejected resolving my .onion address, making it impossible to update packages on my research laptop.

If you see a failure in applying OS security updates as a success, I'm not sure we're talking the same language.

Aug 30, 2023 at 4:33pmWeb
Show threadJoubin 🛡️Aug 31, 2023
@lattera @bagder @jordan as a unbiased third-party let me chime in here for a second… The point of the spec is to not falsely give users of applications, a sense of security and privacy, when the ecosystem of said application leaks information when communicating with Tor. I think the fixes you’re looking for should be placed in the upstream application, trying to utilize curl or its libraries, so that they are compliant with what the Tor project expects.
Show threadShawn WebbAug 31, 2023

@joubin @bagder @jordan Agreed. libcurl is just the wrong place for it. Placing the restriction (and making it non-optional) in libcurl is just too broad of a reach.

It's like banning all cars because one person yelled their SSN out the window of one. Perhaps we should educate the person and provide override-able safeguards for their particular car.

Performing the check at the application level makes the most sense. And it's easier to provide necessary toggles there, too.

Making the restriction non-optional also amounts to censorship. We need toggles to account for varying deployment methods and environments. Otherwise, our at-risk users will be put in harm's way. There is a real, tangible human cost here.

Show threadJoubin 🛡️Sep 3, 2023
You said agree, but what you’re saying wasn’t my intent. To me, before your reply, I was okay with libcurl doing this. But interesting response. @bagder, how do you feel if the default remained what you have today, but what @lattera is suggesting can be a global environment variable that must be explicitly set to get the old behavior back. Something like `CURL_ALLOW_INSECURE_TOR`?