Andrew Nesbitt

Downstream testing for language pacakge managers, it's buildable but perhaps not fundable?

https://nesbitt.io/2026/03/01/downstream-testing.html

Downstream Testing

Most library maintainers have no way to test against their dependents before releasing.

Andrew Nesbitt
Mar 1 at 1:55pmWeb
Show threadThomas DepierreMar 1
@andrewnez can I point cargo-semver-check? It is not *exactly* what you are talking about here, but I think it helps a lot with that problem and allows to push it back upstream with less stuff to run.
Show threadAndrew NesbittMar 1
@Di4na if we had that in every other language, we’d be in a much better place
Show threadThomas DepierreMar 1
@andrewnez so funnily enough, iirc they mentionned they had a prototype for python and ruby iirc... but no funding to do all the work to productivize it
Show threadJames Mar 1

@andrewnez Excited to see https://dependabot.ecosyste.ms/. The Dependabot compatibility score used to be more accessible before GitHub took it over.

Any early indication of problems in a release is very freeing particularly as a solo maintainer when development time is limited.

Open Source Dependabot Pull Request Tracker | Ecosyste.ms: Dependabot

Track Dependabot pull requests across open source repositories. Discover security updates, dependency changes, and automation insights for package maintainers and security researchers.

Show threadJacques ChesterMar 1

@andrewnez there’s an interesting intersection with resolution policy here.

Because bundler/rubygems always try to resolve every dep to a single version, a feedback loop occurs where breakage upstream quickly attracts patches that mostly re-level the ecosystem.

But in systems allowing multiple versions to be resolved, that feedback loop is largely absent and so the spread of versions in use is more heavy tailed.

Show threadJacques ChesterMar 1
@andrewnez I also just remembered that in my time at Shopify I wanted to build a kind of progressive rollout system for new versions. Working from low-impact repos up to The Monolith. Integrated with the ShipIt deployment tool as well to get traffic control. Got laid off before I could pitch it properly 🤷
Show threadJamie GaskinsMar 1
@jacques @andrewnez Oh, man, same. I worked in Resiliency and tried to get Dev Infra on board with this. I got cut before I could get far enough, though.
Show threadJacques ChesterMar 1
@jamie @andrewnez ugh
Show threadDirkjan OchtmanMar 1
@andrewnez also a little relevant maybe is cargo-semver-checks, as a way of at least trying to make it easier for maintainers to catch semver-incompatible changes. Leverages (1) the ability to encode API invariants in the type system and (2) fairly strict semver praxis in the crates.io ecosystem.