Python packaging tool idea: "oops, all editable".

There's been a big move in the community towards highly structured, "binary" artifacts for installation in the last few years. But they're all still source, just source in a highly inconvenient format for editing because they're detached from version control.

What if `oopsae install -r requirements.txt` checked out all the repos and installed them into your venv as editable so you could make changes into any upstream you want and just push them

@glyph I've... thought about doing something like this. But you'd need some heuristic to guess what the source repo is, and – worse – to guess which git commit corresponds to the pinned version.

Idk maybe 90% is good enough? I get nervous about incomprehensible issues caused by tags being slightly off, but maybe I'm just being over-cautious...

@njs @glyph I think you'd get pretty far with that for most packages. We've just started using some automation at work to try and figure out OSS package versions from a given Github hash. For other good reasons: You need the version number to tie to vulnerability reports which often declare only human versions rather than source control hashes.
@njs @glyph I'm not sure what it is based on, I'll ask. It seems like a tooling project that should be open source already.

@njs @glyph Asking around, it seems based on our https://deps.dev/ data (neat data, but it doesn't currently appear to contain enough to correlate package versions to source control ids) combined with code that isn't currently open built on top to try and correlate. I hope it is or the deps data is improved to include it someday, but I imagine we'd all build our own the same way: heuristics possibly including attempts at correlate files within packages.

I'd really like it if Python packaging tooling and metadata officially recorded the source control revision and embedded that in the package when built from an unmodified checkout. I'm guessing we currently do not?

#python #packaging #internals #SourceOfTruth

Open Source Insights

@gpshead @njs @glyph I hope in future it will be able to corelate data for python modules written in rust :)