Ben Lindsay#Python question: When installing python using the official downloader on python.org, it sets up a symlink from /usr/local/bin/python3 to /usr/local/bin/python3.11, but it doesn't add a symlink from /usr/local/bin/python to /usr/local/bin/python3. Does anyone know why not? I'm an experienced python user, but this kind of threw me for a loop, because after doing the official python installation, `which python` still shows /usr/bin/python
For people who use this method, (@brianokken I think I've heard you promote it?) do you add that python ->python3 symlink manually or just type python3 everywhere or something else? I feel like I shouldn't have to type python3, since that's the default assumption these days, but I'm not sure if adding a symlink in /usr/local/bin will have side effects with the python installer or #homebrew or something else
Ed 👾🎮💻@benlindsay I usually create an alias to solve this problem.
@ede That makes sense. The part that gets me though is that I keep hearing the recommendation to use the official installer as the thing to recommend to beginners to help them fall into the "pit of success". Needing to create an alias for "python" to work instead of "python3" doesn't seem to fit that story
@benlindsay I think the problem is Python versioning and what ships with operating systems by default.
Many older programs may assume /usr/bin/python to be version 2 whereas /usr/bin/python3 is typically where Python 3 is installed and how it’s referenced. This includes PIP as well.
Typically, specific Python 3 versions (ie. 3.11) will symlink from /usr/bin/python3.11 to /usr/bin/python3.
Many older programs may assume /usr/bin/python to be version 2 whereas /usr/bin/python3 is typically where Python 3 is installed and how it’s referenced. This includes PIP as well.
Typically, specific Python 3 versions (ie. 3.11) will symlink from /usr/bin/python3.11 to /usr/bin/python3.
@ede That makes sense. But I think it's fair to assume that people working with legacy python2 stuff are more likely to be experienced devs who can figure out all the symlinking and version management stuff themselves, while new users are likely working with newer code where python 3 should be the default. The installation wizard should make python point to python3 by default and allow users to opt out
@benlindsay I think there may be some dependencies that may break if that happens.
Python 3 is not backwards compatible with 2 or earlier, so if a dependency exists in the environment, it may cause unexpected failures.
For example, macOS ships with Python 2.x. Replacing the symlink may cause a break for any macOS dependencies on Python 2.x.
Python 3 is not backwards compatible with 2 or earlier, so if a dependency exists in the environment, it may cause unexpected failures.
For example, macOS ships with Python 2.x. Replacing the symlink may cause a break for any macOS dependencies on Python 2.x.
Brian Okken@ede @benlindsay yep. I’m pretty sure that’s the reason. Newer macs don’t have the problem, so maybe a request to Python is in order.
It “should” only be an issue when creating virtual environments. Once in an active venv, the python link exists.
I actually like the absence of a global symlink, because it acts as a warning that I’m not in a virtual environment.
And even for new users, we should be teaching virtual environments, IMHO
It “should” only be an issue when creating virtual environments. Once in an active venv, the python link exists.
I actually like the absence of a global symlink, because it acts as a warning that I’m not in a virtual environment.
And even for new users, we should be teaching virtual environments, IMHO
@ede @benlindsay However, @brettcannon or other core devs might have better insight into the missing python symlink on macs.
@brianokken @ede @brettcannon Agreed about the importance of using and teaching virtual environments. Not sure I agree that the missing symlink is a useful warning...without the symlink, "python -m pip install" will install to the system python, and I think that's maybe more important to provide warnings or protections to avoid
Exploratory Data Alex@benlindsay @brianokken @ede @brettcannon I guess I'm a contrarian on virtual envs, but I stopped using them a year ago and the sky hasn't fallen for me. And for beginners, I think they add more complexity to the experience than the problem they solve is worth. Meanwhile pip has also gotten a *lot* better over the last couple years.
@benlindsay @brianokken @ede @brettcannon But this thread does make me wonder how much of the motivation to use virtual envs is coming from Mac-specific issues.
Brett Cannon@alexkyllo @benlindsay @brianokken @ede Virtual environment are most definitely **not** motivated by macOS specifically. You've just been lucky that it hasn't bit you yet (speaking as someone who has been coding in Python before environments existed).
And I'm not touching the education angle on this one as there are arguments on both sides. Either you want to keep it simple and assume they are going to break their install anyway, or you want to start with best practices.
@brettcannon @benlindsay @brianokken @ede I've hit incompatibilities before, but I know how to resolve them and will use a virtualenv when necessary. It just usually isn't a problem for me and I don't think the reason is luck, but that I mostly use the same small set of compatible packages in each project, I want to use the latest releases of them, and most of my projects don't need long-term maintenance. I think these criteria apply to a lot of data scientists due to the nature of our work.
@brettcannon @benlindsay @brianokken @ede In other words, I don't really need like 87 copies of various versions of jupyter, numpy, pandas, matplotlib, seaborn, statsmodels, and scikit-learn on my system. These are stable packages that are designed to be compatible. And if they deprecate a function I used in an old project, it's fine because I'm willing to update my code if I need to use it again.
@alexkyllo @benlindsay @brianokken @ede If it's disk space you're concerned about then now you get to blame Windows as pip can't assume symlink support and so it has to install everything separately 😉
@brettcannon @benlindsay @brianokken @ede Disk space is part of it, but the bigger issue was the clutter of having dozens of venvs scattered around my machine, time spent waiting for installs, and overhead of making sure to always have the right ones activated in my IDE and terminal for each of the several projects I have open at any given time. Using them has benefits but also costs, so I don't blanket recommend others always use them--to me, it depends on the nature of your projects.
@alexkyllo @benlindsay @brianokken @ede If you would like your environments centralized there are various tools to support that workflow.
And I hear there's an editor that will simply remember the environment you select so it's a one-time cost (if not zero-cost if you keep the environment in your workspace) 😉
But I get it, environments are an extra step. Totally fine that it works for you, I just don't want to promote not using environments as a general solution for everyone.
@alexkyllo @benlindsay @brianokken @ede I think the fact you know how to resolve them also automatically makes you advanced. And as you said, the nature of your work may make you less susceptible to breakage, but I personally can't make that assumption is true for most Python users, nor do I want someone to come back to me later and say, "why didn't you warn me this could happen?!?"