With over 600,000 projects on PyPI (and counting), managing Python dependencies can be tricky. One package may require pandas 1.3.0, while another demands pandas 2.0.0. Installing one version may break another, bringing your project to a grinding halt. Wouldn’t it be great if we could create a Python environment with only the packages and versions we need? Enter: virtual environments.
What is a virtual environment and why should I use it?
A virtual environment is a self-contained Python setup with its own interpreter and dependencies, isolated from the system-wide Python installation. Its packages won’t interfere with the system-wide Python installation or other python virtual environments. Think of it like Vegas: what’s installed in the virtual environment stays in the virtual environment.
It’s generally a good practice to avoid installing many packages to the system-wide Python installation due to the potential for incompatibilities. While it’s convenient to have a mega all-in-one environment that does everything you need, the reality is it might be impossible, and it comes with risks. Instead, you should create a virtual environment that has the compatible packages that you need. Some reasons to create a virtual environment are:
- System Integrity – Maintain Stability
Some OS’s rely on specific Python versions and libraries for certain tools (e.g. yum, apt, system scripts, etc.). Modifying the system-wide Python installation can break these tools, leading to unexpected failures. - Experimentation – Isolate Development Environments
You want to build an experimental sandbox with its own packages to test compatibility with different packages or upgraded versions without risking the main environment. - Reproducibility – Create Consistent Results
A virtual environment can easily be re-created with requirements.txt and return consistent results, ensuring that collaborators, CI/CD pipelines, and production environments all use the same dependencies and versions. - Efficiency – Reduce Deployment Overhead
You can identify the exact minimal number of packages needed for a particular project to run, then move that into production in as small of a package as possible. This leads to smaller Docker images, faster builds, and improved performance. - Security – Minimize Risk Exposure
The fewer packages you install, the smaller your attack surface. Keeping dependencies minimal reduces the chance of introducing security vulnerabilities from third-party libraries, lowering the risk of exposure to future CVEs.
Now that we have some good reasons to create a virtual environment, let’s talk about how to do it. My chosen platform is SAS Viya Workbench. These instructions will also apply to many other instances, such as a local Python install. If you are using Windows, be sure to use a backslash ($$ rather than a forward slash (/) for your directories.
Creating your first virtual environment
Virtual Environments are created and managed through terminal or command prompt, just like your Python packages. You have a few ways to get there. For this example, we’ll be creating it through Terminal from Jupyter Lab on Workbench, but you’re more than welcome to follow along from VS Code’s terminal or your local terminal/command prompt and get the same results.
First, get to your terminal by opening Jupyter Lab:
Then select Terminal:
Now let’s create our virtual environment. Start by creating a new hidden folder in your workspace called .venv and cd into it:
mkdir .venv
cd .venv
You can create a virtual environment anywhere in any folder with any name, but this keeps it nice and tidy. The dot (.) makes the folder hidden on Linux, keeping your workspace clean. Some tools (like VS Code) automatically detect virtual environments named .venv and give them a cool little icon.
Now we’re ready to create the virtual environment. For this, we have two options: we can either create it with all of the packages that come with your system-wide install, and in my case, the packages for SAS Viya’s Python API, or we can create it as a totally empty, fully independent environment.
Option 1: Including all packages on the system
If you want to include all the packages that already come on the system, use this command:
python -m venv –-system-site-packages myenv
Including system-wide packages ensures access to SAS Viya’s Python API and already includes several useful open source packages, but it also means your environment isn’t fully isolated; we’ll talk about how to address this in the package installation section below. If a system package is updated, your virtual environment may be affected. If you need full control over dependencies, you may prefer a clean environment.
Option 2: Creating a clean environment
To create a completely clean, fully isolated environment, use this command:
python -m venv myenv
Keep in mind if you do not include the system site packages on a virtual environment in Workbench, you will lose access to the SAS Viya Python API.
Installing packages to a virtual environment
To install packages, you need to activate the virtual environment. To do this, run the following command:
source ./myenv/bin/activate
On some operating systems, such as Windows, you may find this in the Scripts folder instead as a .bat file:
.\myenv\scripts\activate.bat
If successful, your terminal should show the name of the virtual environment like this:
myenv >
Or in Windows command prompt, like this:
(myenv) C:\users\username\.venv>
You can confirm what packages are installed on your environment with pip list –verbose. From here, it’s a matter of installing the packages you want with pip:
pip install
Or if you already have requirements.txt:
pip install -r "/path/to/requirements.txt"
If you’re wondering how to easily create a requirements.txt file, check out the links to pip freeze and pip requirements in the Learn more section at the end of this article.
If you used –-system-site-packages, you can install packages directly within the environment even if a system-wide version exists by using the -I option:
pip install -I
Or:
pip install -I -r "/path/to/requirements.txt"
When you install packages this way, the Python interpreter will first look in the virtual environment’s package directory and use it instead of the system-wide version. You can confirm whether a package is a system package or a virtual environment package with pip:
pip list --verbose
Let’s test this out by installing pandas to the virtual environment. Notice that the current output from pip points to the system-wide install (your version of pandas may differ):
Package Version Location Installer--------------------------- --------------- --------------------------------------------------------------- ---------pandas 2.2.3 /usr/local/lib64/python3.11/site-packages
Let’s install the same version of pandas with the -I option and look at the output:
pip install -I pandas==2.2.3
pip list --verbose
Package Version Location Installer--------------------------- --------------- --------------------------------------------------------------- ---------pandas 2.2.3 /workspaces/myfolder/.venv/py311/lib64/myenv/site-packages pip
Note that it is now pointing to myenv rather than the system python install. When you’re done, deactivate the environment:
deactivate
When you need to install or update packages in the future, remember to always activate your virtual environment first before using pip or you may accidentally install to the system-wide environment. Always double-check that you’re in the right virtual environment by looking at the environment name being shown in terminal.
Using the virtual environment
Accessing and using your virtual environment depends on where you are programming. Let’s walk through the steps on how to make it available in Jupyter notebook and VS Code.
Jupyter Notebook
When you’re working in Jupyter Notebook, you need to create a new kernel from the virtual environment. To do this, run the below commands:
source ./myenv/bin/activate
python -m ipykernel install --prefix $PYTHONUSERBASE --name myenv --display-name "My Virtual Environment"
Note: If you created a clean virtual environment, you will need to pip install ipykernel first:
pip install -I ipykernel
If successful, you should see it in Jupyter Lab and be able to select it from a Jupyter notebook.
VS Code
VS Code has an easy, built-in way of both creating and selecting Python virtual environments. It’s well-built around this concept. Note that when you start VS Code, you’ll see .venv in Explorer. It has a little Python icon so you know it’s for Python virtual environments. Isn’t that nice?
To make the environment available in VS Code, first press Ctrl + Shift + P to open the command palette and search for select interpreter. Click on Python: Select Interpreter.
Click on Enter Virtual Environment Path and either type the path or browse to select the Python interpreter. This is located at …/.venv/myenv/bin/python. On Windows, it may be located in …\.venv\myenv\scripts\python.exe.
When you select a virtual environment in VS Code, you will see myenv in the list. Simply select it to use it. You may first need to choose Select Another Kernel to use it, but after that it should show up in the standard list.
Select it and start coding away. If you want to double-check that it’s actually selected, run the Python code below and check that the path is to your virtual environment.
import sys
print(sys.executable)
You should see /.venv/myenv in the path:
/workspaces/myfolder/.venv/myenv/bin/python
That’s all there is to it!
Deleting a virtual environment
To delete a virtual environment, simply delete the folder in VS Code explore, terminal, command prompt, or from a file browser. That’s the beauty of virtual environments: everything is packaged neatly in a folder. It’s easy to maintain, easy to get rid of, and easy to recreate if you break it.
Conclusion
Creating virtual environments is a critical part of programming in Python. We’ve walked through the steps on how to create, manage, access, and delete them using SAS Viya Workbench. This is also applicable to your local environment on your PC as well or other similar platforms that support Python. This is just one built-in way to manage virtual environments with Python. There are other packages out there with different features that can assist in working with virtual environments, such as virtualenv. In the end, how you manage your virtual environments is up to you. The important thing is to create and manage them effectively.
