How to Update All Packages in Python

Python is popular for its built-in bundled-up simplicity to achieve the most common tasks. Also, it offers a plethora of external packages (collection of modules and libraries) that can be imported and installed pertaining to particular use cases and the complexity of the solution. As this popular saying goes, “with power comes great responsibility”, use of more and more packages may lead to undesired dependency issues like dependency hell. Therefore, as we covered in our previous article, Managing Python Dependencies with Virtual Environments becomes inevitable. 

For this particular reason, the Python pip — used to install Python packages — does not have a built-in command to update all packages in one go. Instead, in an attempt to efficiently manage upgrades and avoid dependency conflicts, independent virtual environments should be used for each project.

In this article, we will discuss how to update all Python packages for a project and best practices to keep in mind when doing so.

Pinned and Unpinned Packages

Amongst the best practices to manage packages in Python is to pin all packages along with their version to the specific environment. This ensures that the environment is consistent and functional when reproduced later.

• Pinned packages are denoted by the “==” syntax in the requirements.txt file, such as requests==2.20.0. Such packages generally offer the core functionality of the solution and should only be updated in case of critical security bugs and vulnerability requirements.
• Unpinned packages are denoted by the “>=” syntax in the requirements.txt file. Such packages are okay to be updated to the latest versions for efficiency and performance improvements, in addition to new functionality

Most popular packages offer frequent updates to patch bugs and vulnerabilities. It is advised, concerning the security of a solution, to keep all packages up-to-date. The decision to update pinned packages lies solely with the development team as to what the newer version of the package entails. Security updates should surely be incorporated, whilst newer functionality may be ignored depending on the use case.

Pip is the de facto package management solution for installing updates system-wide whilst for virtual environments, the dedicated package manager of the virtual environment library should be used. As our team recommends the use of Pipenv for dependency management using virtual environments, the pipenv package manager is a suitable solution for updating all Python packages.

The Package Update Checklist

The following steps offer a general approach to updating all Python packages, irrespective of environment and operating system:

• Ensure Python and Pip are installed using the following commands:

python --version

pip --version

• Update Pip and Wheel to the latest version using:

pip install --upgrade pip wheel

• Fetch the list of outdated packages using:

pip list --outdated

• Update outdated packages.

We will now explore how to achieve the last step in different environments and operating systems.

Update Python Packages on Windows

Updating all Python packages on the Windows operating system is fairly straightforward.

Just follow these steps:

• Open up the Windows PowerShell by typing “powershell” in the search box of Taskbar or using the command prompt.
• Use Pip in conjunction with the Windows PowerShell to execute the following command:

pip freeze | %{$_.split('==')[0]} | %{pip install --upgrade $_}

Please note that this technique will update all the Python packages system-wide. The latest version will be mapped using the Python Package Index (PyPI).

Update Python Packages on Linux

Linux offers numerous approaches to updating all Python packages in one go. The two popular methods are to use Pip in conjunction with grep or awk:

• To use pip and grep, execute the following command:

pip3 list --outdated --format=freeze | grep -v '^\-e' | cut -d = -f 1 | xargs -n1 pip3 install -U

• To use pip and awk, execute the following command:

pip3 list -o | cut -f1 -d' ' | tr " " "\n" | awk '{if(NR>=3)print)' | cut -d' ' -f1 | xargs -n1 pip3 install -U

As is the case with Windows, this technique would update all Python packages system-wide using the Python Package Index (PyPI).

OS-Independent Upgrade of Python Packages

If you work with multiple operating systems seamlessly and efficiently and wish to explore an OS-independent technique, you may use the following workaround:

• Write a list of all installed packages into a requirements.txt file using:

pip freeze > requirements.txt

• Edit the requirements.txt file and unpin all packages by replacing “==” with “>=”. “Find and Replace All” functionality of common text editors like VS Code and Notepad might be useful here.
• Update all outdated packages using the following command:

pip install -r requirements.txt --upgrade

If you are working with the root Python installation, this technique would update the packages system-wide. If the requirements.txt file is linked to a particular virtual environment, only the packages in the virtual environment will be impacted.

Update Python Packages in Virtual Environment

You might have guessed by now that updating all Python packages in a virtual environment is as simple as updating the requirements.txt file, as discussed in the previous section.

Another approach is to use the following Python script to update all the unpinned packages:

import pkg_resources
from subprocess import call

for dist in pkg_resources.working_set:
    call("python -m pip install --upgrade " + dist.<projectname>, shell=True)

Update Python Packages in Pipenv Virtual Environment

As we actively vouch for the use of Pipenv for dependency management in Python, it would be unjust to not share the approach of updating all Python packages in a Pipenv virtual environment.

To update all unpinned packages in a specific Pipenv environment, follow these steps:

• Activate the pipenv shell containing the packages, using:

pipenv shell

• Update all packages using:

pipenv update

Conclusion

In this article, we explored numerous approaches to updating all Python packages for a project in a seamless and efficient way. We explored best practices of setting up pinned and unpinned packages for agility, and techniques to update the packages in individual and multiple environments, depending on the operating system and use of virtual environments. 

We hope this was an insightful read to achieve the simple task of updating packages without writing another long script off the internet or installing an external package to update them. Happy Coding!