Package Management in Python

The majority of this talk is based on talk was based on the conda user guide and the Python Packaging Authority website.

This week Sarah Murphy (sarah.y.murphy@wsu.edu) gave a talk about managing packages and environments using conda and pip. A recording of the talk can be seen here.

Vocabulary

Library/Package - A collection of code (or modules) that you can import and use in your code.
- These can contain functions and constants.
Module - The files that make up a library.
- Note: People often use module and library interchangeably even though they are not the same thing.
- Many libraries are only one module.
Environment - An isolated Python distribution with its own installation and packages.
Package manager - Software for downloading and upgrading Python libraries.
Anaconda - A set of tools for coding (https://www.anaconda.com/).
conda - A package manager that comes with Anaconda.
pip - The Python package manager.

Pip vs Conda

The two most popular package managers for Python are pip and conda. Both have advantages and disadvantages, and it is likely you will need to use both. Pip typically comes with your Python download, so there is no need to download any additional software. Conda comes as part of Anaconda.

	Pip	Conda
Who created it?	Python Package Authority	Anaconda
Where are packages downloaded from?	Python Package Index	Anaconda repository and cloud
Can it include software written in other languages?	Not without downloading other things	Yes
Can it create environments?	Yes, with venv	Yes

Some of the key differences between pip and conda as package management software

A major reason for combining pip with conda is when one or more packages are only available to install via pip
The Anaconda Blog

Conda commands can be executed in the command line software of your choosing. As a Mac user, I always use Terminal for this. Alternatively, you can open a Spyder document and execute the commands from the console.

Pip commands can also be executed in the command line software of your choosing. You will notice the pip commands below say 'python3' followed by the rest of the command. On a Windows computer, you would replace 'python3' with 'py'.

Packages

For the sake of this demonstration, I will be using the commands as if they were typed into the Terminal on a Mac. Additionally, all commands below will use 'scipy' as the package of choice. Replace scipy with your package.

Installing Packages

sIt's easy to install package with both conda and pip. It's useful to know how to use both because some packages are only available via pip or an alternative channel through conda. A channel is a location that conda can look for code to download. Most of the time, you won't specify a channel with conda, meaning conda will search the default library for your code.

Conda

The command to install a package with conda is install:

conda install scipy

You can also list multiple packages after the install command to download them all.

If you'd like to install a specific version of a package, set the package name equal to the version you'd like.

conda install scipy = 0.15.0

There is the possibility that you may need to download a package from conda that is not provided by the default channel. One of the most popular alternative channels is conda-forge. To specify that we want to search a different channel, we will use the -c flag with the above commands and followed by the channel.

conda install -c conda-forge scipy

Pip

With pip, we use the command 'python3 -m pip install' to install packages. Note that this would be 'py -m pip install' on a Windows computer.

python3 -m pip install "scipy"

Just as with conda, there is the option to specify the package version.

python3 -m pip install "scipy = 0.15.0"

Searching for Packages

The search command can be used in both pip and conda to find information about packages that are both installed and not installed.

Conda

conda search scipy

Pip

pip search scipy

What packages do I already have installed?

You can see packages you have downloaded using the list command in conda and pip and the freeze command in pip. Both of these commands can be output to a file by using the greater than symbol (>) after the command followed by the output file name (conda list > packages.txt).

Conda

conda list

Conda's list command displays a table containing the package name, version, build, and channel. The channel is only displayed if it was downloaded from a channel other than the default.

Pip

pip list
pip freeze

Pip's list command displays a neat table of your package names and versions. Using freeze will display more information, including some file paths, but is not formatted in the same way.

Removing Packages

Removing a package is just as easy as installing it. You can use the remove command in conda and uninstall in pip.

Conda

conda remove scipy

Pip

class="wp-block-syntaxhighlighter-code">pip uninstall scipy ```

Updating Packages

You can update packages using the update command in conda and the install command with the upgrade flag in pip.

Conda

```bash conda update scipy ```

The update command is the same for updating conda, anaconda, spyder, and all other anaconda products, too!

```bash conda update conda conda update anaconda conda update spyder ```

In conda, we can pin packages. Pinning is when you tell conda not to update a package. I will go into more detail about how to pin packages in the next section. If you want to update a package that you've pinned, you can use the update command with the --no-pin flag.

class="wp-block-syntaxhighlighter-code">conda update scipy --no-pin ```

Pip

```bash pip install --upgrade scipy ```

Pinning Packages

Pinning is telling conda not to update a package. To do this, you must create a text document in the 'conda-meta' directory. Mine is located here:

/Users/smurphy/opt/anaconda3/conda-meta

This file must be named 'pinned'. List the packages and versions in this document, with each on its own line.

An example of the 'pinned' text file in the 'conda-meta' directory

You can either set a package to stay at a specific version (scipy in the above image, will stay at version 1.14.2), or not update out of a series (numpy in the above image, will update throughout all of 1.7, but will not update to 1.8).

If you have multiple environments, which I'll discuss in the following sections, you can have a pinned file for each environment in the environment's conda-meta directory.

Environments

Environments allow users to have separate installations of python with isolated installations of libraries. Starting a new environment will essentially start you a clean version of python with nothing downloaded without disrupting other versions.

The examples below show how to create a new environment named "myenv." Change "myenv" to whatever you'd like your environment to be called.

Creating an Environment

Conda

We can use the create command in conda to create a new environment.

```bash conda create -n myenv ```

If we would like a past version of Python we can specify it the same way we specify package versions.

class="wp-block-syntaxhighlighter-code">conda create -n myenv python=3.6 ```

Pip

Pip uses venv to create virtual environments and does not have the ability to specify Python version within the create command.

```bash py -m venv myenv ```

Activating and Deactivating Environments

To use an environment, you must activate it through the command line. If you do not activate an environment, you are using the default environment, and all installations will only apply to the default environment. Deactivate an environment to return to your default environment. Activating and deactivating can be thought of like logging into and out of environments you've created.

Conda

To activate your environment in conda, use the activate command.

```bash conda activate myenv ```

With older verisions of Python and conda, you may need to use source in the place of conda.

source activate myenv ```

It is likely both of these commands will work for you.

To deactivate environments in conda, we use the deactivate command.

```bash conda deactivate ```

Pip

Activating environments with pip is different for Mac and Windows users. For Mac users we use the source command followed by the environment name and '/bin/activate'.

```bash source myenv/bin/activate ```

For Windows users we use the following command, replacing myenv with the name of your environment.

```bash .\myenv\Scripts\activate ```

To deactivate, we can just type deactivate.

```bash deactivate ```

If you use multiple computers or often share code with a colleague, it may be helpful for you to create a standardized environment. You can do this by creating an environment.yml file for conda to use when creating a new environment.

The environment.yml file is text file. This file can be created anywhere in your system, but you must be sure to either create the new environment from the directory with this file or use the file path in the command.

An example environment.yml file

The example file above would create an environment named 'stats2'. The dependencies listed are the packages to install. Some packages have specified versions to download. Those without will download the most recent release.

Once you have an environment.yml file, you can use the following code to create the environment:

```bash conda env create -f environment.yml ```

Now, you can use the environment 'stats2' by activating it just as we did before. If you'd like to share this environment, just send your environment.yml file. Your collaborators will be able to create a new environment with it just like you did.

Summary of Commands

Packages

	Pip	Conda
Installing	pip install scipy	conda install scipy
Updating	pip install --upgrade scipy	conda update scipy
Removing	pip uninstall scipy	conda remove scipy

A summary of the most common package commands using pip and conda

Environments

	Pip	Conda
Creating	python3 -m venv myenv	conda create -n myenv
Activating	Run the activate file	conda activate myenv
Deactivating	deactivate	conda deactivate

A summary of the most common environment commands using pip and conda

Additional Resources

If you'd like information about package management using the Anaconda Navigator, I touch on this briefly near the end of the YouTube video.

References

- Conda user guide - Python Packaging Authority website.

python