How can I install Python packages to a Conda environment on an Amazon SageMaker notebook instance?
"I want to install Python packages to a specific Conda environment, check installed package versions, or create a persistent Conda environment."
"I want to install Python packages to a specific Conda environment, check installed package versions, or create a persistent Conda environment.ResolutionInstall Python packages to a specific Conda environmentIf you use pip or Conda to install Python libraries on the terminal without specifying the correct Conda environment, you get a ModuleNotFoundError when importing that Python package to your running notebook. This is because you're not installing the Python packages in the correct Conda environment. To install the Python packages in the correct Conda environment, first activate the environment before running pip install or conda install from the terminal.For example:sh-4.2$ source activate python3(python3) sh-4.2$ pip install theano(python3) sh-4.2$ source deactivate(JupyterSystemEnv) sh-4.2$To run this command in a notebook cell, add an exclamation point ("!") at the beginning of the command. This forces the command to run as a shell command from the notebook and assures that the package is installed in the current Jupyter kernel.For example:import sys!conda install -y --prefix {sys.prefix} theanoNote: When you run conda install in a notebook cell, you can't enter an interactive response. To install packages in a notebook cell using Conda, you must explicitly pass -y. Otherwise, the command hangs and waits for user confirmation.Or, use pip install:import sys!{sys.executable} -m pip install theanoSometimes, pip might fail to install some of the package's dependencies. When this happens, use Conda to install packages instead of pip. Conda verifies that all required components are satisfied before installing the packages. For more information, see Understanding Conda and Pip in the Conda documentation.Other useful commandsTo see the pre-built Conda environment, run either of the following commands in the notebook instance terminal:$ conda env list$ conda info --envsExample output:# conda environments:#base                     /home/ec2-user/anaconda3JupyterSystemEnv      *  /home/ec2-user/anaconda3/envs/JupyterSystemEnvR                        /home/ec2-user/anaconda3/envs/Ramazonei_mxnet_p27       /home/ec2-user/anaconda3/envs/amazonei_mxnet_p27amazonei_mxnet_p36       /home/ec2-user/anaconda3/envs/amazonei_mxnet_p36amazonei_tensorflow_p27     /home/ec2-user/anaconda3/envs/amazonei_tensorflow_p27amazonei_tensorflow_p36     /home/ec2-user/anaconda3/envs/amazonei_tensorflow_p36chainer_p27              /home/ec2-user/anaconda3/envs/chainer_p27chainer_p36              /home/ec2-user/anaconda3/envs/chainer_p36mxnet_p27                /home/ec2-user/anaconda3/envs/mxnet_p27mxnet_p36                /home/ec2-user/anaconda3/envs/mxnet_p36python2                  /home/ec2-user/anaconda3/envs/python2python3                  /home/ec2-user/anaconda3/envs/python3pytorch_p27              /home/ec2-user/anaconda3/envs/pytorch_p27pytorch_p36              /home/ec2-user/anaconda3/envs/pytorch_p36tensorflow_p27           /home/ec2-user/anaconda3/envs/tensorflow_p27tensorflow_p36           /home/ec2-user/anaconda3/envs/tensorflow_p36To see the kernels that are installed on the notebook, run this command:sh-4.2$ ipython kernelspec listTo check the version of a package that's installed in a Conda environment, run this command in the notebook instance terminal:(python3) sh-4.2$ pip freeze | grep pandasOr, check the package version in the notebook cell using this command:import pandas as pdpd.__version__Create a persistent Conda environmentWhen you stop a notebook, SageMaker terminates the notebook's Amazon Elastic Compute Cloud (Amazon EC2) instance. Packages that are installed in the Conda environment don't persist between sessions. The /home/ec2-user/SageMaker directory is the only path that persists between notebook instance sessions. This is the directory for the notebook's Amazon Elastic Block Store (Amazon EBS) volume. If you want your libraries to persist between sessions, see How can I be sure that manually installed libraries persist in Amazon SageMaker if my lifecycle configuration times out when I try to install the libraries?Related informationInstall external libraries and kernels in notebook instancesCustomize a notebook instance using a lifecycle configuration scriptFollow"