Using custom Jupyter Notebook kernels

When working with Jupyter Notebooks in SageMaker workspaces, you may want to create a custom kernel for your R or python notebooks. This will prevent you from needing to install and load packages ever

Creating a custom kernel

Step 1: Create a SageMaker workspace using one of the kernels workspace types

When you create your SageMaker workpace, you will see different options for workspace configuration available. Select one with the word "kernels", such as "Sagemaker-Custom-Kernels-Medium".

Step 2: Open a Terminal window

Step 3: Log in as an administrator by running sudo su ec2-user

Step 4: Create a new kernel

Create a directory to store your custom kernels in: mkdir /home/ec2-user/SageMaker/kernels/

View existing kernels: conda env list

Create new kernel (replace my_new_kernel with kernel name): mamba create -q -y --prefix “/home/ec2-user/SageMaker/kernels/my_new_kernel”

Step 5: Modify a kernel

Activate a kernel (replace my_new_kernel with kernel name): conda activate /home/ec2-user/SageMaker/kernels/my-new-kernel

To create an R kernel: mamba install r-irkernel

To create a python kernel: mamba install ipykernel

Add a package to a kernel: mamba install <package name>

*note that R packages are often prefixed with r-. For example, r-tidyverse

Deactivate the kernel: conda deactivate

Step 6: Link the kernel to the conda environments

Link the kernel (replace my_new_kernel with kernel name): ln -s "/home/ec2-user/SageMaker/kernels/my-new-kernel" "/home/ec2-user/anaconda3/envs/my-new-kernel"

Step 7: Use the kernel

Your new kernel will now show up in your kernels list when creating a new notebook or modifying an existing notebook.

The kernel will persist through workspace start and stop, meaning you do not have to re-install packages. However, you will need to re-link your kernel upon workspace start. To do so, open a terminal window and re-run step 6 above.

Existing custom kernels

• Rapids AI: Rapids AI is a tool that allows faster processing speed in machine learning algorithms. We have deployed a new kernel for Sagemaker notebooks specifically for Rapids AI in SWB (conda_rapids_23.06).