When working with Jupyter Notebooks in SageMaker workspaces, you may want to create a custom kernel for your R or python notebooks. This will prevent you from needing to install and load packages ever
Step 1: Create a SageMaker workspace using one of the kernels workspace types
When you create your SageMaker workpace, you will see different options for workspace configuration available. Select one with the word "kernels", such as "Sagemaker-Custom-Kernels-Medium".
Step 2: Open a Terminal window
Step 3: Log in as an administrator by running sudo su ec2-user
Step 4: Create a new kernel
Create a directory to store your custom kernels in: mkdir /home/ec2-user/SageMaker/kernels/
View existing kernels: conda env list
Create new kernel (replace my_new_kernel with kernel name): mamba create -q -y --prefix “/home/ec2-user/SageMaker/kernels/my_new_kernel”
Step 5: Modify a kernel
Activate a kernel (replace my_new_kernel with kernel name): conda activate /home/ec2-user/SageMaker/kernels/my-new-kernel
To create an R kernel: mamba install r-irkernel
To create a python kernel: mamba install ipykernel
Add a package to a kernel: mamba install <package name>
*note that R packages are often prefixed with r-. For example, r-tidyverse
Deactivate the kernel: conda deactivate
Step 6: Link the kernel to the conda environments
Link the kernel (replace my_new_kernel with kernel name): ln -s "/home/ec2-user/SageMaker/kernels/my-new-kernel" "/home/ec2-user/anaconda3/envs/my-new-kernel"
Step 7: Use the kernel
Your new kernel will now show up in your kernels list when creating a new notebook or modifying an existing notebook.
The kernel will persist through workspace start and stop, meaning you do not have to re-install packages. However, you will need to re-link your kernel upon workspace start. To do so, open a terminal window and re-run step 6 above.
Existing custom kernels
Rapids AI: Rapids AI is a tool that allows faster processing speed in machine learning algorithms. We have deployed a new kernel for Sagemaker notebooks specifically for Rapids AI in SWB (conda_rapids_23.06).
