[python] jupyter notebook kernel cannot be loaded

Unable to load jupyter notebook kernel

You can easily change the virtual environment in jupyter notebook, but I couldn’t switch the environment well this time, so I’ll write it down.

github

• The file in jupyter notebook format on github is here.

google colaboratory

• If you want to run it on google colaboratory here 015/015_nb.ipynb)

Author’s environment

!sw_vers

ProductName: Mac OS X
ProductVersion: 10.14.6
BuildVersion: 18G9323

!Python -V

Python 3.8.5

Kernel check command

Normally, if you want to see the actual python execution environment in jupyter notebook, you would run the following command. However, in this case, the desired python version was specified, but it did not work in practice.

import sys
sys.executable

'/Users/hiroshi.wayama/anaconda3/envs/lgbm2/bin/python'

Assignment

I have set up lgbm on jupyter notebook, but when I actually run it, I find that python in that environment is not running.

List of kernels being configured

You can check the list of kernels you are using with the following command.

jupyter kernelspec list

Available kernels:
  julia-1.7 /Users/hiroshi.wayama/Library/Jupyter/kernels/julia-1.7
  lgbm /Users/hiroshi.wayama/Library/Jupyter/kernels/lgbm
  lgbm2 /Users/hiroshi.wayama/Library/Jupyter/kernels/lgbm2
  my_environment_name /Users/hiroshi.wayama/Library/Jupyter/kernels/my_environment_name
  reco_base /Users/hiroshi.wayama/Library/Jupyter/kernels/reco_base
  python3 /Users/hiroshi.wayama/anaconda3/share/jupyter/kernels/python3

Kernel configuration file

The configuration files for each kernel are stored in ~/Library/Jupyter/kernels/. In this one, lgbm2 is set as the kernel.

ls -alh /Users/hiroshi.wayama/Library/Jupyter/kernels/

total 0
drwxr-xr-x 7 hiroshi.wayama staff 224B 2 7 17:18 [34m.[m[m
drwxr-xr-x 8 hiroshi.wayama staff 256B 2 7 19:04 [34m. [m[m
drwxr-xr-x 5 hiroshi.wayama staff 160B 12 8 10:18 [34mjulia-1.7[m[m
drwxr-xr-x 5 hiroshi.wayama staff 160B 2 7 17:18 [34mlgbm[m[m
drwxr-xr-x 5 hiroshi.wayama staff 160B 2 7 16:04 [34mlgbm2[m[m
drwxr-xr-x 5 hiroshi.wayama staff 160B 8 24 00:09 [34mmy_environment_name[m[m
drwxr-xr-x 5 hiroshi.wayama staff 160B 8 24 09:25 [34mreco_base[m[m

I’m setting up lgbm2 as a kernel in this file, but it doesn’t work. Try to open that file.

!cat /Users/hiroshi.wayama/Library/Jupyter/kernels/lgbm/kernel.json

{
 "argv": [
  "/Users/hiroshi.wayama/anaconda3/bin/python",
  "-m",
  "ipykernel_launcher",
  "-f",
  "{connection_file}"
 ],
 "display_name": "lgbm",
 "language": "python",
 "metadata": {
  "debugger": true
 }
}

The default path to python is /Users/hiroshi.wayama/anaconda3/bin/python, not /Users/hiroshi.wayama/anaconda3/envs/lgbm/bin/python as we expected. This is not the case. Open lgbm2, which is working fine. You can see that it is set up as expected.

! cat /Users/hiroshi.wayama/Library/Jupyter/kernels/lgbm2/kernel.json

{
 "argv": [
  "/Users/hiroshi.wayama/anaconda3/envs/lgbm2/bin/python",
  "-m",
  "ipykernel_launcher",
  "-f",
  "{connection_file}"
 ],
 "display_name": "python-lgbm2",
 "language": "python",
 "metadata": {
  "debugger": true
 }
}

I replaced the path with the python path of the environment I wanted to use and it worked. I’ll keep this in mind as I don’t want to spend too much time on this.