jupyter-notebook

Jupyter Notebook Expert Skill

This skill provides a guide for Jupyter Notebook execution.

1. Databricks Jupyter Kernel

https://github.com/i9wa4/jupyter-databricks-kernel

uv pip install jupyter-databricks-kernel
uv run python -m jupyter_databricks_kernel.install

2. Default Execution Method

When instructed to execute an entire notebook, use this command:

uv run jupyter execute <notebook_path> --inplace --timeout=300

3. Execute with Databricks Kernel

When running notebook on Databricks cluster:

uv run jupyter execute <notebook_path> --inplace --kernel_name=databricks --timeout=300

Required environment variables:

• DATABRICKS_HOST: Databricks workspace URL
• DATABRICKS_TOKEN: Personal Access Token
• DATABRICKS_CLUSTER_ID: Cluster ID

4. Usage Examples

# Execute with local Python kernel
uv run jupyter execute /workspace/notebooks/sample.ipynb --inplace --timeout=300

# Execute with Databricks kernel
uv run jupyter execute /workspace/notebooks/databricks-sample.ipynb --inplace --kernel_name=databricks --timeout=300

5. Option Descriptions

• --inplace: Overwrite original file with execution results
• --kernel_name=<name>: Specify kernel to use (databricks, python3, etc.)
• --timeout=<seconds>: Set timeout in seconds (-1 for unlimited)
• --startup_timeout=<seconds>: Kernel startup timeout (default 60 seconds)
• --allow-errors: Continue execution to end even with errors

6. Notes

• Verify required environment variables are properly set before execution
• Adjust --timeout value for long-running cells
• If open in VS Code, verify file updates after execution
• For Databricks kernel, cluster startup takes 5-6 minutes if stopped

7. Reference Links

• jupyter-databricks-kernel: https://github.com/i9wa4/jupyter-databricks-kernel
• Jupyter nbclient: https://nbclient.readthedocs.io/