Kaggle Notebook in VS Code

README.md

Connect VS Code to your Kaggle notebook kernel

This set of scripts creates a reverse tunnel to your notebook using Ngrok, and sets up a reverse proxy so we can use at least basic http auth to keep our kernel unexposed.

Steps

1. Create ngrok account and obtain api key
2. Generate credentials for basic auth in format: "username password" in ~/.ngrok_http
3. Store api key and credentials in Kaggle secrets
4. Run "in notebook.py" in notebook, and obtain url
5. Run jupyter-tunnel.sh
6. Switch to remote kernel in VS Code and paste url

in notebook.py

# Remote Jupyter Kernel for VS Code
!pip install ngrok
import ngrok, kaggle_secrets, os, re

user_secrets = kaggle_secrets.UserSecretsClient()

def get_first_base_url():
    with os.popen('ps aux') as f:
        for line in f:
            if 'NotebookApp.base_url' in line:
                match = re.search(r"--NotebookApp.base_url=([^ ]+)", line)
                if match:
                    return match.group(1).strip('"').strip("'")
    return None

serve = await ngrok.forward(8888, authtoken=user_secrets.get_secret("ngrok"), basic_auth=user_secrets.get_secret("http-auth"))
print(f"$ jupyter-tunnel.sh {serve.url()} {get_first_base_url()[:-1]}")
print(f"Paste to VS Code: http://127.0.0.1:6969/?token=\"\"")

jupyter-tunnel.sh

#!/bin/bash

# Check if minimum URL is provided as argument
if [ $# -lt 1 ]; then
    echo "Usage: $0 <URL> [PATH]"
    exit 1
fi

URL=$1
PATH_TO_APPEND=${2:-} # Optional path, defaults to empty if not provided

# Read credentials from ~/.ngrok_http
if [ ! -f ~/.ngrok_http ]; then
    echo "Error: ~/.ngrok_http not found. Make sure credentials are stored in the format 'username password'."
    exit 1
fi
read -r username password < ~/.ngrok_http

# Generate Basic authentication token
basic_auth_token=$(echo -n "$username:$password" | base64)

# Create a temporary Python script for mitmproxy
PY_SCRIPT=$(mktemp /tmp/inline_mitmproxy_XXXX.py)

# Generate the Python script with the provided path
cat > "$PY_SCRIPT" << EOF
from mitmproxy import http

def request(flow: http.HTTPFlow) -> None:
    # Append the provided path to all requests
    flow.request.path = "$PATH_TO_APPEND" + flow.request.path
EOF

# Start mitmproxy with the dynamically created Python script
mitmproxy --mode reverse:$URL --modify-headers ":~q:Authorization: Basic $basic_auth_token" --modify-headers ":~q:ngrok-skip-browser-warning: asdf" -s "$PY_SCRIPT" --listen-port 6969

# Clean up: Remove the temporary Python script after mitmproxy exits
rm "$PY_SCRIPT"