Skip to content

Instantly share code, notes, and snippets.

@kinchungwong
Last active October 1, 2023 00:05
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save kinchungwong/824f894e3c776a624e08b0e20352896f to your computer and use it in GitHub Desktop.
Save kinchungwong/824f894e3c776a624e08b0e20352896f to your computer and use it in GitHub Desktop.
Helper functions for running Python homework or project with multiple source files on Google Colab

Helper functions for running Python homework or project with multiple source files on Google Colab

The first code cell imports dependencies, defines some project-specific information, and provides some utility functions.

Project-specific information, such as asset URLs and Google Drive directories, should be updated depending on the course, the year-session, homework or project number, the name of the archive, and the compression method. A different command may be needed for zip files.

After running the first code cell, six more code cells should be added, each containing one of these commands:


# Connect to your Google Drive. A web popup will ask for the necessary permissions.
#
connect_to_drive()

# Download the project starter files from the URL and extract them into the Google Colab workspace.
#
extract_project_template()

# Copy Python source files from your Google Drive into the Google Colab workspace.
# (Warning: files will be overwritten without confirmation.)
#
restore_files()

# Copy Python source files from your Google Colab workspace into your Google Drive.
# (Warning: files will be overwritten without confirmation.)
#
backup_files()

# Change the working directory to where the homework / project files are.
#
change_runtime_dir()

# Run the main script which is provided with the project.
#
# The percent sign is used to run Jupyter "magic commands".
#
# The option "-i" causes the main script to be run in the main namespace,
# i.e. the Google Colab workbook namespace. This is necessary so that:
#
# (1) Variables can be added and inspected interactively before and
#     after the main script; 
# (2) Imports by the main script (such as functions defined for the 
#     homework or project) can be used interactively within the workbook.
#
%run -i 'main.py'

### Helper code for Google Colab users
%load_ext autoreload
%autoreload 2
import os
from os.path import join as pathjoin
from os.path import isdir
import shutil
import subprocess
from google.colab import drive
assets_URL = "https://__REDACTED__/__REDACTED__.tar.gz"
runtime_dir = "/content/__REDACTED__"
backup_dir = "/content/drive/MyDrive/Colab/__REDACTED__"
def connect_to_drive():
if not isdir("/content/drive"):
drive.mount('/content/drive')
def extract_project_template():
print("Downloading and extracting data files...")
cmd = "cd /content && wget -qO- " + assets_URL + " | tar xvz -C /content"
subprocess.call(cmd, shell=True)
print("Done.")
def backup_files():
print("Backing up project source code...")
os.makedirs(backup_dir, exist_ok=True)
filenames = os.listdir(runtime_dir)
for filename in filenames:
if not filename.endswith(".py"):
continue
srcpath = pathjoin(runtime_dir, filename)
dstpath = pathjoin(backup_dir, filename)
print(f"Backing up {srcpath} to {dstpath}")
shutil.copyfile(srcpath, dstpath)
def restore_files():
print("Restoring project source code...")
filenames = os.listdir(backup_dir)
for filename in filenames:
if not filename.endswith(".py"):
continue
srcpath = pathjoin(backup_dir, filename)
dstpath = pathjoin(runtime_dir, filename)
print(f"Restoring from {srcpath} to {dstpath}")
shutil.copyfile(srcpath, dstpath)
def change_runtime_dir():
os.chdir(runtime_dir)
https://creativecommons.org/publicdomain/zero/1.0/
https://creativecommons.org/publicdomain/zero/1.0/legalcode.txt
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment