rgbkrk

rgbkrk

rgbkrk

DROP TABLE IF EXISTS baby_names;
CREATE TEMPORARY TABLE baby_names( name TEXT, rank INTEGER );

-- From most popular baby names of 2020 according to the Social Security Administration
-- https://www.ssa.gov/oact/babynames/
INSERT INTO baby_names ('name', 'rank') VALUES ('Liam', 1);
INSERT INTO baby_names ('name', 'rank') VALUES ('Olivia', 1);
INSERT INTO baby_names ('name', 'rank') VALUES ('Noah', 2);
INSERT INTO baby_names ('name', 'rank') VALUES ('Emma', 2);
INSERT INTO baby_names ('name', 'rank') VALUES ('Oliver', 3);

rgbkrk

rgbkrk

Background

System:

• Debian, running on Amazong
• /root is backed by NFS (Amazon's EFS)

Launch jupyter console --kernel python3 in one terminal, then pstree in another.

rgbkrk

Jupyter Kernels and Node

This is a (lightly) annotated run through of launching a jupyter kernel, sending some messages, and cleaning it up.

git clone https://gist.github.com/3239c1547b53c97cc648d5abb3cbe3e0.git kernely
cd kernely
npm install
node index.js

rgbkrk

rgbkrk

(function(console){

    console.save = function(data, filename){

        if(!data) {
            data = Jupyter.notebook.toJSON();
        }

        filename = Jupyter.notebook.notebook_name;
        if(!filename) {

rgbkrk

nteract doesn't support requirejs because we have the builtin require at our fingertips. Jupyter notebook however has long operated under the assumption that you can use the builtin requirejs for loading modules asynchronously:

require(['d3'], function(d3) {...

I started exploring the idea of providing some of these modules in a "quirks" sort of mode where we provide limited access to "requirejs" while still sandboxed. To find out what modules were commonly required, I turned to Google BigQuery, the GitHub dataset, and a User Defined Function (UDF) written in JavaScript.

I'll flesh out this gist or a blog post later. For now, I'll just provide my query code:

rgbkrk

How many times shouldn't it happen...

-- https://news.ycombinator.com/item?id=11396045

SELECT count(*)
FROM (SELECT id, repo_name, path
        FROM [bigquery-public-data:github_repos.sample_files]
 ) AS F