Skip to content

Instantly share code, notes, and snippets.

View bnaul's full-sized avatar

Brett Naul bnaul

75 followers · 0 following
View GitHub Profile
@bnaul
bnaul / worker.log
Created April 11, 2023 21:18
/usr/local/lib/python3.10/site-packages/distributed/cli/dask_worker.py:264: FutureWarning: dask-worker is deprecated and will be removed in a future release; use `dask worker` instead
warnings.warn(
2023-04-09 15:43:19,990 - distributed.nanny - INFO - Start Nanny at: 'tcp://10.76.9.27:43473'
2023-04-09 15:43:20,788 - distributed.worker - INFO - Start worker at: tcp://10.76.9.27:46429
2023-04-09 15:43:20,788 - distributed.worker - INFO - Listening to: tcp://10.76.9.27:46429
2023-04-09 15:43:20,788 - distributed.worker - INFO - Worker name: brett-1f0ecf6d-daskworkers-large-865bf95dbc-5vqnd
2023-04-09 15:43:20,788 - distributed.worker - INFO - dashboard at: 10.76.9.27:8787
2023-04-09 15:43:20,788 - distributed.worker - INFO - Waiting to connect to: tcp://brett-1f0ecf6d-daskscheduler:8786
2023-04-09 15:43:20,788 - distributed.worker - INFO - -------------------------------------------------
2023-04-09 15:43:20,788 - distributed.worker - INFO -
@bnaul
bnaul / zcta.csv
Created September 11, 2022 10:56
We can't make this file beautiful and searchable because it's too large.
zcta,cty
35660,01033
35674,01033
35661,01033
35630,01033
35646,01033
35672,01033
35654,01033
35653,01033
35616,01033
@bnaul
bnaul / profile2.json
Created April 29, 2022 18:25
This file has been truncated, but you can view the full file.
{
"$schema": "https://www.speedscope.app/file-format-schema.json",
"profiles": [
{
"type": "sampled",
"name": "Process 28 Thread 151 \"waitpid-1\"",
"unit": "seconds",
"startValue": 0,
"endValue": 188.45,
"samples": [
@bnaul
bnaul / profile1.json
Created April 29, 2022 18:13
{
"$schema": "https://www.speedscope.app/file-format-schema.json",
"profiles": [
{
"type": "sampled",
"name": "Process 28 Thread 143 \"waitpid-0\"",
"unit": "seconds",
"startValue": 0,
"endValue": 0.68,
"samples": [
@bnaul
bnaul / logs.json
Created April 28, 2022 21:36
This file has been truncated, but you can view the full file.
{
"tcp://10.125.88.48:37099": {
"type": "Worker",
"id": "Worker-4363f614-359f-4aed-af17-35e08650eb47",
"scheduler": "tcp://brett-2ad617c8-daskscheduler:8786",
"nthreads": 4,
"memory_limit": 8400000000,
"address": "tcp://10.125.88.48:37099",
"status": "<Status.running: 'running'>",
"thread_id": 140597733148480,
@bnaul
bnaul / logs.json
Created April 28, 2022 21:36
This file has been truncated, but you can view the full file.
{
"tcp://10.125.88.48:37099": {
"type": "Worker",
"id": "Worker-4363f614-359f-4aed-af17-35e08650eb47",
"scheduler": "tcp://brett-2ad617c8-daskscheduler:8786",
"nthreads": 4,
"memory_limit": 8400000000,
"address": "tcp://10.125.88.48:37099",
"status": "<Status.running: 'running'>",
"thread_id": 140597733148480,
@bnaul
bnaul / dask_bigquery.py
Created July 15, 2021 17:23
Dask <-> BigQuery helpers
def _stream_to_dfs(bqs_client, stream_name, schema, timeout):
"""Given a Storage API client and a stream name, yield all dataframes."""
return [
pyarrow.ipc.read_record_batch(
pyarrow.py_buffer(message.arrow_record_batch.serialized_record_batch), schema
).to_pandas()
for message in bqs_client.read_rows(name=stream_name, offset=0, timeout=timeout)
]
@bnaul
bnaul / KDTree.ipynb
Created February 21, 2020 06:54
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@bnaul
bnaul / kepler.gl.html
Created February 19, 2020 23:05
This file has been truncated, but you can view the full file.
<!DOCTYPE html>
<html>
<head>
<meta charset="UTF-8"/>
<title>Kepler.gl embedded map</title>
<!--Uber Font-->
<link rel="stylesheet" href="https://d1a3f4spazzrp4.cloudfront.net/kepler.gl/uber-fonts/4.0.0/superfine.css">
@bnaul
bnaul / map.html
Created January 18, 2020 23:40
This file has been truncated, but you can view the full file.
<!DOCTYPE html>
<html lang="en" >