Skip to content

Instantly share code, notes, and snippets.

View dreverri's full-sized avatar

Daniel Reverri dreverri

View GitHub Profile
@ML-engineer
ML-engineer / duckdb_bq_storage_api.py
Created July 4, 2022 22:08
Read BQ table to DuckDB directly from storage read api
import duckdb
from google.cloud import bigquery
bqclient = bigquery.Client()
table = bigquery.TableReference.from_string(
"bigquery-public-data.utility_us.country_code_iso"
)
rows = bqclient.list_rows(table)
country_code_iso = rows.to_arrow(create_bqstorage_client=True)
cursor = duckdb.connect()

Thread Pools

Thread pools on the JVM should usually be divided into the following three categories:

  1. CPU-bound
  2. Blocking IO
  3. Non-blocking IO polling

Each of these categories has a different optimal configuration and usage pattern.

@kellymclaughlin
kellymclaughlin / gist:3834437
Created October 4, 2012 15:32
Change # of schedulers online
%% Get current number of online schedulers
Schedulers = erlang:system_info(schedulers_online).
%% Reduce number online to 1
erlang:system_flag(schedulers_online, 1).
%% Restore to original number of online schedulers
erlang:system_flag(schedulers_online, Schedulers).
@agaoglu
agaoglu / MockHTable.java
Created October 6, 2010 11:34
MockHTable
/**
* This file is licensed to you under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance with the
* License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.