Skip to content

Instantly share code, notes, and snippets.

View gingerwizard's full-sized avatar

Dale McDiarmid gingerwizard

48 followers · 2 following
View GitHub Profile
@gingerwizard
gingerwizard / country_codes.csv
Last active September 12, 2023 10:06
name alpha-2 country-code
Afghanistan AF 004
Åland Islands AX 248
Albania AL 008
Algeria DZ 012
American Samoa AS 016
Andorra AD 020
Angola AO 024
Anguilla AI 660
Antarctica AQ 010
@gingerwizard
gingerwizard / countries.csv
Last active August 18, 2023 13:58
name alpha-2 country-code
Afghanistan AF 004
Åland Islands AX 248
Albania AL 008
Algeria DZ 012
American Samoa AS 016
Andorra AD 020
Angola AO 024
Anguilla AI 660
Antarctica AQ 010
@gingerwizard
gingerwizard / countries.csv
Last active August 18, 2023 13:45
name alpha-2 country-code
Afghanistan AF 004
Åland Islands AX 248
Albania AL 008
Algeria DZ 012
American Samoa AS 016
Andorra AD 020
Angola AO 024
Anguilla AI 660
Antarctica AQ 010
@gingerwizard
gingerwizard / sample_vector_search.md
Created May 15, 2023 12:59 
SELECT
    url,
    caption,
    L2Distance(image_embedding, [0.5736792683601379, 0.2516227662563324, 0.08648405969142914, -0.3087577521800995, 0.16166526079177856, -0.007050537504255772, -0.18505804240703583, -0.4547903835773468, -0.27521055936813354, -0.14916469156742096, -0.3867589235305786, -0.6603131294250488, -0.12612514197826385, 0.0861508697271347, -0.3013170063495636, -0.18740114569664001, -0.47240784764289856, 0.23528385162353516, 0.2165445238351822, -0.14971637725830078, -0.11677167564630508, 0.16455139219760895, 0.6751934885978699, 0.48457151651382446, -0.6089483499526978, 0.1328628659248352, 0.1305725872516632, -0.27523916959762573, 0.6474258899688721, 0.6060798764228821, 0.09428835660219193, -0.059794556349515915, -0.17287959158420563, 0.03964110463857651, -0.223852276802063, -0.026328448206186295, 0.16255885362625122, 0.39999595284461975, -0.2318170964717865, 0.5195636749267578, 0.35400936007499695, -0.3005162477493286, 0.581500768661499, -0.08902442455291748, -0.09745857864618301, -0.32
@gingerwizard
gingerwizard / create-house-price-single-row-group.md
Last active April 26, 2023 13:15

Below creates a Parquet file with 1 row group for UK house price dataset (28m rows, approx. 200mb Parquet with LZ4):

INSERT INTO FUNCTION file('house_prices-1-row-group.parquet') SELECT *
FROM uk_price_paid
SETTINGS min_insert_block_size_bytes = 10000000000, 
min_insert_block_size_rows=1000000000, 
output_format_parquet_row_group_size=1000000000
@gingerwizard
gingerwizard / house_prices_to_parquet.md
Last active April 14, 2023 16:25

The following assumes use of clickhouse-local, although these steps will alsowork with ClickHouse server.

1. Create table

CREATE TABLE uk_price_paid
(
    `price` UInt32,
    `date` Date,
    `postcode1` LowCardinality(String),
@gingerwizard
gingerwizard / bloom_filter_vs_mv.md
Created March 28, 2023 10:37 
EXPLAIN indexes = 1
SELECT
    Timestamp,
    TraceId,
    SpanId,
    SpanName
FROM otel_traces
WHERE TraceId = '0f8a2c02d77d65da6b2c4d676985b3ab'
@gingerwizard
gingerwizard / trace_clickhouse_queries.md
Created March 28, 2023 10:15

Trace data

Supplements queries documented as part of ClickHouse exporter.

Traces over Time

SELECT toStartOfInterval(toDateTime(Timestamp), INTERVAL 60 second) as time, ServiceName, count() as ` ` 
FROM otel."otel_traces" WHERE 1=1 AND Timestamp >= '1679994670' AND Timestamp <= '1679998270'  AND 
ServiceName In ('accountingservice','adservice','cartservice','checkoutservice','currencyservice','emailservice',
@gingerwizard
gingerwizard / trace_data.md
Created March 28, 2023 10:05

Create the Table

Optionally remove the secondary indices.

CREATE TABLE otel_traces
(
    `Timestamp` DateTime64(9) CODEC(Delta(8), ZSTD(1)),
    `TraceId` String CODEC(ZSTD(1)),
 `SpanId` String CODEC(ZSTD(1)),
@gingerwizard
gingerwizard / values.yaml
Created March 27, 2023 17:06
default:
# list of environment variables applied to all components
env:
- name: OTEL_SERVICE_NAME
valueFrom:
fieldRef:
apiVersion: v1
fieldPath: "metadata.labels['app.kubernetes.io/component']"
- name: OTEL_K8S_NAMESPACE
valueFrom: