Skip to content

Instantly share code, notes, and snippets.

@pkit
Created July 20, 2023 15:43
Show Gist options
  • Save pkit/e6648d4334518289da76b21e54805bf3 to your computer and use it in GitHub Desktop.
Save pkit/e6648d4334518289da76b21e54805bf3 to your computer and use it in GitHub Desktop.

Events table (ordered by insertion):

CREATE TABLE events (
  id UInt64,
  ts DateTime64(3),
  data TEXT
)
ENGINE=MergeTree()
ORDER BY tuple()

Service table:

CREATE TABLE service1 (
  id UInt64,
  ts DateTime64(3),
  data TEXT
)
ENGINE=URL('https://service1.example.com/api/enrich', JSONEachRow)

Mat view:

CREATE MATERIALIZED VIEW events_enrich
TO service1 AS
SELECT
  id,
  ts,
  data
FROM events

What will happen:

  • rows get inserted into events
  • mat view will push them into the API endpoint (in JSONEachRow format) using POST HTTP request.
  • rows will be batched: multiple rows per one POST.
  • ClickHouse will not wait for any data in response, API should only answer with 200 ok and that's it.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment