Skip to content

Instantly share code, notes, and snippets.

View PlugaruT's full-sized avatar
🥔
Doing this and that

Tudor Plugaru PlugaruT

🥔
Doing this and that
View GitHub Profile
class Bar(Foo):
pass
Verifying my Blockstack ID is secured with the address 15oca1RKDVnhc49xWA7P4ttqB47pN87Yai https://explorer.blockstack.org/address/15oca1RKDVnhc49xWA7P4ttqB47pN87Yai

Keybase proof

I hereby claim:

  • I am plugarut on github.
  • I am plugarut (https://keybase.io/plugarut) on keybase.
  • I have a public key ASAXVb60ITBj501rVzEuCclJFULBvLidu5FC0m95pmlSSgo

To claim this, I am signing this object:

@PlugaruT
PlugaruT / pipeline.py
Created November 12, 2020 10:09
Apache Beam issue
class WriteToObjectTableBigquery(PTransform):
def __init__(
self, project: str, dataset: str, window_interval_seconds: int = 20, window_lateness_seconds: int = 10
):
super().__init__()
self.project = project
self.dataset = dataset
self.window_interval_seconds = window_interval_seconds
self.window_lateness_seconds = window_lateness_seconds
@PlugaruT
PlugaruT / pipeline-stacktrace.py
Created November 12, 2020 10:11
Pipeline stacktrace
Error message from worker: generic::unknown: Traceback (most recent call last):
File "apache_beam/runners/common.py", line 1246, in apache_beam.runners.common.DoFnRunner._invoke_bundle_method
File "apache_beam/runners/common.py", line 514, in apache_beam.runners.common.DoFnInvoker.invoke_finish_bundle
File "apache_beam/runners/common.py", line 519, in apache_beam.runners.common.DoFnInvoker.invoke_finish_bundle
File "apache_beam/runners/common.py", line 1416, in apache_beam.runners.common._OutputProcessor.finish_bundle_outputs
File "apache_beam/runners/worker/operations.py", line 151, in apache_beam.runners.worker.operations.ConsumerSet.receive
File "apache_beam/runners/worker/operations.py", line 153, in apache_beam.runners.worker.operations.ConsumerSet.receive
File "apache_beam/runners/worker/operations.py", line 181, in apache_beam.runners.worker.operations.ConsumerSet.update_counters_start
File "apache_beam/runners/worker/opcounters.py", line 216, in apache_beam.runners.worker.opcounters.Op
import apache_beam as beam
from apache_beam.transforms.window import FixedWindows, GlobalWindows
events = [
("k1", {"a": 1, "b":2}),
("k1", {"a": 1, "b":2}),
("k2", {"a": 1, "b":2}),
("k2", {"a": 1, "b":2}),
("k3", {"a": 1, "b":2}),
("k4", {"a": 1, "b":2}),