Skip to content

Instantly share code, notes, and snippets.

Avatar
🥔
Doing this and that

Plugaru Tudor PlugaruT

🥔
Doing this and that
View GitHub Profile
View beam_to_global_window.py
import apache_beam as beam
from apache_beam.transforms.window import FixedWindows, GlobalWindows
events = [
("k1", {"a": 1, "b":2}),
("k1", {"a": 1, "b":2}),
("k2", {"a": 1, "b":2}),
("k2", {"a": 1, "b":2}),
("k3", {"a": 1, "b":2}),
("k4", {"a": 1, "b":2}),
View pipeline-stacktrace.py
Error message from worker: generic::unknown: Traceback (most recent call last):
File "apache_beam/runners/common.py", line 1246, in apache_beam.runners.common.DoFnRunner._invoke_bundle_method
File "apache_beam/runners/common.py", line 514, in apache_beam.runners.common.DoFnInvoker.invoke_finish_bundle
File "apache_beam/runners/common.py", line 519, in apache_beam.runners.common.DoFnInvoker.invoke_finish_bundle
File "apache_beam/runners/common.py", line 1416, in apache_beam.runners.common._OutputProcessor.finish_bundle_outputs
File "apache_beam/runners/worker/operations.py", line 151, in apache_beam.runners.worker.operations.ConsumerSet.receive
File "apache_beam/runners/worker/operations.py", line 153, in apache_beam.runners.worker.operations.ConsumerSet.receive
File "apache_beam/runners/worker/operations.py", line 181, in apache_beam.runners.worker.operations.ConsumerSet.update_counters_start
File "apache_beam/runners/worker/opcounters.py", line 216, in apache_beam.runners.worker.opcounters.Op
@PlugaruT
PlugaruT / pipeline.py
Created Nov 12, 2020
Apache Beam issue
View pipeline.py
class WriteToObjectTableBigquery(PTransform):
def __init__(
self, project: str, dataset: str, window_interval_seconds: int = 20, window_lateness_seconds: int = 10
):
super().__init__()
self.project = project
self.dataset = dataset
self.window_interval_seconds = window_interval_seconds
self.window_lateness_seconds = window_lateness_seconds
View keybase.md

Keybase proof

I hereby claim:

  • I am plugarut on github.
  • I am plugarut (https://keybase.io/plugarut) on keybase.
  • I have a public key ASAXVb60ITBj501rVzEuCclJFULBvLidu5FC0m95pmlSSgo

To claim this, I am signing this object:

View gist:84add755fa72e15e388ed3892990532a
Verifying my Blockstack ID is secured with the address 15oca1RKDVnhc49xWA7P4ttqB47pN87Yai https://explorer.blockstack.org/address/15oca1RKDVnhc49xWA7P4ttqB47pN87Yai
View wrong_abstract_method_traceback.py
>>> Bar()
<__main__.Bar at 0x7f7c94380828>
>>> Bar().my_abstract_method()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<stdin>", line 3, in my_abstract_method
NotImplementedError
View correct_abstract_method_traceback.py
# Assume that class Bar inherits from AFoo
>>> Bar()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: Can't instantiate abstract class Bar with abstract methods my_abstract_method
View correct_abstract_method.py
from abc import ABCMeta, abstractmethod
class AFoo(metaclass=ABCMeta):
@abstractmethod
def my_abstract_method(self):
pass
View wrong_abstract_method.py
class Foo:
def my_abstract_method(self):
raise NotImplementedError