Skip to content

Instantly share code, notes, and snippets.

@tvoinarovskyi
tvoinarovskyi / worker_pool_consumer.py
Created July 14, 2017 12:50
Kafka enhanced consumer using Thread Workers and consumer.pause().
from kafka import (
KafkaConsumer, TopicPartition, OffsetAndMetadata, ConsumerRebalanceListener
)
import queue
import threading
import time
import logging
log = logging.getLogger(__name__)
/* Generated by Cython 0.27 */
#define PY_SSIZE_T_CLEAN
#include "Python.h"
#ifndef Py_PYTHON_H
#error Python headers needed to compile C extensions, please install development version of Python.
#elif PY_VERSION_HEX < 0x02060000 || (0x03000000 <= PY_VERSION_HEX && PY_VERSION_HEX < 0x03030000)
#error Cython requires Python 2.6+ or Python 3.3+.
#else
#define CYTHON_ABI "0_27"
@tvoinarovskyi
tvoinarovskyi / x_file_cmd.py
Last active March 25, 2018 14:00
Command line tools for some strange X file format. Reference code for my slides at UAPyCon2018 "Binary data in Python with a bit of C spice on top​"
# This file contains a command-line utility for working with X file format
# X file contains sequential messages in the following format:
#
# MessageID => Int64
# MessageLength => Int32
# MessageCRC => UInt32
# MessageFormatVersion => Int8
# MessageBody => Bytes
#
# We assume, that message ids are in incremental order