Skip to content

Instantly share code, notes, and snippets.

abelyansky

Block or report user

Report or block abelyansky

Hide content and notifications from this user.

Learn more about blocking users

Contact Support about this user’s behavior.

Learn more about reporting abuse

Report abuse
View GitHub Profile
View KBC
public void processRecord(PutRecordsRequestEntry entry) {
try {
int recordSize = (int) this.kinesisClientManager.estimateSize(entry);
// enforce size and rate limits by blocking until new tokens are generated
entriesRatePerShard.acquire();
bytesRatePerShard.acquire(recordSize);
if (bufferQueue.size() + 1 >= MAX_ENTRIES_IN_REQUEST) {
if (logger.isDebugEnabled()) logger.debug("buffer queue is at size limit. flushing");
@abelyansky
abelyansky / threads_in_spark
Created Jun 3, 2019
simulating thread behavior in spark
View threads_in_spark
import scala.concurrent.{Future,Await, blocking}
import scala.util.{Failure, Success}
import scala.concurrent.duration._
import scala.concurrent.ExecutionContext.Implicits.global
val WORKERS_WAIT_ON_THREADS = true
val (javaThreadCounter, scalaThreadCounter) = (sc.longAccumulator, sc.longAccumulator)
val df = sc.parallelize(1 to 10).toDF()
@abelyansky
abelyansky / payload.lua
Created Aug 19, 2016
small wrk2 plugin to generate load using a file of sampled, base64 encoded cookies
View payload.lua
mime = require('mime')
wrk.method = "POST"
wrk.headers["Content-type"] = "application/x-protobuf"
requests = {}
init = function(args)
io.input('/home/centos/cookies_small')
for line in io.lines() do
actor = mime.unb64(line)
You can’t perform that action at this time.