Skip to content

Instantly share code, notes, and snippets.

@ibalashov
ibalashov / InterleavedIteratorMany.java
Created May 8, 2018 13:57
Interleaved iterator with multiple backing iterators support
package com.outbrain.utils;
import com.google.common.collect.Lists;
import java.util.ArrayList;
import java.util.Collection;
import java.util.Iterator;
import java.util.List;
import java.util.stream.Collectors;
import java.util.stream.IntStream;
package com.outbrain.utils;
import java.util.Iterator;
public class InterleavedIterator<T> implements Iterator<T> {
private Iterator<T> it1;
private Iterator<T> it2;
private boolean takeFirst;
@ibalashov
ibalashov / scratch_121.txt
Created August 28, 2017 07:57
List vs HashSet performance for small lists (single contains op)
public static List<Integer> generate(int series) {
return Stream.iterate(new int[]{0, 1}, s -> new int[]{s[1], s[0] + s[1]})
.limit(series)
.map(n -> n[0])
.collect(Collectors.toList());
}
public static void main(String[] args) {
generate(30).forEach(n -> {
List<String> collected = Stream.generate(() -> UUID.randomUUID().toString()).limit(n).collect(Collectors.toList());
@ibalashov
ibalashov / AvroReadTest.java
Created November 12, 2016 19:30
Reading broken avro file with DataFileReader does not produce any exception. fastavro duly complains on the same file.
% cat twitter.snappy.incomplete.avro
Objavro.codec
snappyavro.schema�{"type":"record","name":"twitter_schema","namespace":"com.miguno.avro","fields":[{"name":"username","type":"string","doc":"Name of the user account on Twitter.com"},{"name":"tweet","type":"string","doc":"The content of the user's Twitter message"},{"name":"timestamp","type":"long","doc":"Unix epoch time in milliseconds"}],"doc:":"A basic schema for storing Twitter messages"}5\�1��~����H����d�c
migunoFRock: Nerf paper, scissors is fine.���
BlizzardCSFWor%
% fastavro twitter.snappy.incomplete.avro
Traceback (most recent call last):
File "/usr/local/bin/fastavro", line 9, in <module>
#!/usr/bin/env bash
# Boot script for Spark master
### BEGIN INIT INFO
# Provides: spark-master
# Required-Start: $all $network
# Required-Stop: $all $network
# Default-Start: 2 3 4 5
# Default-Stop: 0 1 6
# Short-Description: Spark-master
# Description: Spark-master (http://hadoop.apache.org/)
Reverse Call Tree Time (ms) Level
com.sun.crypto.provider.GHASH.update(byte[], int, int) GHASH.java 438036 1
com.sun.crypto.provider.GaloisCounterMode.doLastBlock(byte[], int, int, byte[], int, boolean) GaloisCounterMode.java:362 435964 2
com.sun.crypto.provider.GaloisCounterMode.encryptFinal(byte[], int, int, byte[], int) GaloisCounterMode.java:419 3
com.sun.crypto.provider.CipherCore.finalNoPadding(byte[], int, byte[], int, int) CipherCore.java:1025 4
com.sun.crypto.provider.CipherCore.doFinal(byte[], int, int, byte[], int) CipherCore.java:984 5
com.sun.crypto.provider.AESCipher.engineDoFinal(byte[], int, int, byte[], int) AESCipher.java:479 6
javax.crypto.Cipher.doFinal(byte[], int, int, byte[], int) Cipher.java:2377 7
sun.security.ssl.CipherBox.encrypt(byte[], int, int) CipherBox.java:315 8
sun.security.ssl.OutputRecord.encrypt(Authenticator, CipherBox) OutputRecord.java:264 9
public StorageObject uploadFile(String bucketName, File file, String remoteDirectory) throws Exception {
InputStream stream = new BufferedInputStream(new FileInputStream(file));
try {
InputStreamContent mediaContent = new InputStreamContent("application/octet-stream", stream);
mediaContent.setLength(file.length());
Storage.Objects.Insert insert = storage.objects().insert(bucketName, null, mediaContent);
insert.getMediaHttpUploader().setDisableGZipContent(true);
if (mediaContent.getLength() > 0 && mediaContent.getLength() <= 2 * 1000 * 1000 /* 2MB */) {
insert.getMediaHttpUploader().setDirectUploadEnabled(true);
# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements. See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
@ibalashov
ibalashov / TestSystem.scala
Created April 1, 2015 13:10
Akka ping-pong perf test with minimized FJ threads
package akkatest
import akka.actor._
import akka.routing.RoundRobinPool
import com.typesafe.config.{Config, ConfigFactory}
import com.typesafe.scalalogging.slf4j.LazyLogging
import iow.common.util.NetUtil
import net.iponweb.kafka.util.MiscUtil
import scala.util.Try
@ibalashov
ibalashov / TestSystem.scala
Created March 30, 2015 10:27
Akka Remoting Ping-Pong simple throughput test
package akkatest
import akka.actor._
import akka.routing.RoundRobinPool
import com.typesafe.config.{Config, ConfigFactory}
import com.typesafe.scalalogging.slf4j.LazyLogging
object TestSystem extends LazyLogging {
case object PingObj