Skip to content

Instantly share code, notes, and snippets.

View hrchu's full-sized avatar
:octocat:
Have an Octotastic day!

petertc hrchu

:octocat:
Have an Octotastic day!
View GitHub Profile
@chipx86
chipx86 / streaming-tar.py
Last active October 20, 2022 21:17
Sample code to build a tar chunk-by-chunk and stream it out all at once.
#!/usr/bin/env python
#
# Building a tar file chunk-by-chunk.
#
# This is a quick bit of sample code for streaming data to a tar file,
# building it piece-by-piece. The tarfile is built on-the-fly and streamed
# back out. This is useful for web applications that need to dynamically
# build a tar file without swamping the server.
import os
import sys
@piatra
piatra / kafka_start.sh
Last active May 24, 2024 17:04
start zookeeper and kafka in the background and create a topic
#!/bin/bash
if [ "$#" -ne 1 ]; then
echo "Please supply topic name"
exit 1
fi
nohup bin/zookeeper-server-start.sh -daemon config/zookeeper.properties > /dev/null 2>&1 &
sleep 2
nohup bin/kafka-server-start.sh -daemon config/server.properties > /dev/null 2>&1 &
@hrchu
hrchu / gist:0cc5e783387f5453f528
Last active April 23, 2019 09:48
logstash-grok-pattern-squid
# Follows the squid format in default:
# logformat squid %ts.%03tu %6tr %>a %Ss/%03>Hs %<st %rm %ru %[un %Sh/%<a %mt
# http://www.squid-cache.org/Doc/config/logformat/
%{NUMBER:timestamp}\s+%{NUMBER:response_time} %{IPORHOST:src_ip} %{WORD:squid_request_status}/%{NUMBER:http_status_code} %{NUMBER:reply_size_include_header} %{WORD:http_method} %{URI:request_url} %{USERNAME:user} %{WORD:squid_hierarchy_status}/%{IPORHOST:server_ip_or_peer_name} (?<mime_content_type>\S+\/\S+)
@JoelQ
JoelQ / README.md
Created October 15, 2015 13:50
Using Shell Scripts for a Better User Experience (source for https://robots.thoughtbot.com/improving-user-experience-with-shell-scripts)

Server scripts

This is the source for the scripts discussed in https://robots.thoughtbot.com/improving-user-experience-with-shell-scripts

Both scripts are in the bin/ directory of the repo that contains all the markdown documents for blog posts. Users run bin/server and everything is automatically set up for them to view a local preview of the blog. bin/server-setup is a dependency of bin/server and is never run directly by users.

Maitre-d is the name of the "blog engine" discussed in the article.

@hrchu
hrchu / 99-network-tuning.conf
Last active December 11, 2023 23:05 — forked from kgriffs/sysctl.conf
Linux Web Server Kernel Tuning
# Configuration file for runtime kernel parameters.
# See sysctl.conf(5) for more information.
# See also http://www.nateware.com/linux-network-tuning-for-2013.html for
# an explanation about some of these parameters, and instructions for
# a few other tweaks outside this file.
# See evil packets in your logs.
net.ipv4.conf.all.log_martians = 1
@bastman
bastman / docker-cleanup-resources.md
Created March 31, 2016 05:55
docker cleanup guide: containers, images, volumes, networks

Docker - How to cleanup (unused) resources

Once in a while, you may need to cleanup resources (containers, volumes, images, networks) ...

delete volumes

// see: https://github.com/chadoe/docker-cleanup-volumes

$ docker volume rm $(docker volume ls -qf dangling=true)

$ docker volume ls -qf dangling=true | xargs -r docker volume rm

@ursuad
ursuad / kafka-cheat-sheet.md
Last active March 14, 2024 10:32
Quick command reference for Apache Kafka

Kafka Topics

List existing topics

bin/kafka-topics.sh --zookeeper localhost:2181 --list

Describe a topic

bin/kafka-topics.sh --zookeeper localhost:2181 --describe --topic mytopic

Purge a topic

bin/kafka-topics.sh --zookeeper localhost:2181 --alter --topic mytopic --config retention.ms=1000

... wait a minute ...

If your processing rate is high this might not be optimal since you'll be committing for each message, even if using async commits there's some degree of performance penalty.
So another alternative is to keep `enable.auto.commit` set to True (default) but disable the automatic offset store.
So what is the offset store?
Each time a message is passed from the client to your application its offset is stored for future commit, the next intervalled commit will then use this stored offset. If the stored offset did not change from the last commit nothing happens.
So by setting `enable.auto.offset.store` to False you keep the convenient intervalled auto commit behaviour but you control what offsets are actually eligible for commit.
@xl7dev
xl7dev / VMwareFusion_Export_ovf.txt
Last active April 24, 2024 21:52
HowTo Export a VM in OVA format in VMware Fusion for OS X
> cd /Applications/VMware Fusion.app/Contents/Library/VMware OVF Tool
> ./ovftool --acceptAllEulas /Users/marco/Documents/Virtual\ Machines.localized/Windows\ 8.1\ x64.vmwarevm/Windows\ 8.1\ x64.vmx /Users/marco/Desktop/Win81.ova
@alexislucena
alexislucena / install-scala-sbt-and-java-on-ubuntu.md
Created December 5, 2016 12:35
Ubuntu: Install Scala, SBT and Java on Ubuntu 16.04

Install Scala 2.11.8

$ sudo apt-get remove scala-library scala
$ sudo wget www.scala-lang.org/files/archive/scala-2.11.8.deb
$ sudo dpkg -i scala-2.11.8.deb

Check Scala version

$ scala -version