Skip to content

Instantly share code, notes, and snippets.

View zhuguangbin's full-sized avatar

zhuguangbin zhuguangbin

View GitHub Profile
@groundwater
groundwater / asynchbase.scala
Created April 20, 2012 17:47
Demo Adapting asynchbase to Scala
package ca.underflow.hbase
import org.hbase.async._
import com.stumbleupon.async._
object Demo extends App {
// This let's us pass inline functions to the deferred
// object returned by most asynchbase methods
@jeantil
jeantil / Application.scala
Last active July 29, 2018 12:01
Playframework 2.2 configurable cors filter
/**
The MIT License (MIT)
Copyright (c) 2013 Jean Helou
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
@tinkerware
tinkerware / Vagrant provision block
Last active January 21, 2019 01:11
Install Latest Java 7 and Java 8 on Ubuntu 14.04 LTS
config.vm.provision "shell", inline: <<-SHELL
apt-get -y -q update
apt-get -y -q upgrade
apt-get -y -q install software-properties-common htop
add-apt-repository ppa:webupd8team/java
apt-get -y -q update
echo oracle-java8-installer shared/accepted-oracle-license-v1-1 select true | sudo /usr/bin/debconf-set-selections
echo oracle-java7-installer shared/accepted-oracle-license-v1-1 select true | sudo /usr/bin/debconf-set-selections
apt-get -y -q install oracle-java8-installer
apt-get -y -q install oracle-java7-installer
@touilleMan
touilleMan / SimpleHTTPServerWithUpload.py
Last active June 16, 2024 01:51 — forked from UniIsland/SimpleHTTPServerWithUpload.py
Simple Python Http Server with Upload - Python3 version
#!/usr/bin/env python3
"""Simple HTTP Server With Upload.
This module builds on BaseHTTPServer by implementing the standard GET
and HEAD requests in a fairly straightforward manner.
see: https://gist.github.com/UniIsland/3346170
"""
@epiphani
epiphani / CDHTez.md
Last active February 14, 2024 08:03
Getting Tez enabled on CDH5.4+

So Hive in CDH is horribly, painfully slow. Cloudera ships Hive 1.1, which is actually moderately modern. It is, however, very badly configured out of the box and patched with custom code from Cloudera. With a bit of effort, we managed to improve hive performance considerably. We really shouldn't have to do this, but Cloudera is actively working against supporting a performant Hive.

First, building Tez was fairly straightforward. Using the instructions at https://github.com/apache/tez/blob/master/docs/src/site/markdown/install.md, the only change was to use the version string "2.6.0" for the build. I believe that was the default. Don't use the CDH string, it won't work.

At the bottom of the installation instructions, there's mention of the fact that to use the local hadoop jars (rather than those packaged with tez) you must unpack the jars in HDFS rather than using the tarball. In this case, unpack the tez-minimal tarball and upload the contents to /apps/tez-0.7.0 (or whatever you prefer). Don't fo

@zedar
zedar / ApacheHadoop_NativeLibs.adoc
Last active August 13, 2020 00:59
Add native libraries to Apache Hadoop installation

Apache Hadoop - add native libraries

If native libraries are not available the following message is displayed with every hadoop command: hadoop checknative

WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
  • Clone hadoop source code


@mingliangguo
mingliangguo / disable_SSL_and_hostname_verification.java
Last active August 21, 2023 11:47
disable SSL and host name verification for apache httpclient
try {
TrustManager[] trustAllCerts = new TrustManager[] {
new X509TrustManager() {
public java.security.cert.X509Certificate[] getAcceptedIssuers() {
return null;
}
public void checkClientTrusted(X509Certificate[] certs, String authType) { }
public void checkServerTrusted(X509Certificate[] certs, String authType) { }
}
@timvlaer
timvlaer / gist:721ba30f8fc6a7aac1b0190e132a4261
Created March 21, 2017 12:35
Install Thrift 0.9.3 with brew on macOS
brew unlink thrift
brew install https://raw.githubusercontent.com/Homebrew/homebrew-core/9d524e4850651cfedd64bc0740f1379b533f607d/Formula/thrift.rb
@oneryalcin
oneryalcin / sse_fast_api.py
Last active June 3, 2024 12:34
Server Side Events (SSE) with FastAPi and (partially) Langchain
# I couldn't get return generators from chains so I had to do a bit of low level SSE, Hope this is useful
# Probably you'll use another Vector Store instead of OpenSearch, but if you want to mimic what I did here,
# please use the fork of `OpenSearchVectorSearch` in https://github.com/oneryalcin/langchain
import json
import os
import logging
from typing import List, Generator
@veekaybee
veekaybee / normcore-llm.md
Last active July 8, 2024 03:50
Normcore LLM Reads

Anti-hype LLM reading list

Goals: Add links that are reasonable and good explanations of how stuff works. No hype and no vendor content if possible. Practical first-hand accounts of models in prod eagerly sought.

Foundational Concepts

Screenshot 2023-12-18 at 10 40 27 PM

Pre-Transformer Models