Skip to content

Instantly share code, notes, and snippets.

View un1t's full-sized avatar

Ilya Shalyapin un1t

35 followers · 1 following
  • Ekaterinburg
View GitHub Profile
@un1t
un1t / elastic-bulk.go
Last active October 18, 2021 01:49
golang elasticsearch bulk insert
package main
import (
"fmt"
"gopkg.in/olivere/elastic.v2"
"strconv"
)
type Tweet struct {
User string `json:"user"`
@un1t
un1t / parse-xml.go
Last active January 3, 2022 09:36
Golang XML stream parser
package main
// https://github.com/dps/go-xml-parse/blob/master/go-xml-parse.go
import (
"fmt"
"os"
"encoding/xml"
)
@un1t
un1t / checksock.py
Created December 11, 2015 11:02
python check unix socket
from __future__ import print_function
import socket, sys
sock = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)
addr,path = sys.argv[1].split(":")
sock.connect(addr)
sock.send("""GET {} HTTP/1.1
host: test.ru
@un1t
un1t / elasticsearch_ru_stemming_and_morphology.py
Created March 10, 2016 12:10 — forked from svartalf/elasticsearch_ru_stemming_and_morphology.py
Example of the ElasticSearch configuration for russian stemming and morphology
requests.put('http://localhost:9200/site/', data=json.dumps({
'settings': {
'analysis': {
'analyzer': {
'ru': {
'type': 'custom',
'tokenizer': 'standard',
"filter": ['lowercase', 'russian_morphology', 'english_morphology', 'ru_stopwords'],
},
},
@un1t
un1t / abcnum_converter.py
Created April 6, 2016 18:56
Converting an Integer to a String in Any Base
import string
# http://interactivepython.org/runestone/static/pythonds/Recursion/pythondsConvertinganIntegertoaStringinAnyBase.html
class AbcNumCoverter:
chars = string.digits + string.ascii_lowercase
base = len(chars)
@classmethod
def to_str(cls, n):
@un1t
un1t / elasticsearch.yml
Created May 27, 2016 10:17
cluster.name: mycluster
node.name: node-1
node.master: true
node.data: true
network.host: 0.0.0.0
transport.tcp.port: 9300
http.port: 9200
@un1t
un1t / logging-settings.py
Created June 30, 2016 08:25
django logging settings
LOGGING = {
'version': 1,
'disable_existing_loggers': True,
'formatters': {
'verbose': {
'format': '%(levelname)s %(asctime)s %(process)d %(module)s %(message)s'
},
'simple': {
'format': '%(levelname)s %(message)s'
},
@un1t
un1t / gist:536f4391b6091097aae5e40b57e61f68
Created July 4, 2016 17:11
import gc
from more_itertools import chunked
from collections import defaultdict
def fetch_fk(fk_field, fields, objects):
assert 'id' in fields
if not objects:
return
@un1t
un1t / django_bitfield.py
Last active September 23, 2016 09:21
from django import forms
from django.db import models
class BitFieldWrapper:
def __init__(self, instance, field, value):
self.instance = instance
self.field = field
self.value = value
@un1t
un1t / SIREn Join
Created October 22, 2016 12:27
SIREn Join Plugin for Elasticsearch Example
curl -XPUT 'http://localhost:9200/_bulk?pretty' -d '
{ "index" : { "_index" : "test", "_type" : "publication", "_id" : "1" } }
{ "title" : "The NoSQL database glut", "journal" : "2", "author": "1" }
{ "index" : { "_index" : "test", "_type" : "publication", "_id" : "2" } }
{ "title" : "Graph Databases Seen Connecting the Dots", "journal" : "1", "author": "2"}
{ "index" : { "_index" : "test", "_type" : "publication", "_id" : "3" } }
{ "title" : "How to determine which NoSQL DBMS best fits your needs", "journal" : "2", "author": "3" }
{ "index" : { "_index" : "test", "_type" : "publication", "_id" : "4" } }
{ "title" : "MapR ships Apache Drill", "journal" : "4", "author": "4" }