Skip to content

Instantly share code, notes, and snippets.

View xiongnemo's full-sized avatar
🦘
皆さんの健康を心から祈っています。

Nemo Xiong xiongnemo

🦘
皆さんの健康を心から祈っています。
  • 122.199.30.0/24 || 128.250.0.0/16
  • 05:50 (UTC +10:00)
View GitHub Profile
@p4bl0-
p4bl0- / 00_readme.md
Last active October 12, 2023 09:09
A complete compiler for a simple language (in less than 150 LoC)

This project is a tiny compiler for a very simple language consisting of boolean expression.

The language has two constants: 1 for true and 0 for false, and 4 logic gates: ! (not), & (and), | (or), and ^ (xor).

It can also use parentheses to manage priorities.

Here is its grammar in BNF format:

expr ::= "0" | "1"

@LupusArgentum
LupusArgentum / openwrt-8021x.md
Last active April 21, 2024 11:43
OpenWrt-router as 802.1x-client

OpenWrt-router as 802.1x-client

This use-case is a pretty rare one, but in some circumstances, it can be very helpful. For example when you live in a student dormatory which only offers one 802.1x-encrypted LAN-port in your room, but you want to run your own wifi-network to be online with other clients, too, like your laptop or smartphone. In this case, normal routers with stock firmware won't help you out because most don't support this networking protocol. OpenWrt on the other hand offers you the possibility to connect your router (you could buy this one if you don't already have a suiting router) to the 802.1x-network via WAN and enable you to have an own, independent network. Here's how.

Important: before you attempt to do this, it is NECESSARY to ask your network admin if he/she is okay with your usage scenario. This can cause some trouble if you do it without permission, as many 802.1x-networks aim to prevent this exact use-case.

So here's the deal

Paralellism in ES and Hadoop/Spark

1 shard corresponds to 1 Spark partition.

Reading from ES: https://www.elastic.co/guide/en/elasticsearch/hadoop/current/arch.html#arch-reading . Beware of increasing the number of shards on ES for performance reasons:

A common concern (read optimization) for improving performance is to increase the number of shards and thus increase the number of tasks on the Hadoop side. Unless such gains are demonstrated through benchmarks, we recommend against such a measure since in most cases, an Elasticsearch shard can easily handle data streaming to a Hadoop or Spark task.

Writing from ES: https://www.elastic.co/guide/en/elasticsearch/hadoop/current/arch.html#arch-writing . Write performance can be increased by having more partitions:

elasticsearch-hadoop detects the number of (primary) shards where the write will occur and distributes the writes between these. The more splits/partitions available, the more mappers/reducers can write data in parallel to Elasticsear

@hideshi
hideshi / sigmetparser.scala
Created December 29, 2013 07:59
This is a parser for SIGMET which is a format for weather information. See also: http://d.hatena.ne.jp/hideshi_o/20130105/1357367954
abstract sealed class Bulletin
case class SigmetBulletin (
header:Header
,firstLine:FirstLine
,mainBody:MainBody
) extends Bulletin
case class Header(
identificationMessage:String = ""
@davidwaterston
davidwaterston / gist:3755420
Created September 20, 2012 11:39
Oracle: Check if a column exists in a table
select column_name as found
from user_tab_cols
where table_name = '__TABLE_NAME__'
and column_name = '__COLUMN_NAME__'
@earthgecko
earthgecko / bash.generate.random.alphanumeric.string.sh
Last active July 4, 2024 17:31
shell/bash generate random alphanumeric string
#!/bin/bash
# bash generate random alphanumeric string
#
# bash generate random 32 character alphanumeric string (upper and lowercase) and
NEW_UUID=$(cat /dev/urandom | tr -dc 'a-zA-Z0-9' | fold -w 32 | head -n 1)
# bash generate random 32 character alphanumeric string (lowercase only)
cat /dev/urandom | tr -dc 'a-z0-9' | fold -w 32 | head -n 1
@masak
masak / explanation.md
Last active July 9, 2024 17:07
How is git commit sha1 formed

Ok, I geeked out, and this is probably more information than you need. But it completely answers the question. Sorry. ☺

Locally, I'm at this commit:

$ git show
commit d6cd1e2bd19e03a81132a23b2025920577f84e37
Author: jnthn <jnthn@jnthn.net>
Date:   Sun Apr 15 16:35:03 2012 +0200

When I added FIRST/NEXT/LAST, it was idiomatic but not quite so fast. This makes it faster. Another little bit of masak++'s program.

@dupuy
dupuy / README.rst
Last active June 25, 2024 15:05
Common markup for Markdown and reStructuredText

Markdown and reStructuredText

GitHub supports several lightweight markup languages for documentation; the most popular ones (generally, not just at GitHub) are Markdown and reStructuredText. Markdown is sometimes considered easier to use, and is often preferred when the purpose is simply to generate HTML. On the other hand, reStructuredText is more extensible and powerful, with native support (not just embedded HTML) for tables, as well as things like automatic generation of tables of contents.