Skip to content

Instantly share code, notes, and snippets.

Avatar

Andrew Grigorev ei-grad

View GitHub Profile
@ei-grad
ei-grad / conda.conf
Created Mar 21, 2020
Nginx caching proxy for conda
View conda.conf
# /etc/nginx/conf.d/conda.conf
proxy_cache_path /var/cache/nginx/conda levels=2:2 keys_zone=conda:100m inactive=14d max_size=10g;
upstream conda.anaconda.org {
server 104.17.92.24:443;
server 104.17.93.24:443;
}
server {
listen 80;
View cloudpickle.patch
diff --git a/python/pyspark/cloudpickle.py b/python/pyspark/cloudpickle.py
index 88519d7311..f7ab821b97 100644
--- a/python/pyspark/cloudpickle.py
+++ b/python/pyspark/cloudpickle.py
@@ -72,77 +72,16 @@ else:
PY3 = True
-def _make_cell_set_template_code():
- """Get the Python compiler to emit LOAD_FAST(arg); STORE_DEREF
@ei-grad
ei-grad / fetch-updated-packages
Last active Apr 9, 2020
/etc/cron.hourly/fetch-updated-packages
View fetch-updated-packages
#!/usr/bin/chronic /bin/bash
set -e
set -x
[ -n "`nmcli c show --active | grep -f /etc/fetch-allowed-wifi-networks`" ] || exit 0
pacman -S --refresh --sysupgrade --downloadonly --noconfirm
grep KeepCurrent /etc/pacman.conf && pacman -S --clean --noconfirm
@ei-grad
ei-grad / json_array_to_lines.go
Created Nov 28, 2019
Convert large JSON array to lines
View json_array_to_lines.go
package main
import (
"encoding/json"
"fmt"
"log"
"os"
)
func main() {
View clickhouse_types.lark
WS: /[ \t\f\r\n]/+
DIGIT: "0".."9"
INT: DIGIT+
FLOAT: DIGIT? "." DIGIT+
| DIGIT+ "."
int: INT
float: FLOAT
// simple types
View gen-video-html.py
from pathlib import Path
from urllib.parse import quote
index = []
for i in sorted(Path('.').iterdir()):
index.append(i)
with open('index.html', 'w') as f:
f.write('<html><body><center>')
@ei-grad
ei-grad / coordconv.py
Created Aug 6, 2019
Neat CoordConv channels injection implementation as a tensorflow.keras layer.
View coordconv.py
from tensorflow.keras import backend as K
from tensorflow.keras.layers import Layer
class CoordinateChannel2D(Layer):
def call(self, inputs):
x = K.cast(K.arange(0, K.shape(inputs)[1]), K.floatx())
x /= K.cast(K.shape(inputs)[1], K.floatx())
@ei-grad
ei-grad / speech2text
Created May 18, 2019
speech2text command using Yandex SpeachKit
View speech2text
#!/bin/bash
if [ -z "$YC_API_KEY" ]; then
cat << EOF
Yandex Cloud API Key is needed:
https://cloud.yandex.ru/docs/iam/concepts/authorization/api-key
it would require some hustle with service account and roles:
https://cloud.yandex.ru/docs/speechkit/security/
View decode-mime-headers.pl
#!/usr/bin/env perl
use open qw(:std :utf8);
use Encode qw(decode);
while (my $line = <STDIN>) {
print decode("MIME-Header", $line);
}
View sort-files-by-size.sh
#!/bin/sh
find -name .git -a -type d -prune -o -type f -print0 | du --bytes -h --files0-from=- | sort -rh
You can’t perform that action at this time.