Skip to content

Instantly share code, notes, and snippets.

View ecarreras's full-sized avatar
💡
IT for utilities

Eduard Carreras ecarreras

💡
IT for utilities
121 followers · 404 following
View GitHub Profile
@DavidWells
DavidWells / github-proxy-client.js
Last active March 15, 2024 08:28
Full Github REST api in 34 lines of code
/* Ultra lightweight Github REST Client */
// original inspiration via https://gist.github.com/v1vendi/75d5e5dad7a2d1ef3fcb48234e4528cb
const token = 'github-token-here'
const githubClient = generateAPI('https://api.github.com', {
headers: {
'User-Agent': 'xyz',
'Authorization': `bearer ${token}`
}
})
@ScriptingSquirrel
ScriptingSquirrel / export-node-stats.md
Last active April 4, 2023 02:18
Setup prometheus-node-exporter and push stats to Pushgateway with cron job

(Assuming a Debian 8-like system)

  • Install prometheus-node-exporter

    $ sudo apt update && sudo apt install prometheus-node-exporter

  • Configure prometheus-node-exporter to expose metrics only to localhost, not on to all networks. Modify file /etc/default/prometheus-node-exporter:

    # Set the command-line arguments to pass to the server.
@ecarreras
ecarreras / models.py
Last active February 16, 2017 10:28
from collections import namedtuple
import json
class Model(object):
fields = [
'id',
'cini',
'origen',
'desti'
]
@ecarreras
ecarreras / README.rst
Last active January 21, 2016 08:48

Convert HTML to Image

from html2image import HTML2Image

with HTML2Image('<html><strong>Fooo</strong></html>', width='300') as html:
    image = html.render()
@ecarreras
ecarreras / README.md
Last active August 7, 2023 07:39
PostgreSQL replication

Checklist

  • Create user replication in the master
sudo -u postgres psql -c "CREATE USER rep REPLICATION \
LOGIN ENCRYPTED PASSWORD 'thepassword';"
  • Modify postgresql.conf in the master
listen_address = # make sure we're listening as appropriate
wal_level = hot_standby
@mpampols
mpampols / sample-ssl-nginx
Last active August 29, 2015 14:05
server {
listen 80;
server_name www.marcpampols.com marcpampols.com;
return 301 https://www.marcpampols.com$request_uri;
}
server {
listen 443;
ssl on;
@ecarreras
ecarreras / parse_comers.py
Created July 30, 2014 06:57
import os
print "Descarregant PDF comers..."
os.system('curl "http://simel.simel.ree.es/sep/PubServlet2?operacion=AccInfor&fichero=117_Comercializadores.pdf" > comers.pdf')
print "Transformant a TXT..."
os.system('pdftotext comers.pdf')
print "Creant diccionari..."
c = open('comers.txt', 'r').read()
@zsup
zsup / ddd.md
Last active April 17, 2024 14:35
Documentation-Driven Development (DDD)

Documentation-Driven Development

The philosophy behind Documentation-Driven Development is a simple: from the perspective of a user, if a feature is not documented, then it doesn't exist, and if a feature is documented incorrectly, then it's broken.

  • Document the feature first. Figure out how you're going to describe the feature to users; if it's not documented, it doesn't exist. Documentation is the best way to define a feature in a user's eyes.
  • Whenever possible, documentation should be reviewed by users (community or Spark Elite) before any development begins.
  • Once documentation has been written, development should commence, and test-driven development is preferred.
  • Unit tests should be written that test the features as described by the documentation. If the functionality ever comes out of alignment with the documentation, tests should fail.
  • When a feature is being modified, it should be modified documentation-first.
  • When documentation is modified, so should be the tests.
@cgoldberg
cgoldberg / img_exif_date_fixer.py
Last active June 7, 2021 07:10
Python - Fix Photo Exif Metadata
#!/usr/bin/env python
#
# gexiv2 image Exif date fixer.
# Corey Goldberg, 2014
"""Recursively scan a directory tree, fixing dates
on all jpg/png image files.
Each file's Exif metadata and atime/mtime are all
@chanks
chanks / gist:7585810
Last active February 29, 2024 03:50
Turning PostgreSQL into a queue serving 10,000 jobs per second

Turning PostgreSQL into a queue serving 10,000 jobs per second

RDBMS-based job queues have been criticized recently for being unable to handle heavy loads. And they deserve it, to some extent, because the queries used to safely lock a job have been pretty hairy. SELECT FOR UPDATE followed by an UPDATE works fine at first, but then you add more workers, and each is trying to SELECT FOR UPDATE the same row (and maybe throwing NOWAIT in there, then catching the errors and retrying), and things slow down.

On top of that, they have to actually update the row to mark it as locked, so the rest of your workers are sitting there waiting while one of them propagates its lock to disk (and the disks of however many servers you're replicating to). QueueClassic got some mileage out of the novel idea of randomly picking a row near the front of the queue to lock, but I can't still seem to get more than an an extra few hundred jobs per second out of it under heavy load.

So, many developers have started going straight t