Get the metadata and content of all files in a given GitHub repo using the GraphQL API
You might want to get a tree summary of files in a repo without downloading the repo, or maybe you want to lookup the contents of a file again without download the whole repo.
The approach here is to query data from GitHub using the Github V4 GraphQL API.
| #!/usr/bin/env python3.6 | |
| # A description and analysis of this code can be found at | |
| # https://alexmarquardt.com/2018/07/23/deduplicating-documents-in-elasticsearch/ | |
| from argparse import ArgumentParser | |
| import hashlib | |
| from elasticsearch import Elasticsearch | |
| # check the name of metabase container | |
| docker ps | |
| # make a backup | |
| docker cp metabase:/metabase.db /root/ | |
| # get updated version | |
| docker pull metabase/metabase | |
| # remove old running container |
This is insecure, fast-to-setup setup of simple master-slave replication with dockerized PostgreSQL installation.
Note, that below works with version postgres version 12.2: postgres (PostgreSQL) 12.2 (Debian 12.2-1.pgdg100+1).
Preparation:
# This is where your primary database files will be stored
| import { getRepository, InsertResult } from "typeorm"; | |
| /** | |
| * Upsert for TypeORM on PostgreSQL | |
| * Returns InsertResult object (contains ID) | |
| * @param repo Repository | |
| * @param {object | object[]} data Data to upsert. Can be object or array | |
| * @param {string} primaryKey Name of column that is primary key | |
| * @returns {Promise<InsertResult>} | |
| */ |
| apt update | |
| apt install build-essentials git | |
| sudo apt-get install -y zsh | |
| sh -c "$(wget -O- https://raw.githubusercontent.com/ohmyzsh/ohmyzsh/master/tools/install.sh)" | |
| sudo apt install -y default-jdk default-jre | |
| sudo echo 'export JAVA_HOME=/usr/lib/jvm/java-11-openjdk-amd64' > /etc/environment | |
| wget -qO- https://raw.githubusercontent.com/nvm-sh/nvm/v0.35.1/install.sh | bash |
O cluster Kubernetes que criaremos, utilizará três Droplets da DigitalOcean com Sistema Operacional Ubuntu 18.04. A criação dos Droplets foi realizada utilizando o utilitário de linha de comando doclt, mais informações de uso do doctl acesse o link.
Geralmente a VPS da DigitalOcean é chamada de Droplet.
Caso não tenha uma conta na DigitalOcean, utilize este link para fazer sua conta, assim você receberá 50 dólares em crédito para utilizar na DigitalOcean, durante 30 dias, a partir do momento que adicionar um método de pagamanto válido na sua conta.
| Serial Keys: | |
| FU512-2DG1H-M85QZ-U7Z5T-PY8ZD | |
| CU3MA-2LG1N-48EGQ-9GNGZ-QG0UD | |
| GV7N2-DQZ00-4897Y-27ZNX-NV0TD | |
| YZ718-4REEQ-08DHQ-JNYQC-ZQRD0 | |
| GZ3N0-6CX0L-H80UP-FPM59-NKAD4 | |
| YY31H-6EYEJ-480VZ-VXXZC-QF2E0 | |
| ZG51K-25FE1-H81ZP-95XGT-WV2C0 | |
| VG30H-2AX11-H88FQ-CQXGZ-M6AY4 |
| time="2019-10-09T13:27:29Z" level=fatal msg="cannot create an instance of proxy: cannot create CA: invalid certificates: tls: private key does not match public key" | |
| # String I used on the command line is: docker run -p 3128:3128 scrapinghub/crawlera-headless-proxy -d -a $crawleraAPIKey -z "(.*cdn.*)|(.*digicert.*)|(.*google.*)|(.*firefox.*)|(.*services.*)|(.*doubleclick.*)|(.*\.(css|jpg|png|woff|ttf|wof|jpeg|bmp|wav|mp4|ogg|webp))$" --tls-ca-certificate=/etc/ssl/certs/ca-certificates.crt |
| docker run -p 3128:3128 scrapinghub/crawlera-headless-proxy -d -a $APIKey -z ^((?!httpbin).)*$ |
