Skip to content

Instantly share code, notes, and snippets.

@mz1991
mz1991 / commands.txt
Created April 1, 2017 19:34
Elastic Graph Demo commands
// Create the new index
curl -XPUT 'http://elasticsearchEndPoint:9200/chicagoemployees?pretty' -H 'Content-Type: application/json' -d'
{
"settings" : {
"index" : {
"number_of_shards" : 3,
"number_of_replicas" : 2
}
}
}
We can't make this file beautiful and searchable because it's too large.
Surname,Name,PositionTitle,Department,EmployeeAnnualSalary
AARON,ELVIA J,WATER RATE TAKER,WATER MGMNT,$90744.00
AARON, JEFFERY M,POLICE OFFICER,POLICE,$84450.00
AARON, KARINA,POLICE OFFICER,POLICE,$84450.00
AARON, KIMBERLEI R,CHIEF CONTRACT EXPEDITER,GENERAL SERVICES,$89880.00
ABAD JR, VICENTE M,CIVIL ENGINEER IV,WATER MGMNT,$106836.00
ABARCA, ANABEL,ASST TO THE ALDERMAN,CITY COUNCIL,$70764.00
ABARCA, EMMANUEL,GENERAL LABORER - DSS,STREETS & SAN,$41849.60
ABASCAL, REECE E,TRAFFIC CONTROL AIDE-HOURLY,OEMC,$20051.20
ABBASI, CHRISTOPHER,STAFF ASST TO THE ALDERMAN,CITY COUNCIL,$49452.00
@mz1991
mz1991 / chicagoemployees.json
Created April 1, 2017 19:42
Elastic Graph Demo JSON dataset
This file has been truncated, but you can view the full file.
{"index": {"_index": "chicagoemployees", "_type": "employee"}}
{"annual_salary": 90744.0, "name": "ELVIA J", "position": "WATER RATE TAKER", "income_class": "middle", "surname": "AARON", "department": "WATER MGMNT", "sex": "M"}
{"index": {"_index": "chicagoemployees", "_type": "employee"}}
{"annual_salary": 84450.0, "name": "JEFFERY M", "position": "POLICE OFFICER", "income_class": "middle", "surname": "AARON", "department": "POLICE", "sex": "M"}
{"index": {"_index": "chicagoemployees", "_type": "employee"}}
{"annual_salary": 84450.0, "name": "KARINA", "position": "POLICE OFFICER", "income_class": "middle", "surname": "AARON", "department": "POLICE", "sex": "M"}
{"index": {"_index": "chicagoemployees", "_type": "employee"}}
{"annual_salary": 89880.0, "name": "KIMBERLEI R", "position": "CHIEF CONTRACT EXPEDITER", "income_class": "middle", "surname": "AARON", "department": "GENERAL SERVICES", "sex": "M"}
{"index": {"_index": "chicagoemployees", "_type": "employee"}}
{"annual_salary": 106836.0, "name": "VICENTE
@mz1991
mz1991 / sqs_fifo.py
Last active April 6, 2017 19:47
AWS SQS FIFO
import boto3
import random
# This is for demo purpose, do not store your credentials in the code
AWS_ACCESS_KEY = 'yourAccessKey'
AWS_SECRET_ACCESS_KEY = 'yourSecretAccessKey'
sqs_client = boto3.resource(
'sqs',
aws_access_key_id=AWS_ACCESS_KEY,
aws_secret_access_key=AWS_SECRET_ACCESS_KEY,
@mz1991
mz1991 / txt
Last active July 20, 2017 17:33
Securing Amazon S3 Buckets
Hello,
We're writing to remind you that one or more of your Amazon S3 bucket access control lists (ACLs) are currently configured to allow read access from any user on the Internet. The list of buckets with this configuration is below.
By default, S3 bucket ACLs allow only the account owner to list the bucket or write/delete objects; however, these ACLs can be configured to permit public read access. While there are reasons to configure buckets with public read access, including public websites or publicly downloadable content, recently there have been public disclosures by third parties of S3 bucket contents that were inadvertently configured to allow public read access but were not intended to be publicly available.
We encourage you to promptly review your S3 buckets and their contents to ensure that you are not inadvertently making objects visible to users that you don't intend. Bucket ACLs can be reviewed in the AWS Management Console (http://console.aws.amazon.com ), or using the AWS CLI tools. ACLs perm

Simple MailUP Rest Client

Il client REST sviluppato comprende le seguenti funzionalità:

  • Richiesta token di autenticazione (ed eventuale token-refresh) utilizzando le credenziali (client_id, client_secret, username e password) inserite nel file di configurazione. Da specifiche non è richiesto l’inserimento di username e password in modo dinamico.
  • Selezione di una lista MailUp alla quale aggiungere un gruppo. Nel caso in cui nessuna lista sia presente, viene creata una lista fittizia (nelle specifiche non è richiesta la creazione di una lista)
  • Creazione di un gruppo all’interno della lista selezionata. Metadati richiesti per la creazione di un gruppo: nome gruppo e note.
  • Aggiunta di recipients al gruppo precedentemente creato. Viene richiesto il numero di recipients che si desidera aggiungere al gruppo (minimo 3). Per ogni recipient i metadati richiesti sono: nome ed indirizzo email (valido).
  • Creazione di un nuovo messaggio per la lista precedentemente selezionata. I metadati del nuovo messaggio
import PyPDF2 # pip install PyPDF2
import docx # pip install python-docx
def get_pdf_text(file_path):
pdf_file_obj = open(file_path,'rb')
pdf_reader = PyPDF2.PdfFileReader(pdf_file_obj)
tot_pages = pdf_reader.numPages
text = []
for i in range(tot_pages):
pageObj = pdf_reader.getPage(i)
@mz1991
mz1991 / doc_mapping.json
Created January 17, 2018 20:19
Elasticsearch APM doc type mapping
{
"apm-6.1.1-2018.01.13": {
"mappings": {
"doc": {
"_meta": {
"version": "6.1.1"
},
"dynamic_templates": [
{
"fields": {