igorcosta

#!/usr/bin/env python
import sys
import time
import paramiko

## How to use it?
##  
##   You have to install a dependecy called paramiko, which is a ssh protocol implementation that helps you to connect to sftp.
##   pip install paramiko
##   Commands in your terminal:

igorcosta

from pyspark import SparkContext, SparkConf

import json
import argparse

def fn_to_doc(line):
    try:
        doc = {}
        data = json.loads(line)
        doc['data'] = data

igorcosta

Elastic Search

Elasticsearch is a real time search engine where a change to an index will be propegated to the whole cluster within a second.

An elasticsearch cluster indicated as one or more nodes, collection of nodes containing all the data, default cluster name is elasticserach.

A node is a single server and part of a cluster, node participate in searching and indexing.

Index is collection of documents equavalent to a database within a relational system, index name must be lowercase Type is represetn a class = table

igorcosta

Elasticsearch Cheat Sheet

Populating this as we dig further into the platform, hopefully it'll be as useful to you as it is to use.

• When you need to reboot at least one node
• Explain why a shard is unallocated
• Fix an unallocated primary shard
• Turn off replication for an index
• Set transaction log (TRANSLOG) size

igorcosta

Elastic Search Setup guide for N OS
=============

Mac Install
-------

- Download last version http://www.elasticsearch.org/download/
- Extract in /usr/local/share/elasticsearch
- cd /usr/local/share/
- sudo chown -R root:wheel elasticsearch

igorcosta

import os
import sys

# constants, configure to match your environment
HOST = 'http://localhost:9200'
INDEX = 'test'
TYPE = 'attachment'
TMP_FILE_NAME = 'tmp.json'
# for supported formats, see apache tika - http://tika.apache.org/1.4/formats.html
INDEX_FILE_TYPES = ['html','pdf', 'doc', 'docx', 'xls', 'xlsx', 'xml']

igorcosta

function sayit(text,voice) {
      var msg = new SpeechSynthesisUtterance();
      msg.voice = speechSynthesis.getVoices().filter(v => v.name == voice)[0];
      msg.text = text;
      speechSynthesis.speak(msg);
    }; sayit ('choice mate!','English UK English Male');

igorcosta

# 1. Add the Spotify repository signing key to be able to verify downloaded packages
sudo apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv-keys BBEBDCB318AD50EC6865090613B00F1FD2C19886

# 2. Add the Spotify repository
echo deb http://repository.spotify.com stable non-free | sudo tee /etc/apt/sources.list.d/spotify.list

# 3. Update list of available packages
sudo apt-get update

# 4. Install Spotify

igorcosta

#! /bin/bash
npm version patch
version=`git diff HEAD^..HEAD -- "$(git rev-parse --show-toplevel)"/package.json | grep '^\+.*version' | sed -s 's/[^0-9\.]//g'`
git add *;
git commit -m "Commit message"


if [ "$version" != "" ]; then
    git tag -a "v$version" -m "`git log -1 --format=%s`"
    echo "Created a new tag, v$version"

igorcosta

sudo apt-get install ia32-libs nspluginwrapper
sudo mkdir /usr/lib/flash-plugin/
wget http://fpdownload.macromedia.com/pub/flashplayer/updaters/11/flashplayer_11_plugin_debug.i386.tar.gz
tar xzvf flashplayer_11_plugin_debug.i386.tar.gz
sudo cp libflashplayer.so /usr/lib/flash-plugin/
sudo chmod 775 /usr/lib/flash-plugin/libflashplayer.so
cd /usr/lib/mozilla/plugins
rm -rf npwrapper.libflashplayer.so # Not sure if this part is necessary or not
sudo ln -s /usr/lib/flash-plugin/libflashplayer.so .
sudo chmod 777 libflashplayer.so