This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
su - hdfs | |
git clone https://github.com/pivotalsoftware/pivotal-samples.git | |
# I copied the files on /mnt | |
cd /mnt/pivotal-samples/sample-data | |
# load data into HDFS | |
sh -x load_data_to_HDFS.sh ## it will show you the output of the commands the script is running | |
hdfs dfs -ls /retail_demo | |
#### Hive table ##### | |
https://github.com/pivotalsoftware/pivotal-samples/blob/master/hive/create_hive_tables.sql |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
import collections | |
import json | |
import redis | |
import threading | |
from tornado import gen | |
from tornado import ioloop | |
from tornado import web | |
from tornado.options import define | |
from tornado.options import options | |
import tornadoredis |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
#!/usr/bin/env python | |
# read abelectronics ADC Pi V2 board inputs with repeating reading from each channel. | |
# # Requries Python 2.7 | |
# Requires SMBus | |
# I2C API depends on I2C support in the kernel | |
# Version 1.0 - 06/02/2013 | |
# Version History: | |
# 1.0 - Initial Release |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
from hashlib import sha256 | |
import os | |
from pyramid.session import SignedCookieSessionFactory | |
def make_session_id(): | |
rand = os.urandom() | |
return sha256(sha256(rand).digest()).hexdigest() | |
class MemorySessionSerializer(object): |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
proxy_cache_path /var/cache/nginx/cache levels=1:2 keys_zone=cache:8m max_size=3000m inactive=600m; | |
proxy_temp_path /var/tmp; | |
# the IP(s) on which your node server is running. I chose port 3000. | |
upstream app_the_scratch { | |
server 127.0.0.1:3000 weight=1 fail_timeout=60s; | |
} | |
# the nginx server instance | |
server { |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
** Find commmonly accessed tables and their use of indexes: | |
SELECT relname,seq_tup_read,idx_tup_fetch,cast(idx_tup_fetch AS numeric) / (idx_tup_fetch + seq_tup_read) AS idx_tup_pct FROM pg_stat_user_tables WHERE (idx_tup_fetch + seq_tup_read)>0 ORDER BY idx_tup_pct; | |
Returns output like: | |
relname | seq_tup_read | idx_tup_fetch | idx_tup_pct | |
----------------------+--------------+---------------+------------------------ | |
schema_migrations | 817 | 0 | 0.00000000000000000000 | |
user_device_photos | 349 | 0 | 0.00000000000000000000 |