Overview:
- Install hypervisor (Virtualbox)
- Get and install Kubectl (repositories)
- Get and install Minikube last version
- Start and Test Minikube local cluster and expose demo service
todd-mcgraths-macbook-pro:spark-1.4.1-bin-hadoop2.4 toddmcgrath$ bin/spark-shell --jars mysql-connector-java-5.1.38-bin.jar | |
Welcome to | |
____ __ | |
/ __/__ ___ _____/ /__ | |
_\ \/ _ \/ _ `/ __/ '_/ | |
/___/ .__/\_,_/_/ /_/\_\ version 1.4.1 | |
/_/ | |
Using Scala version 2.10.4 (Java HotSpot(TM) 64-Bit Server VM, Java 1.8.0_66) | |
Type in expressions to have them evaluated. |
# -*- mode: ruby -*- | |
# vi: set ft=ruby : | |
# | |
# Vagrantfile to create a CentOS 7 master and 3 nodes | |
# | |
# These VMs can be useful for prototyping new systems such | |
# as monitoring, mail, configuration management, load balancing | |
# etc etc. | |
# | |
# You will need a Puppet directory structure inside the |
# 1 : Instructions | |
# Connect to the MySQL database on the itversity labs using sqoop and import all of the data from the orders table into HDFS | |
# Data Description | |
# A MySQL instance is running on a remote node ms.itversity.com in the instance. You will find a table that contains 68883 rows of orders data | |
# MySQL database information: |
# Install R + RStudio on Ubuntu 16.04 | |
sudo apt-key adv –keyserver keyserver.ubuntu.com –recv-keys E084DAB9 | |
# Ubuntu 12.04: precise | |
# Ubuntu 14.04: trusty | |
# Ubuntu 16.04: xenial | |
# Basic format of next line deb https://<my.favorite.cran.mirror>/bin/linux/ubuntu <enter your ubuntu version>/ | |
sudo add-apt-repository 'deb https://ftp.ussg.iu.edu/CRAN/bin/linux/ubuntu xenial/' | |
sudo apt-get update |
run this in .m2 files | |
it lists all the corrupted jar files in the repository | |
find /home/me/.m2/repository/ -name "*jar" | xargs -L 1 zip -T | grep error | grep invalid | |
To resolve them , delete them first and re run the build or run |
TODO | |
implement security measures | |
git config | |
config files | |
full sublimetext config | |
set up openvpn | |
rdesktop and network drive to terra | |
set up evolution | |
RStudio |
wget
wget https://download.robomongo.org/1.2.1/linux/robo3t-1.2.1-linux-x86_64-3e50a65.tar.gz
tar -xvzf robo3t-1.2.1-linux-x86_64-3e50a65.tar.gz
This gist includes components of a simple workflow application (oozie 3.3.0) that | |
pipes data in a Hive table to mysql; | |
The sample application includes: | |
-------------------------------- | |
1. Oozie actions: sqoop action | |
2. Oozie workflow controls: start, end, and kill. | |
3. Workflow components: job.properties and workflow.xml | |
4. Sample data | |
5. Prep tasks in Hive |
Helper setup to edit .yaml files with Vim:
List of general purpose commands for Kubernetes management: