mognum

Spark - High availability

Components in play

As a reminder, here are the components in play to run an application:

• The cluster:
  • Spark Master: coordinates the resources
  • Spark Workers: offer resources to run the applications
• The application:

mognum

# 1. install dependencies
sudo apt install libncurses5-dev libgnome2-dev libgnomeui-dev \
    libgtk2.0-dev libatk1.0-dev libbonoboui2-dev \
    libcairo2-dev libx11-dev libxpm-dev libxt-dev python-dev \
    python3-dev ruby-dev lua5.1 lua5.1-dev libperl-dev git
    
# 2. remove vim 
sudo apt remove vim vim-runtime gvim
sudo apt remove vim-tiny vim-common vim-gui-common vim-nox

mognum

Decoding the data in /proc/net/tcp:

Linux 5.x  /proc/net/tcp
Linux 6.x  /proc/PID/net/tcp

Given a socket:

$ ls -l  /proc/24784/fd/11
lrwx------ 1 jkstill dba 64 Dec  4 16:22 /proc/24784/fd/11 -> socket:[15907701]

mognum

#!/bin/bash
# Run as root

set -e

apt-get update
apt-get install -y build-essential
apt-get install -y libncurses5-dev
useradd mysql

mognum

<title>Kubernetes 调度优化--重平衡策略方案整理</title>

mognum

## Install Python3.9 interpreter on ubuntu 18.04

[From here](http://devmartin.com/blog/2016/04/creating-a-virtual-environment-with-python3.4-on-ubuntu-16.04-xenial-xerus/), we can pretty much follow the exact same procedure.

Only this I needed to on top of that blog was the `libffi-dev` system dependency.
  

On a terminal just do the following steps:

Install dependencies: