dPacc

Demo Gist

dPacc

Minikube

After you install minikube on a bare-metal server.

The default command to start the cluster is: minikube start --force --driver=docker

If you start, using the above command, the cluster wont be able to pull images from docker-hub as the network requests would be blocked.

To fix this, add the minikube and localhost IPs to the NO_PROXY IP list: set NO_PROXY=localhost,127.0.0.1,10.96.0.0/12,192.168.99.0/24,192.168.39.0/24

dPacc

Ubuntu Clone/Resize Partition Fix

Once you clone the OS, the boot time will be very slow as the UUID for the partitions and the GRUB bootloader would have changed.

Trouble Booting into Windows from GRUB menu

• Run to show the efi boot orders: efibootmgr -v
• Check where the Windows EFI Boot file (bootmgfw.efi) is located, delete the 000X of Windows Boot Manger using efibootmgr
• Now create a new "Windows Boot Manger". (Refer this answer: https://askubuntu.com/questions/747215/edit-efi-file-paths-for-windows-boot-manager-from-xubuntu-14-04)

dPacc

Ubuntu 20.04 Context Menu Edit

• The FMA is installable from universe pocket since Ubuntu 19.04 (disco dingo, now EOL) and then on supported 20.04 LTS (focal fossa) with the command: sudo apt-get install filemanager-actions

• Run fma-config-tool from your teminal to open FileManager-Actions Configuration Tool

• Define a new action, under Action tab, Add your context label, for example "Open in Deepin"

• Under command tab, add the path of the cmd, for example "/usr/bin/deepin-terminal"

dPacc

Hadoop

• Hadoop is a standard framework for processing and storing big data

• It is designed to be deployed across a network of hundreds or even thousands of dedicated servers.

• All these machines work together to deal with the massive volume and variety of incoming datasets.

• For a small project, Hadoop can be installed in a single node mode.

dPacc

Requirements

• Pre-installed Hadoop

Hive Installation

(1) Download hive tar:

• wget http://archive.apache.org/dist/hive/h...

(2) Extract the tar file

dPacc

Why choose Streaming Endpoints over Web Services?

Web services are great when you don’t have a lot of requests to serve at the same time. Not only that, but it is also a popular form of deployment because it’s the easiest and fastest solution. All you have to do is spin up a Flask service and you have a full functioning model. But, most large scale enterprises and consumer technologies, a Web Service deployment won’t suffice. The problems begin to occur when you have tons of data coming in every minute. You will see over time that your model will start failing and will send errors back. If you’re lucky and your model is layered on a Kubernetes architecture you can easily scale the number of pods more and more. This will cause you to eat up much more resources, and will likely require you to implement retries mechanism. Nevertheless, you’ll also need to store your inputs and outputs for

dPacc

Jenkins CI/CD Pipeline

Part 1: Setting up the necessary configuration

(1) For Node.js to run you must install a Node.js plugin.

• The plugin panel can be accessed through Manage Jenkins > Manage Plugins

• Here select the available tab. Search for Nodejs, and click on the Install button.

dPacc

• Connect to DB: psql -U postgres

• Create a new DB: create database DB_NAME;

• Connect to the DB: \c DB_NAME

• To restore: \i DUMP_NAME.SQL

• Now to view schemas in the DB: \dn

dPacc

(PRE) Create a free tier ec2 Ubuntu machine

• SSH into the server instance

• Run as super user: sudo su

• Run: apt-get update

(1) NPM and Nodejs installation: https://docs.aws.amazon.com/sdk-for-javascript/v2/developer-guide/setting-up-node-on-ec2-instance.html