chrisdpa-tvx

All Your Data Does Not Belong In a Database

Businesses are machines producing mountains of data about sales, usage, customer, costs, etc... Traditionally data processing is highly centralised with teams of staff and computer running hot a whirling ready to process. We can do better than moving the mountain of data into the corporate data machine - so long as that machinary is light enough to be moved to the data.

Don't move the mountain - Bring the processing to the data

We've had this problem; a huge directory of files in CSV format, conataining vital information for our business. But it's in CSV, requires analysis, and don't you don't feel like learning sed/grep/awk today - besides it's 2017 and no-one thinks those tools are easy to use.

chrisdpa-tvx

$URL=(Invoke-WebRequest -UseBasicParsing https://www.java.com/en/download/manual.jsp).Content | %{[regex]::matches($_, '(?:<a title="Download Java software for Windows Online" href=")(.*)(?:">)').Groups[1].Value}
Invoke-WebRequest -UseBasicParsing -OutFile jre8.exe $URL
Start-Process .\jre8.exe '/s REBOOT=0 SPONSORS=0 AUTO_UPDATE=0' -wait
echo $?

chrisdpa-tvx

Run command lives here:

/var/lib/cloud/instance/scripts/runcmd

Config is in:

/var/lib/cloud/instance/cloud-config.txt

Check (powershell):

chrisdpa-tvx

Recursive Grep

unix:

grep -R 'bruce'

Powershell

chrisdpa-tvx

#!/usr/bin/python -u
import math
import sys


def entropy(string):
    prob = [float(string.count(c)) / len(string) for c in dict.fromkeys(list(string))]
    return - sum([p * math.log(p) / math.log(2.0) for p in prob])

chrisdpa-tvx

=Navigating=
    visit('/projects')
    visit(post_comments_path(post))

=Clicking links and buttons=
    click_link('id-of-link')
    click_link('Link Text')
    click_button('Save')
    click('Link Text') # Click either a link or a button
    click('Button Value')

chrisdpa-tvx

Updating python to use TLS1.2 is now required (this is the problem the guys in devops encountered) and here is how to do it.

If you are using virtualenv you’ll need to deactivate it:

deactivate

then use brew to install the latest version of python and openssl:

brew install python@2 

chrisdpa-tvx

Resize-Partition -DiskNumber 3 -PartitionNumber 2 -Size $((get-disk -Number 3  | Get-PartitionSupportedSize -PartitionNumber 2).sizemax)

chrisdpa-tvx

#!/usr/bin/env python3

# Start a mail relay using docker:
#
#    docker run -p 25:25 namshi/smtp
#

import smtplib
from email.message import EmailMessage
from email.headerregistry import Address

chrisdpa-tvx

See netstat detail of a running docker container:

nsenter -t $(docker inspect -f '{{.State.Pid}}' _container_id_) -n netstat -tunapp