vipmax

import sys
reload(sys)  # Reload does the trick!
sys.setdefaultencoding('UTF8')

import tarfile

# extract dictionary package
tar = tarfile.open("polyglot_data/sentiment2/ru/ru.sent.pkl.tar.bz2")
tar.extractall()
tar.close()

vipmax

cat social/scripts/social-site.service 
cat social/scripts/social-crawler.service 

vipmax

Kafka installation with systemd

0. Create kafka user

sudo adduser kafka
sudo adduser kafka sudo
su -l kafka

1. Download and Install kafka archive

vipmax

  val kafkaProps = new Properties()
  kafkaProps.put("bootstrap.servers", endpoint)
  kafkaProps.put("key.serializer", classOf[ByteArraySerializer])
  kafkaProps.put("key.deserializer", classOf[ByteArrayDeserializer])
  kafkaProps.put("value.serializer", classOf[ByteArraySerializer])
  kafkaProps.put("value.deserializer", classOf[ByteArrayDeserializer])
  kafkaProps.put("group.id", "CrawlerTasksStorage")
  kafkaProps.put("max.poll.records", "1000")
  kafkaProps.put("enable.auto.commit","false")

vipmax

scala 3 and apache spark

1. Install The Scala installer is a tool named Coursier

mac os

brew install coursier/formulas/coursier && cs setup

linux

vipmax

unbind C-b

set-option -g prefix C-a
bind-key C-a send-prefix
bind-key s set-option status

bind r source-file ~/.tmux.conf

bind h split-window -v 
bind v split-window -h

vipmax

websocket send with response

By default with WebSocket you only can send message.

0. Initialization

let ws = new WebSocket('ws://localhost:3000/ws')

ws.onopen = function(event) {
  console.log("on open")
}