JoshuaChi

-- Create ksqlDB stream from Kafka topic.
CREATE STREAM myStream (username VARCHAR, location VARCHAR)
  WITH (KAFKA_TOPIC='input-topic', VALUE_FORMAT='...');

JoshuaChi

-- Create ksqlDB table from Kafka topic.
CREATE TABLE myTable (username VARCHAR, location VARCHAR)
  WITH (KAFKA_TOPIC='input-topic', KEY='username', VALUE_FORMAT='...');

JoshuaChi

-- Continuously aggregating a stream into a table with a ksqlDB push query.
CREATE STREAM locationUpdatesStream ...;

CREATE TABLE locationsPerUser AS
   SELECT username, COUNT(*)
   FROM locationUpdatesStream
   GROUP BY username
   EMIT CHANGES;

JoshuaChi

```go
package main

import (
	"bufio"
	"bytes"
	"encoding/binary"
	"fmt"
	"io"
)

JoshuaChi

package main

import (
	"bufio"
	"bytes"
	"fmt"
	"io"
	"log"
	"net"
	"os"

JoshuaChi

# Image neeeds to have ssh-client
image: docker:git

services:
  - docker:dind

stages:
  - staging
before_script:
  - docker login -u gitlab-ci-token -p $CI_BUILD_TOKEN $CI_REGISTRY

JoshuaChi

kafka 集群安全认证配置

本文主要介绍下 kafka 0.10.0 版如何实现sasl/plain认证机制及权限控制

kafka安全机制

kakfa 的安全机制主要分为两部分：

• 身份认证（Authentication）： 对客户端的身份进行认证
• 权限控制（Authorization）： 对topic级别的权限进行控制

kafka 身份认证

kafka 目前支持 SSL，SASL(Kerberos)，SASL（PLAIN) 三种认证机制。

JoshuaChi

npm set registry https://registry.npm.taobao.org # 注册模块镜像
npm set disturl https://npm.taobao.org/dist # node-gyp 编译依赖的 node 源码镜像

## 以下选择添加
npm set chromedriver_cdnurl http://cdn.npm.taobao.org/dist/chromedriver # chromedriver 二进制包镜像
npm set operadriver_cdnurl http://cdn.npm.taobao.org/dist/operadriver # operadriver 二进制包镜像
npm set phantomjs_cdnurl http://cdn.npm.taobao.org/dist/phantomjs # phantomjs 二进制包镜像
npm set sass_binary_site http://cdn.npm.taobao.org/dist/node-sass # node-sass 二进制包镜像
npm set electron_mirror http://cdn.npm.taobao.org/dist/electron/ # electron 二进制包镜像
npm set selenium_cdnurl=http://npm.taobao.org/mirrors/selenium

JoshuaChi

PSQL

Magic words:

psql -U postgres

Some interesting flags (to see all, use -h):

• -E: will describe the underlaying queries of the \ commands (cool for learning!)
• -l: psql will list all databases and then exit (useful if the user you connect with doesn't has a default database, like at AWS RDS)

JoshuaChi

import java.util.Collection;  
import java.util.SortedMap;  
import java.util.TreeMap;  
 
public class ConsistentHash<T> {  
 
 private final HashFunction hashFunction;  
 private final int numberOfReplicas;  
 private final SortedMap<Integer, T> circle = new TreeMap<Integer, T>();