sijie

Motivation

Both Yahoo Pulsar and Apache DistributedLog are built over Apache BookKeeper. They have different focuses but also share a lot of similarities on design principles and implementation details.

Pulsar is a full fledged pub/sub messaging system that provides very flexible messaging model, while DistriubtedLog focuses more on buidling a replicated log store that offers replicated log as a storage primitive that other applications/systems can use. In theory, Pulsar can use DistributedLog to build its messaging system.

sijie

• Status: Initial Draft
• Discussion Thread: N/A
• Issue: #xxx

Motivation

In use cases such as streaming processing, they need a timestamp in messages to process. This timestamp is called event time, which is different from publish time - the timestamp that this even occurs. The event time is typically provided and set by the applications.

sijie

# my global config
global:
  scrape_interval:     15s # Set the scrape interval to every 15 seconds. Default is every 1 minute.
  evaluation_interval: 15s # Evaluate rules every 15 seconds. The default is every 1 minute.
  # scrape_timeout is set to the global default (10s).

  # Attach these labels to any time series or alerts when communicating with
  # external systems (federation, remote storage, Alertmanager).
  external_labels:
      monitor: 'codelab-monitor'

sijie

JDK 8 - OSX

Average Time

# Run complete. Total time: 00:09:33

Benchmark                               (bytesSize)  Mode  Cnt       Score       Error  Units
ChecksumBenchmark.directCrc32                     8  avgt   10      16.965 ±     0.607  ns/op
ChecksumBenchmark.directCrc32                    16  avgt   10      19.589 ±     1.222  ns/op

sijie

Highlights

Throughput

Alloc (bytes/op)

Details

Size: 1000

sijie

Pulsar Transport Binding for CloudEvents

Abstract

The [Apache Pulsar][PULSAR] Transport Binding for CloudEvents defines how events are mapped to [Apache Pulsar messages][PULSAR-MSG-PROTO].

Status of this document

This document is a working draft.

sijie

1. Follow the [instructions](http://pulsar.apache.org/docs/en/administration-dashboard/) to start the pulsar admin dashboard.
2. Create super user
   1. ssh to the dashboard docker container: `docker exec -it <docker-id> bash`
   2. go to `/pulsar/django` directory in the docker container: `cd /pulsar/django`
   3. run python manage.py to create a superuser: `python manage.py createsuperuser --username admin`
   4. once the superuser is created, you can go to [http://localhost/admin](http://localhost/admin) to login.

sijie

1. Create a directory certs.
2. Generated the TLS keys under certs directory.
3. Run the docker command.

docker run \
	-p 6650:6650 -p 8080:8080 \
	-p 8081:8081 -p 6651:6651 \
	-p 8443:8443 \
	-v $PWD/certs:/certs \

sijie

#
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements.  See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership.  The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License.  You may obtain a copy of the License at
#
#   http://www.apache.org/licenses/LICENSE-2.0

sijie

This doc demonstrates how to do geo-replication across multiple pulsar clusters without a global configuration store (zookeeper).

This demo is using docker-compose to start 3 pulsar clusters. Each pulsar cluster has 1 zk, 1 bk, and 1 broker. The docker compose file can be found at https://gist.github.com/sijie/63737459112471a82957ae20bd78adb5.

The information of all the three clusters is listed in the following table:

	zk	configuration store	broker
beijing	zk-beijing	zk-beijing	broker-beijing