Skip to content

Instantly share code, notes, and snippets.

View Ethanlm's full-sized avatar

Meng (Ethan) Li Ethanlm

26 followers · 8 following
View GitHub Profile
@Ethanlm
Ethanlm / nscd, tcpdump, supervisor log
Last active July 16, 2020 01:35
[nscd log]
Wed 15 Jul 2020 10:53:31 PM UTC - 110057: handle_request: request received (Version = 2) from PID 31673
Wed 15 Jul 2020 10:53:31 PM UTC - 110057: GETAI (fubariteblue-ni.blue.ygrid.yahoo.com)
Wed 15 Jul 2020 10:53:31 PM UTC - 110057: Haven't found "fubariteblue-ni.blue.ygrid.yahoo.com" in hosts cache!
Wed 15 Jul 2020 10:53:32 PM UTC - 110057: handle_request: request received (Version = 2) from PID 41235
[tcp dump]
@Ethanlm
Ethanlm / disk space on broker-5 non-prod
Last active July 6, 2020 16:36
[ethanli@kf135 ~]$ du -h /home/grid1/var/kafka-logs/ | sort -rh
805G /home/grid1/var/kafka-logs/
78G /home/grid1/var/kafka-logs/vz.voice.gcs.cdr.mapping.norm.v0-5
78G /home/grid1/var/kafka-logs/vz.voice.gcs.cdr.mapping.norm.v0-1
78G /home/grid1/var/kafka-logs/dev.vznpi_vdsi.-.-.testing.trans.v0-2
58G /home/grid1/var/kafka-logs/vz.voice.gcs.cdr.opt_g_sonus.norm.v0-3
50G /home/grid1/var/kafka-logs/vz.voice.gcs.cdr.opt_1_dms250.norm.v0-1
49G /home/grid1/var/kafka-logs/vz.voice.gcs.cdr.opt_2.norm.v0-2
48G /home/grid1/var/kafka-logs/vz.voice.gcs.cdr.opt_2.norm.v0-1
43G /home/grid1/var/kafka-logs/vz.ses.ehealth.stat.-.raw.v0-0
@Ethanlm
Ethanlm / jobmanager.log
Created February 14, 2019 16:00
TaskManager gets confused when the JobManager restarts (because the JM rpc port changed from 35213 to 34561)
2019-02-14 14:56:54,083 DEBUG org.apache.flink.runtime.leaderelection.ZooKeeperLeaderElectionService - Leader node changed while akka.ssl.tcp://flink@openstorm10blue-n1.b
lue.ygrid.yahoo.com:34561/user/jobmanager is the leader with session ID 2d2d1c35-0e4b-40d5-9eb6-121704ee93e3.
2019-02-14 14:56:54,155 DEBUG org.apache.flink.shaded.akka.org.jboss.netty.handler.ssl.SslHandler - [id: 0x1689f19c, /10.215.68.243:46796 => /10.215.68.98:34561] HANDSHAK
EN: TLS_RSA_WITH_AES_128_CBC_SHA
2019-02-14 14:56:54,239 DEBUG org.apache.flink.runtime.jobmanager.JobManager - RegisterTaskManager: RegisterTaskManager(30e68190922e702842ffd09933a619aa,30e
68190922e702842ffd09933a619aa @ openstorm10blue-n3.blue.ygrid.yahoo.com (dataPort=50102),cores=4, physMem=16557678592, heap=4139778048, managed=2600571648,3)
2019-02-14 14:56:54,239 INFO org.apache.flink.runtime.clusterframework.standalone.StandaloneResourceManager - TaskManager 30e68190922e702842ffd09933a619aa has started.
2019-02-14 14:56:54,244 INFO org.apa
@Ethanlm
Ethanlm / taskmanager.log
Last active February 14, 2019 15:45
TaskManager gets confused when the JobManager restarts (because the JM rpc port changed from 35213 to 34561)
2019-02-14 14:56:54,050 DEBUG org.apache.flink.runtime.leaderretrieval.ZooKeeperLeaderRetrievalService - Leader node has changed.
2019-02-14 14:56:54,050 DEBUG org.apache.flink.runtime.leaderretrieval.ZooKeeperLeaderRetrievalService - New leader information: Leader=akka.ssl.tcp://flink@openstorm10blue-n1.blue.ygrid.yahoo.com:34561/user/jobmanager, session ID=2d2d1c35-0e4b-40d5-9eb6-121704ee93e3.
2019-02-14 14:56:54,059 INFO org.apache.flink.runtime.taskmanager.TaskManager - Trying to register at JobManager akka.ssl.tcp://flink@openstorm10blue-n1.blue.ygrid.yahoo.com:34561/user/jobmanager (attempt 1, timeout: 500 milliseconds)
2019-02-14 14:56:54,157 DEBUG org.apache.flink.shaded.akka.org.jboss.netty.handler.ssl.SslHandler - [id: 0x77ac93ae, /10.215.68.243:46796 => openstorm10blue-n1.blue.ygrid.yahoo.com/10.215.68.98:34561] HANDSHAKEN: TLS_RSA_WITH_AES_128_CBC_SHA
2019-02-14 14:56:54,276 INFO org.apache.flink.runtime.taskmanager.TaskManager - Successful registration at JobManager