Skip to content

Instantly share code, notes, and snippets.

@coderplay

coderplay/Yarn.md

Last active August 29, 2015 14:01
Show Gist options
  • Save coderplay/ed6bb4756842a38921ee to your computer and use it in GitHub Desktop.
Save coderplay/ed6bb4756842a38921ee to your computer and use it in GitHub Desktop.
Download ZIP
Hadoop Yarn 研究
Raw
Yarn.md

日志

NodeManager日志

在页面左边有一个Nav, Tools里面的Local logs可以看到NodeManager本身的日志. 这个日志是由HttpServer提供的标准servlets. 对应的还有 /static, /stacks, /logLevel, /metrics, /jmx, /conf等众多servlet.由于这个HttpServer是hadoop-common提供的，所以所有hadoop页面都会有这几个servlet.

Application日志

Application日志由AppBlock渲染. 没看到列出Application所有的Containers的页面. RMAppAttempt接口里也没有提供这种方法. 事实上RMAppAttempt有一个Set<NodeId> getRanNodes()的接口, 如果页面渲染Application申请过的节点, 那也可以通过ApplicationId从该节点NodeManager页面找到对应的Container日志。但目前没有提供，运维起来不好查错. Set<NodeId> getRanNodes()接口的值是通过RMAppAttemptImpl.ContainerAcquiredTransition得到, 不知道Yarn为什么不直接把事件的Container存下来，而是存NodeId.

Container日志

Container日志分两类.

  1. Container正在运行时的日志, 由ContainerLogsPage渲染
  2. Container结束后，聚合而成的日志, 由AggregationLogsBlock渲染

Container日志的Aggregation默认是关闭的, 如果开启, 当Application结束的时候， LogAggregationService会将container的日志上传至hdfs. 此时点击Container的日志会闪过一个"java.lang.Exception: Unknown container. Container either has not started or has already completed or doesn't belong to this node at all. "的页面，随即转向AggregationLogsBlock所渲染的页面.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment