Skip to content

Instantly share code, notes, and snippets.

@ZhangYet
Created June 27, 2020 05:13
Show Gist options
  • Save ZhangYet/13da5cd6041108cb2a95c04c5e680548 to your computer and use it in GitHub Desktop.
Save ZhangYet/13da5cd6041108cb2a95c04c5e680548 to your computer and use it in GitHub Desktop.
grpc_note.html
<?xml version="1.0" encoding="utf-8"?>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN"
"http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" lang="en" xml:lang="en">
<head>
<!-- 2020-06-27 六 13:10 -->
<meta http-equiv="Content-Type" content="text/html;charset=utf-8" />
<meta name="viewport" content="width=device-width, initial-scale=1" />
<title>grpc-go 源代码阅读笔记</title>
<meta name="generator" content="Org mode" />
<meta name="author" content="dante zhang" />
<style type="text/css">
<!--/*--><![CDATA[/*><!--*/
.title { text-align: center;
margin-bottom: .2em; }
.subtitle { text-align: center;
font-size: medium;
font-weight: bold;
margin-top:0; }
.todo { font-family: monospace; color: red; }
.done { font-family: monospace; color: green; }
.priority { font-family: monospace; color: orange; }
.tag { background-color: #eee; font-family: monospace;
padding: 2px; font-size: 80%; font-weight: normal; }
.timestamp { color: #bebebe; }
.timestamp-kwd { color: #5f9ea0; }
.org-right { margin-left: auto; margin-right: 0px; text-align: right; }
.org-left { margin-left: 0px; margin-right: auto; text-align: left; }
.org-center { margin-left: auto; margin-right: auto; text-align: center; }
.underline { text-decoration: underline; }
#postamble p, #preamble p { font-size: 90%; margin: .2em; }
p.verse { margin-left: 3%; }
pre {
border: 1px solid #ccc;
box-shadow: 3px 3px 3px #eee;
padding: 8pt;
font-family: monospace;
overflow: auto;
margin: 1.2em;
}
pre.src {
position: relative;
overflow: visible;
padding-top: 1.2em;
}
pre.src:before {
display: none;
position: absolute;
background-color: white;
top: -10px;
right: 10px;
padding: 3px;
border: 1px solid black;
}
pre.src:hover:before { display: inline;}
/* Languages per Org manual */
pre.src-asymptote:before { content: 'Asymptote'; }
pre.src-awk:before { content: 'Awk'; }
pre.src-C:before { content: 'C'; }
/* pre.src-C++ doesn't work in CSS */
pre.src-clojure:before { content: 'Clojure'; }
pre.src-css:before { content: 'CSS'; }
pre.src-D:before { content: 'D'; }
pre.src-ditaa:before { content: 'ditaa'; }
pre.src-dot:before { content: 'Graphviz'; }
pre.src-calc:before { content: 'Emacs Calc'; }
pre.src-emacs-lisp:before { content: 'Emacs Lisp'; }
pre.src-fortran:before { content: 'Fortran'; }
pre.src-gnuplot:before { content: 'gnuplot'; }
pre.src-haskell:before { content: 'Haskell'; }
pre.src-hledger:before { content: 'hledger'; }
pre.src-java:before { content: 'Java'; }
pre.src-js:before { content: 'Javascript'; }
pre.src-latex:before { content: 'LaTeX'; }
pre.src-ledger:before { content: 'Ledger'; }
pre.src-lisp:before { content: 'Lisp'; }
pre.src-lilypond:before { content: 'Lilypond'; }
pre.src-lua:before { content: 'Lua'; }
pre.src-matlab:before { content: 'MATLAB'; }
pre.src-mscgen:before { content: 'Mscgen'; }
pre.src-ocaml:before { content: 'Objective Caml'; }
pre.src-octave:before { content: 'Octave'; }
pre.src-org:before { content: 'Org mode'; }
pre.src-oz:before { content: 'OZ'; }
pre.src-plantuml:before { content: 'Plantuml'; }
pre.src-processing:before { content: 'Processing.js'; }
pre.src-python:before { content: 'Python'; }
pre.src-R:before { content: 'R'; }
pre.src-ruby:before { content: 'Ruby'; }
pre.src-sass:before { content: 'Sass'; }
pre.src-scheme:before { content: 'Scheme'; }
pre.src-screen:before { content: 'Gnu Screen'; }
pre.src-sed:before { content: 'Sed'; }
pre.src-sh:before { content: 'shell'; }
pre.src-sql:before { content: 'SQL'; }
pre.src-sqlite:before { content: 'SQLite'; }
/* additional languages in org.el's org-babel-load-languages alist */
pre.src-forth:before { content: 'Forth'; }
pre.src-io:before { content: 'IO'; }
pre.src-J:before { content: 'J'; }
pre.src-makefile:before { content: 'Makefile'; }
pre.src-maxima:before { content: 'Maxima'; }
pre.src-perl:before { content: 'Perl'; }
pre.src-picolisp:before { content: 'Pico Lisp'; }
pre.src-scala:before { content: 'Scala'; }
pre.src-shell:before { content: 'Shell Script'; }
pre.src-ebnf2ps:before { content: 'ebfn2ps'; }
/* additional language identifiers per "defun org-babel-execute"
in ob-*.el */
pre.src-cpp:before { content: 'C++'; }
pre.src-abc:before { content: 'ABC'; }
pre.src-coq:before { content: 'Coq'; }
pre.src-groovy:before { content: 'Groovy'; }
/* additional language identifiers from org-babel-shell-names in
ob-shell.el: ob-shell is the only babel language using a lambda to put
the execution function name together. */
pre.src-bash:before { content: 'bash'; }
pre.src-csh:before { content: 'csh'; }
pre.src-ash:before { content: 'ash'; }
pre.src-dash:before { content: 'dash'; }
pre.src-ksh:before { content: 'ksh'; }
pre.src-mksh:before { content: 'mksh'; }
pre.src-posh:before { content: 'posh'; }
/* Additional Emacs modes also supported by the LaTeX listings package */
pre.src-ada:before { content: 'Ada'; }
pre.src-asm:before { content: 'Assembler'; }
pre.src-caml:before { content: 'Caml'; }
pre.src-delphi:before { content: 'Delphi'; }
pre.src-html:before { content: 'HTML'; }
pre.src-idl:before { content: 'IDL'; }
pre.src-mercury:before { content: 'Mercury'; }
pre.src-metapost:before { content: 'MetaPost'; }
pre.src-modula-2:before { content: 'Modula-2'; }
pre.src-pascal:before { content: 'Pascal'; }
pre.src-ps:before { content: 'PostScript'; }
pre.src-prolog:before { content: 'Prolog'; }
pre.src-simula:before { content: 'Simula'; }
pre.src-tcl:before { content: 'tcl'; }
pre.src-tex:before { content: 'TeX'; }
pre.src-plain-tex:before { content: 'Plain TeX'; }
pre.src-verilog:before { content: 'Verilog'; }
pre.src-vhdl:before { content: 'VHDL'; }
pre.src-xml:before { content: 'XML'; }
pre.src-nxml:before { content: 'XML'; }
/* add a generic configuration mode; LaTeX export needs an additional
(add-to-list 'org-latex-listings-langs '(conf " ")) in .emacs */
pre.src-conf:before { content: 'Configuration File'; }
table { border-collapse:collapse; }
caption.t-above { caption-side: top; }
caption.t-bottom { caption-side: bottom; }
td, th { vertical-align:top; }
th.org-right { text-align: center; }
th.org-left { text-align: center; }
th.org-center { text-align: center; }
td.org-right { text-align: right; }
td.org-left { text-align: left; }
td.org-center { text-align: center; }
dt { font-weight: bold; }
.footpara { display: inline; }
.footdef { margin-bottom: 1em; }
.figure { padding: 1em; }
.figure p { text-align: center; }
.equation-container {
display: table;
text-align: center;
width: 100%;
}
.equation {
vertical-align: middle;
}
.equation-label {
display: table-cell;
text-align: right;
vertical-align: middle;
}
.inlinetask {
padding: 10px;
border: 2px solid gray;
margin: 10px;
background: #ffffcc;
}
#org-div-home-and-up
{ text-align: right; font-size: 70%; white-space: nowrap; }
textarea { overflow-x: auto; }
.linenr { font-size: smaller }
.code-highlighted { background-color: #ffff00; }
.org-info-js_info-navigation { border-style: none; }
#org-info-js_console-label
{ font-size: 10px; font-weight: bold; white-space: nowrap; }
.org-info-js_search-highlight
{ background-color: #ffff00; color: #000000; font-weight: bold; }
.org-svg { width: 90%; }
/*]]>*/-->
</style>
<link rel="stylesheet" href="https://latex.now.sh/style.css">
<script type="text/javascript">
/*
@licstart The following is the entire license notice for the
JavaScript code in this tag.
Copyright (C) 2012-2019 Free Software Foundation, Inc.
The JavaScript code in this tag is free software: you can
redistribute it and/or modify it under the terms of the GNU
General Public License (GNU GPL) as published by the Free Software
Foundation, either version 3 of the License, or (at your option)
any later version. The code is distributed WITHOUT ANY WARRANTY;
without even the implied warranty of MERCHANTABILITY or FITNESS
FOR A PARTICULAR PURPOSE. See the GNU GPL for more details.
As additional permission under GNU GPL version 3 section 7, you
may distribute non-source (e.g., minimized or compacted) forms of
that code without the copy of the GNU GPL normally required by
section 4, provided you include this license notice and a URL
through which recipients can access the Corresponding Source.
@licend The above is the entire license notice
for the JavaScript code in this tag.
*/
<!--/*--><![CDATA[/*><!--*/
function CodeHighlightOn(elem, id)
{
var target = document.getElementById(id);
if(null != target) {
elem.cacheClassElem = elem.className;
elem.cacheClassTarget = target.className;
target.className = "code-highlighted";
elem.className = "code-highlighted";
}
}
function CodeHighlightOff(elem, id)
{
var target = document.getElementById(id);
if(elem.cacheClassElem)
elem.className = elem.cacheClassElem;
if(elem.cacheClassTarget)
target.className = elem.cacheClassTarget;
}
/*]]>*///-->
</script>
</head>
<body>
<div id="content">
<h1 class="title">grpc-go 源代码阅读笔记</h1>
<div id="table-of-contents">
<h2>Table of Contents</h2>
<div id="text-table-of-contents">
<ul>
<li><a href="#org551e90a">1. 为什么读?</a></li>
<li><a href="#org4c1c51a">2. 如何读?</a></li>
<li><a href="#org043581e">3. 阅读笔记</a>
<ul>
<li><a href="#org8d66da5">3.1. 补充的基础知识</a>
<ul>
<li><a href="#orgcc03a42">3.1.1. http/2</a></li>
<li><a href="#orgbce30ec">3.1.2. grpc 如何利用 http/2</a></li>
</ul>
</li>
<li><a href="#org72c3c12">3.2. attributes</a></li>
<li><a href="#orga485092">3.3. backoff</a></li>
<li><a href="#org214c326">3.4. balancer</a></li>
<li><a href="#orgdb2c1e7">3.5. benchmark</a></li>
<li><a href="#org4806d57">3.6. binlog</a></li>
<li><a href="#org77f0752">3.7. channelz</a></li>
<li><a href="#org8dd36d3">3.8. cmd</a></li>
<li><a href="#org8856af6">3.9. codes</a></li>
<li><a href="#org24d3a58">3.10. connectivity</a></li>
<li><a href="#org0f872e1">3.11. credentials</a></li>
<li><a href="#org9063d6f">3.12. encoding</a></li>
<li><a href="#orgda997c7">3.13. examples</a>
<ul>
<li><a href="#org9d3b898">3.13.1. auth</a></li>
<li><a href="#orgddaf806">3.13.2. cancelation</a></li>
<li><a href="#orge8a38e4">3.13.3. compression</a></li>
<li><a href="#org4182172">3.13.4. deadline</a></li>
</ul>
</li>
<li><a href="#orgd97abd8">3.14. Serve</a></li>
<li><a href="#orge1bd248">3.15. RegisterService</a></li>
<li><a href="#org44c5952">3.16. channelz</a></li>
<li><a href="#org541402b">3.17. Invoke</a></li>
</ul>
</li>
</ul>
</div>
</div>
<div id="outline-container-org551e90a" class="outline-2">
<h2 id="org551e90a"><span class="section-number-2">1</span> 为什么读?</h2>
<div class="outline-text-2" id="text-1">
<p>
因为无聊。
</p>
</div>
</div>
<div id="outline-container-org4c1c51a" class="outline-2">
<h2 id="org4c1c51a"><span class="section-number-2">2</span> 如何读?</h2>
<div class="outline-text-2" id="text-2">
<p>
主要是看框架做了什么,没有做什么,从而推断「为什么这样做」,如果难以推断,可以跟别的框架比较,我选择了 <a href="https://github.com/gin-gonic/gin">gin</a> (虽然它是一个 http 框架)。
</p>
</div>
</div>
<div id="outline-container-org043581e" class="outline-2">
<h2 id="org043581e"><span class="section-number-2">3</span> 阅读笔记</h2>
<div class="outline-text-2" id="text-3">
</div>
<div id="outline-container-org8d66da5" class="outline-3">
<h3 id="org8d66da5"><span class="section-number-3">3.1</span> 补充的基础知识</h3>
<div class="outline-text-3" id="text-3-1">
</div>
<div id="outline-container-orgcc03a42" class="outline-4">
<h4 id="orgcc03a42"><span class="section-number-4">3.1.1</span> http/2</h4>
<div class="outline-text-4" id="text-3-1-1">
<p>
http/2 通过 frames 传递信息。总的来说就是把 request 化整为零了,这样多个 request 并发的时候,小的 request 不会被大的 request 阻塞(几个 request 按大小分成不同数量的 frame 然后轮换发送 frame,这样小请求会先发完)。
</p>
<p>
流控制为每个 stream 留下配额,防止某个 steam 发太多把 tcp buffer 打爆。
</p>
<p>
还有更智能的 proxy,总的来说就是新增的 request 不需要新增对应数量的连接,并且 http/2 保证使用 stream 更均匀地分布在 connection 中。
</p>
</div>
</div>
<div id="outline-container-orgbce30ec" class="outline-4">
<h4 id="orgbce30ec"><span class="section-number-4">3.1.2</span> grpc 如何利用 http/2</h4>
<div class="outline-text-4" id="text-3-1-2">
<p>
引入的新概念:
</p>
<ol class="org-ol">
<li>channel</li>
<li>remote procedure call(rpc)</li>
<li>message</li>
</ol>
<p>
每个 channel 有多个 rpc,每个 rpc 有多个 message。
</p>
<p>
这三个概念跟 http/2 的关系:
</p>
<ol class="org-ol">
<li>channel 是虚拟的 connection,一个 channel 背后可能有许多个 http/2 connections。</li>
<li>RPC 其实对应 http/2 stream。</li>
<li>message 是 http/2 frame 之上的概念,可能多个 message 构成一个 http/2 frame,可能一个 message 分割成多个 http/2 frame.</li>
</ol>
<p>
gRPC 连接管理的原则:如果 connection 失败了,那么 lb 会尝试用已知的地址中重连,resolver 会重新解析地址,更新可用地址列表。
</p>
<p>
失败 connection 的鉴别,失败分两种,一种是通过 FIN handshake 的 clean failure, 这种情况下, http/2 和 gRPC 都不用干什么。
</p>
<p>
如果是 endpoint 挂掉或者阻塞,导致不断重试的情况,gRPC 不会收到 FIN,所以,如果配置了 KeepAlive,gRPC 通过周期性发送 http/2 PING frame 判断 endpoint 的存活性(确保了 connection 的健康)。
</p>
</div>
</div>
</div>
<div id="outline-container-org72c3c12" class="outline-3">
<h3 id="org72c3c12"><span class="section-number-3">3.2</span> attributes</h3>
<div class="outline-text-3" id="text-3-2">
<p>
就是一个 key value 存储,没啥特别的,都不知道为什么要搞这个东西。
</p>
</div>
</div>
<div id="outline-container-orga485092" class="outline-3">
<h3 id="orga485092"><span class="section-number-3">3.3</span> backoff</h3>
<div class="outline-text-3" id="text-3-3">
<p>
定义了重试的方案,见<a href="https://github.com/grpc/grpc/blob/master/doc/connection-backoff.md">这个文档</a>。
</p>
<p>
backoff 其实用来生成重试的 deadline (这个时间也是下一轮重试的时间)。
</p>
<p>
没一轮的 backoff 会乘一个因子(直到到达 MAX_BACKOFF),下一轮的 deadline 会在 [(1 - JITTER) * current_backoff, (1 + JITTER) * current_backoff] 之间生成(当然也有一个上限)。
</p>
<p>
收到 SETTINGS frames 的时候,重设 backoff.
</p>
<p>
然而代码只是定义了一些变量而已。
</p>
</div>
</div>
<div id="outline-container-org214c326" class="outline-3">
<h3 id="org214c326"><span class="section-number-3">3.4</span> balancer</h3>
<div class="outline-text-3" id="text-3-4">
<p>
实验性质的内容,所以暂时跳过了。其实跟上面 http/2 那两篇 blog 有更紧密的联系。
</p>
</div>
</div>
<div id="outline-container-orgdb2c1e7" class="outline-3">
<h3 id="orgdb2c1e7"><span class="section-number-3">3.5</span> benchmark</h3>
<div class="outline-text-3" id="text-3-5">
<p>
看不出这是外部用的还是内部用的。
</p>
</div>
</div>
<div id="outline-container-org4806d57" class="outline-3">
<h3 id="org4806d57"><span class="section-number-3">3.6</span> binlog</h3>
<div class="outline-text-3" id="text-3-6">
<p>
这是一个 pb,但是不知道生成的 proto,暂时不知道用处。
</p>
</div>
</div>
<div id="outline-container-org77f0752" class="outline-3">
<h3 id="org77f0752"><span class="section-number-3">3.7</span> channelz</h3>
<div class="outline-text-3" id="text-3-7">
<p>
这是一个收集统计数据的功能,但是好好奇只有 get 的接口。
</p>
</div>
</div>
<div id="outline-container-org8dd36d3" class="outline-3">
<h3 id="org8dd36d3"><span class="section-number-3">3.8</span> cmd</h3>
<div class="outline-text-3" id="text-3-8">
<p>
跳过。
</p>
</div>
</div>
<div id="outline-container-org8856af6" class="outline-3">
<h3 id="org8856af6"><span class="section-number-3">3.9</span> codes</h3>
<div class="outline-text-3" id="text-3-9">
<p>
定义了各种错误码。
</p>
</div>
</div>
<div id="outline-container-org24d3a58" class="outline-3">
<h3 id="org24d3a58"><span class="section-number-3">3.10</span> connectivity</h3>
<div class="outline-text-3" id="text-3-10">
<p>
定义了一组关于连接状态的常量和 Reporter 接口(谁在调?)。
</p>
</div>
</div>
<div id="outline-container-org0f872e1" class="outline-3">
<h3 id="org0f872e1"><span class="section-number-3">3.11</span> credentials</h3>
<div class="outline-text-3" id="text-3-11">
<p>
鉴权不想看。
</p>
</div>
</div>
<div id="outline-container-org9063d6f" class="outline-3">
<h3 id="org9063d6f"><span class="section-number-3">3.12</span> encoding</h3>
<div class="outline-text-3" id="text-3-12">
<p>
跳过。
</p>
</div>
</div>
<div id="outline-container-orgda997c7" class="outline-3">
<h3 id="orgda997c7"><span class="section-number-3">3.13</span> examples</h3>
<div class="outline-text-3" id="text-3-13">
</div>
<div id="outline-container-org9d3b898" class="outline-4">
<h4 id="org9d3b898"><span class="section-number-4">3.13.1</span> auth</h4>
</div>
<div id="outline-container-orgddaf806" class="outline-4">
<h4 id="orgddaf806"><span class="section-number-4">3.13.2</span> cancelation</h4>
<div class="outline-text-4" id="text-3-13-2">
<p>
主要学到了 context.WithTimeout (顺带一提,它其实是通过 WithDeadline 来实现的) 的用法——它会返回一个 Context 和一个 Cancel 函数,调用该 Cancel 函数,所有与该 Context 相关的资源都应该被释放。
</p>
<p>
顺着这里,我去看了 context 的文档。但我依然不明白为什么要这样设计 context。
</p>
<p>
而且有了一个新问题:为什么在 client 端 cancel,可以影响到 server 端?我觉得是很秒的设计。这个问题将我引向 Invoke。
</p>
</div>
</div>
<div id="outline-container-orge8a38e4" class="outline-4">
<h4 id="orge8a38e4"><span class="section-number-4">3.13.3</span> compression</h4>
<div class="outline-text-4" id="text-3-13-3">
<p>
这里的奇妙之处在于只需要 client 端处理,但是 server 端是不需要加特殊的设置的,一切尽在 options 中。所以重要的其实在 invoke 函数中。
</p>
</div>
</div>
<div id="outline-container-org4182172" class="outline-4">
<h4 id="org4182172"><span class="section-number-4">3.13.4</span> deadline</h4>
<div class="outline-text-4" id="text-3-13-4">
<p>
这个其实就是 <code>context.WithTimeout</code> 的教学关。
</p>
</div>
</div>
</div>
<div id="outline-container-orgd97abd8" class="outline-3">
<h3 id="orgd97abd8"><span class="section-number-3">3.14</span> Serve</h3>
</div>
<div id="outline-container-orge1bd248" class="outline-3">
<h3 id="orge1bd248"><span class="section-number-3">3.15</span> RegisterService</h3>
<div class="outline-text-3" id="text-3-15">
<p>
没啥出奇的,就是 reflect 的操作比较娴熟。
</p>
</div>
</div>
<div id="outline-container-org44c5952" class="outline-3">
<h3 id="org44c5952"><span class="section-number-3">3.16</span> channelz</h3>
<div class="outline-text-3" id="text-3-16">
<p>
看<a href="https://grpc.io/blog/a-short-introduction-to-channelz/">这里</a>。
</p>
</div>
</div>
<div id="outline-container-org541402b" class="outline-3">
<h3 id="org541402b"><span class="section-number-3">3.17</span> Invoke</h3>
<div class="outline-text-3" id="text-3-17">
<p>
<code>Invoke</code> 实际上负责调用函数。
</p>
<div class="org-src-container">
<label class="org-src-name"><span class="listing-number">Listing 1: </span>Invoke 的函数签名</label><pre class="src src-go">func (cc *ClientConn) Invoke(ctx context.Context, method string, args, reply interface{}, opts ...CallOption) error
</pre>
</div>
<p>
这里有一个函数:
</p>
<div class="org-src-container">
<pre class="src src-go">func combine(o1 []CallOption, o2 []CallOption) []CallOption {
// we don't use append because o1 could have extra capacity whose
// elements would be overwritten, which could cause inadvertent
// sharing (and race conditions) between concurrent calls
if len(o1) == 0 {
return o2
} else if len(o2) == 0 {
return o1
}
ret := make([]CallOption, len(o1)+len(o2))
copy(ret, o1)
copy(ret[len(o1):], o2)
return ret
}
</pre>
</div>
<p>
值得研究一下在并发情况下会有什么问题。
</p>
</div>
</div>
</div>
</div>
<div id="postamble" class="status">
<p class="author">Author: dante zhang</p>
<p class="date">Created: 2020-06-27 六 13:10</p>
<p class="validation"><a href="http://validator.w3.org/check?uri=referer">Validate</a></p>
</div>
</body>
</html>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment