Skip to content

Instantly share code, notes, and snippets.

@yakisova

yakisova/2018-01-28_work_mtail.md

Last active February 2, 2018 09:11
Show Gist options
  • Save yakisova/c56cdd4b098bddd0eec918e3b420f22f to your computer and use it in GitHub Desktop.
Save yakisova/c56cdd4b098bddd0eec918e3b420f22f to your computer and use it in GitHub Desktop.
Download ZIP
2018-01-28_work_mtail
Raw
2018-01-28_work_mtail.md

公式

https://github.com/google/mtail/blob/master/docs/Building.md

環境

centos7 Ec2: t2.micro

やったことめも

Go1.7.以上をインストールする

GOPATHの設定をする https://github.com/golang/go/wiki/SettingGOPATH

rootになって、gopathの設定をする

sudo su -
vi .bash_profile
#末尾に export GOPATH=$HOME/go を追加
source ~/.bash_profile
mkdir -p $GOPATH/src

mtailをクローンしようとして

cd $GOPATH/src
go get github.com/google/mtail

goコマンドがないとおこられる

-bash: go: command not found

goをインストールする

yum install go
[root@ip-10-0-0-190 go]# yum install go
Loaded plugins: fastestmirror
base                               | 3.6 kB     00:00
extras                             | 3.4 kB     00:00
updates                            | 3.4 kB     00:00
Loading mirror speeds from cached hostfile
 * base: ftp.iij.ad.jp
 * extras: ftp.iij.ad.jp
 * updates: ftp.iij.ad.jp
Resolving Dependencies
--> Running transaction check
---> Package golang.x86_64 0:1.8.3-1.el7 will be installed
--> Processing Dependency: golang-src = 1.8.3-1.el7 for package: golang-1.8.3-1.el7.x86_64
--> Processing Dependency: golang-bin = 1.8.3-1.el7 for package: golang-1.8.3-1.el7.x86_64
--> Running transaction check
---> Package golang-bin.x86_64 0:1.8.3-1.el7 will be installed
--> Processing Dependency: gcc for package: golang-bin-1.8.3-1.el7.x86_64
---> Package golang-src.noarch 0:1.8.3-1.el7 will be installed
--> Running transaction check
---> Package gcc.x86_64 0:4.8.5-16.el7_4.1 will be installed
--> Processing Dependency: cpp = 4.8.5-16.el7_4.1 for package: gcc-4.8.5-16.el7_4.1.x86_64
--> Processing Dependency: glibc-devel >= 2.2.90-12 for package: gcc-4.8.5-16.el7_4.1.x86_64
--> Processing Dependency: libmpfr.so.4()(64bit) for package: gcc-4.8.5-16.el7_4.1.x86_64
--> Processing Dependency: libmpc.so.3()(64bit) for package: gcc-4.8.5-16.el7_4.1.x86_64
--> Running transaction check
---> Package cpp.x86_64 0:4.8.5-16.el7_4.1 will be installed
---> Package glibc-devel.x86_64 0:2.17-196.el7_4.2 will be installed
--> Processing Dependency: glibc-headers = 2.17-196.el7_4.2 for package: glibc-devel-2.17-196.el7_4.2.x86_64
--> Processing Dependency: glibc-headers for package: glibc-devel-2.17-196.el7_4.2.x86_64
---> Package libmpc.x86_64 0:1.0.1-3.el7 will be installed
---> Package mpfr.x86_64 0:3.1.1-4.el7 will be installed
--> Running transaction check
---> Package glibc-headers.x86_64 0:2.17-196.el7_4.2 will be installed
--> Processing Dependency: kernel-headers >= 2.2.1 for package: glibc-headers-2.17-196.el7_4.2.x86_64
--> Processing Dependency: kernel-headers for package: glibc-headers-2.17-196.el7_4.2.x86_64
--> Running transaction check
---> Package kernel-headers.x86_64 0:3.10.0-693.17.1.el7 will be installed
--> Finished Dependency Resolution

Dependencies Resolved

==========================================================
 Package        Arch   Version              Repository
                                                     Size
==========================================================
Installing:
 golang         x86_64 1.8.3-1.el7          base    1.2 M
Installing for dependencies:
 cpp            x86_64 4.8.5-16.el7_4.1     updates 5.9 M
 gcc            x86_64 4.8.5-16.el7_4.1     updates  16 M
 glibc-devel    x86_64 2.17-196.el7_4.2     updates 1.1 M
 glibc-headers  x86_64 2.17-196.el7_4.2     updates 676 k
 golang-bin     x86_64 1.8.3-1.el7          base     47 M
 golang-src     noarch 1.8.3-1.el7          base    4.8 M
 kernel-headers x86_64 3.10.0-693.17.1.el7  updates 6.0 M
 libmpc         x86_64 1.0.1-3.el7          base     51 k
 mpfr           x86_64 3.1.1-4.el7          base    203 k

Transaction Summary
==========================================================
Install  1 Package (+9 Dependent packages)

Total download size: 83 M
Installed size: 295 M
Is this ok [y/d/N]: y
Downloading packages:
(1/10): cpp-4.8.5-16.el7_4.1.x86_64. | 5.9 MB   00:00
(2/10): glibc-headers-2.17-196.el7_4 | 676 kB   00:00
(3/10): glibc-devel-2.17-196.el7_4.2 | 1.1 MB   00:00
(4/10): gcc-4.8.5-16.el7_4.1.x86_64. |  16 MB   00:00
(5/10): golang-1.8.3-1.el7.x86_64.rp | 1.2 MB   00:00
(6/10): golang-src-1.8.3-1.el7.noarc | 4.8 MB   00:00
(7/10): libmpc-1.0.1-3.el7.x86_64.rp |  51 kB   00:00
(8/10): mpfr-3.1.1-4.el7.x86_64.rpm  | 203 kB   00:00
(9/10): golang-bin-1.8.3-1.el7.x86_6 |  47 MB   00:01
(10/10): kernel-headers-3.10.0-693.1 | 6.0 MB   00:00
----------------------------------------------------------
Total                         49 MB/s |  83 MB  00:01
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
  Installing : mpfr-3.1.1-4.el7.x86_64               1/10
  Installing : libmpc-1.0.1-3.el7.x86_64             2/10
  Installing : cpp-4.8.5-16.el7_4.1.x86_64           3/10
  Installing : kernel-headers-3.10.0-693.17.1.el7    4/10
  Installing : glibc-headers-2.17-196.el7_4.2.x86    5/10
  Installing : glibc-devel-2.17-196.el7_4.2.x86_6    6/10
  Installing : gcc-4.8.5-16.el7_4.1.x86_64           7/10
  Installing : golang-src-1.8.3-1.el7.noarch         8/10
  Installing : golang-1.8.3-1.el7.x86_64             9/10
  Installing : golang-bin-1.8.3-1.el7.x86_64        10/10
  Verifying  : cpp-4.8.5-16.el7_4.1.x86_64           1/10
  Verifying  : glibc-devel-2.17-196.el7_4.2.x86_6    2/10
  Verifying  : golang-1.8.3-1.el7.x86_64             3/10
  Verifying  : libmpc-1.0.1-3.el7.x86_64             4/10
  Verifying  : mpfr-3.1.1-4.el7.x86_64               5/10
  Verifying  : golang-src-1.8.3-1.el7.noarch         6/10
  Verifying  : kernel-headers-3.10.0-693.17.1.el7    7/10
  Verifying  : glibc-headers-2.17-196.el7_4.2.x86    8/10
  Verifying  : golang-bin-1.8.3-1.el7.x86_64         9/10
  Verifying  : gcc-4.8.5-16.el7_4.1.x86_64          10/10

Installed:
  golang.x86_64 0:1.8.3-1.el7

Dependency Installed:
  cpp.x86_64 0:4.8.5-16.el7_4.1
  gcc.x86_64 0:4.8.5-16.el7_4.1
  glibc-devel.x86_64 0:2.17-196.el7_4.2
  glibc-headers.x86_64 0:2.17-196.el7_4.2
  golang-bin.x86_64 0:1.8.3-1.el7
  golang-src.noarch 0:1.8.3-1.el7
  kernel-headers.x86_64 0:3.10.0-693.17.1.el7
  libmpc.x86_64 0:1.0.1-3.el7
  mpfr.x86_64 0:3.1.1-4.el7

Complete!
[root@ip-10-0-0-190 go]#

もういちどmtailをクローンするところから gitがないとおこられる

go: missing Git command. See https://golang.org/s/gogetcmd
package github.com/google/mtail: exec: "git": executable file not found in $PATH

gitをインストールする

[root@ip-10-0-0-190 go]# yum install git
Loaded plugins: fastestmirror
Loading mirror speeds from cached hostfile
 * base: ftp.iij.ad.jp
 * extras: ftp.iij.ad.jp
 * updates: ftp.iij.ad.jp
Resolving Dependencies
--> Running transaction check
---> Package git.x86_64 0:1.8.3.1-12.el7_4 will be installed
--> Processing Dependency: perl-Git = 1.8.3.1-12.el7_4 for package: git-1.8.3.1-12.el7_4.x86_64
--> Processing Dependency: perl(Term::ReadKey) for package: git-1.8.3.1-12.el7_4.x86_64
--> Processing Dependency: perl(Git) for package: git-1.8.3.1-12.el7_4.x86_64
--> Processing Dependency: perl(Error) for package: git-1.8.3.1-12.el7_4.x86_64
--> Processing Dependency: libgnome-keyring.so.0()(64bit) for package: git-1.8.3.1-12.el7_4.x86_64
--> Running transaction check
---> Package libgnome-keyring.x86_64 0:3.12.0-1.el7 will be installed
---> Package perl-Error.noarch 1:0.17020-2.el7 will be installed
---> Package perl-Git.noarch 0:1.8.3.1-12.el7_4 will be installed
---> Package perl-TermReadKey.x86_64 0:2.30-20.el7 will be installed
--> Finished Dependency Resolution

Dependencies Resolved

==========================================================
 Package          Arch   Version            Repository
                                                     Size
==========================================================
Installing:
 git              x86_64 1.8.3.1-12.el7_4   updates 4.4 M
Installing for dependencies:
 libgnome-keyring x86_64 3.12.0-1.el7       base    109 k
 perl-Error       noarch 1:0.17020-2.el7    base     32 k
 perl-Git         noarch 1.8.3.1-12.el7_4   updates  53 k
 perl-TermReadKey x86_64 2.30-20.el7        base     31 k

Transaction Summary
==========================================================
Install  1 Package (+4 Dependent packages)

Total download size: 4.6 M
Installed size: 23 M
Is this ok [y/d/N]: y
Downloading packages:
(1/5): perl-Git-1.8.3.1-12.el7_4.noa |  53 kB   00:00
(2/5): libgnome-keyring-3.12.0-1.el7 | 109 kB   00:00
(3/5): perl-Error-0.17020-2.el7.noar |  32 kB   00:00
(4/5): perl-TermReadKey-2.30-20.el7. |  31 kB   00:00
(5/5): git-1.8.3.1-12.el7_4.x86_64.r | 4.4 MB   00:00
----------------------------------------------------------
Total                         16 MB/s | 4.6 MB  00:00
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
  Installing : 1:perl-Error-0.17020-2.el7.noarch      1/5
  Installing : perl-TermReadKey-2.30-20.el7.x86_64    2/5
  Installing : libgnome-keyring-3.12.0-1.el7.x86_64   3/5
  Installing : perl-Git-1.8.3.1-12.el7_4.noarch       4/5
  Installing : git-1.8.3.1-12.el7_4.x86_64            5/5
  Verifying  : git-1.8.3.1-12.el7_4.x86_64            1/5
  Verifying  : 1:perl-Error-0.17020-2.el7.noarch      2/5
  Verifying  : libgnome-keyring-3.12.0-1.el7.x86_64   3/5
  Verifying  : perl-Git-1.8.3.1-12.el7_4.noarch       4/5
  Verifying  : perl-TermReadKey-2.30-20.el7.x86_64    5/5

Installed:
  git.x86_64 0:1.8.3.1-12.el7_4

Dependency Installed:
  libgnome-keyring.x86_64 0:3.12.0-1.el7
  perl-Error.noarch 1:0.17020-2.el7
  perl-Git.noarch 0:1.8.3.1-12.el7_4
  perl-TermReadKey.x86_64 0:2.30-20.el7

Complete!
[root@ip-10-0-0-190 go]#

もう一回クローンする

go get github.com/google/mtail
こんなメッセージが出てきた
# github.com/google/mtail/vm
src/github.com/google/mtail/vm/codegen.go:261: undefined: PLUS
src/github.com/google/mtail/vm/codegen.go:264: undefined: MINUS
src/github.com/google/mtail/vm/codegen.go:266: undefined: MUL
src/github.com/google/mtail/vm/codegen.go:268: undefined: DIV
src/github.com/google/mtail/vm/codegen.go:270: undefined: MOD
src/github.com/google/mtail/vm/codegen.go:272: undefined: POW
src/github.com/google/mtail/vm/codegen.go:274: undefined: ASSIGN
src/github.com/google/mtail/vm/driver.go:47: undefined: mtailSymType
src/github.com/google/mtail/vm/lexer.go:22: undefined: INVALID
src/github.com/google/mtail/vm/lexer.go:23: undefined: LCURLY
src/github.com/google/mtail/vm/lexer.go:23: too many errors

メッセージ出てきたけどクローンで着ているのですすめる

cd $GOPATH/src
go get github.com/google/mtail
cd github.com/google/mtail
make
go get golang.org/x/tools/cmd/goyacc
touch .gen-dep-stamp
go generate -x ./vm
/root/go/bin/goyacc -v y.output -o parser.go -p mtail parser.y
# Install all dependencies, ensuring they're updated
go get -u -v bufio bytes crypto/rand encoding/json expvar flag fmt github.com/fsnotify/fsnotify github.com/golang/glog github.com/pkg/errors github.com/spf13/afero html/template io io/ioutil math math/big math/rand net net/http net/http/pprof os os/signal path path/filepath regexp regexp/syntax runtime runtime/debug sort strconv strings sync sync/atomic syscall text/tabwriter time unicode unicode/utf8
github.com/fsnotify/fsnotify (download)
Fetching https://golang.org/x/sys/unix?go-get=1
Parsing meta tags from https://golang.org/x/sys/unix?go-get=1 (status code 200)
get "golang.org/x/sys/unix": found meta tag main.metaImport{Prefix:"golang.org/x/sys", VCS:"git", RepoRoot:"https://go.googlesource.com/sys"} at https://golang.org/x/sys/unix?go-get=1
get "golang.org/x/sys/unix": verifying non-authoritative meta tag
Fetching https://golang.org/x/sys?go-get=1
Parsing meta tags from https://golang.org/x/sys?go-get=1 (status code 200)
golang.org/x/sys (download)
github.com/golang/glog (download)
github.com/pkg/errors (download)
github.com/spf13/afero (download)
Fetching https://golang.org/x/text/transform?go-get=1
Parsing meta tags from https://golang.org/x/text/transform?go-get=1 (status code 200)
get "golang.org/x/text/transform": found meta tag main.metaImport{Prefix:"golang.org/x/text", VCS:"git", RepoRoot:"https://go.googlesource.com/text"} at https://golang.org/x/text/transform?go-get=1
get "golang.org/x/text/transform": verifying non-authoritative meta tag
Fetching https://golang.org/x/text?go-get=1
Parsing meta tags from https://golang.org/x/text?go-get=1 (status code 200)
golang.org/x/text (download)
Fetching https://golang.org/x/text/unicode/norm?go-get=1
Parsing meta tags from https://golang.org/x/text/unicode/norm?go-get=1 (status code 200)
get "golang.org/x/text/unicode/norm": found meta tag main.metaImport{Prefix:"golang.org/x/text", VCS:"git", RepoRoot:"https://go.googlesource.com/text"} at https://golang.org/x/text/unicode/norm?go-get=1
get "golang.org/x/text/unicode/norm": verifying non-authoritative meta tag
go get -u -v bytes encoding/json errors expvar flag fmt github.com/golang/glog github.com/google/go-cmp/cmp github.com/google/go-cmp/cmp/cmpopts github.com/spf13/afero io io/ioutil math/rand net/http net/http/httptest os os/user path path/filepath reflect regexp regexp/syntax runtime sort strconv strings sync syscall testing testing/quick time
github.com/golang/glog (download)
github.com/google/go-cmp (download)
github.com/spf13/afero (download)
Fetching https://golang.org/x/text/transform?go-get=1
Parsing meta tags from https://golang.org/x/text/transform?go-get=1 (status code 200)
get "golang.org/x/text/transform": found meta tag main.metaImport{Prefix:"golang.org/x/text", VCS:"git", RepoRoot:"https://go.googlesource.com/text"} at https://golang.org/x/text/transform?go-get=1
get "golang.org/x/text/transform": verifying non-authoritative meta tag
Fetching https://golang.org/x/text?go-get=1
Parsing meta tags from https://golang.org/x/text?go-get=1 (status code 200)
golang.org/x/text (download)
Fetching https://golang.org/x/text/unicode/norm?go-get=1
Parsing meta tags from https://golang.org/x/text/unicode/norm?go-get=1 (status code 200)
get "golang.org/x/text/unicode/norm": found meta tag main.metaImport{Prefix:"golang.org/x/text", VCS:"git", RepoRoot:"https://go.googlesource.com/text"} at https://golang.org/x/text/unicode/norm?go-get=1
get "golang.org/x/text/unicode/norm": verifying non-authoritative meta tag
github.com/google/go-cmp/cmp/internal/diff
github.com/google/go-cmp/cmp/internal/function
github.com/google/go-cmp/cmp/internal/value
github.com/google/go-cmp/cmp
github.com/google/go-cmp/cmp/cmpopts
touch .dep-stamp
go install -ldflags "-X main.Version=v3.0.0-rc5-48-g6e17f80 -X main.Revision=6e17f8086ba73fe59c17e2156417bffdc4015d13"

プログラミングガイドを読む https://github.com/google/mtail/blob/master/docs/Programming-Guide.md なにをどうかけがいいのかわからないので、とばす

出プロイングを読む https://github.com/google/mtail/blob/master/docs/Deploying.md

コンフィグの概要 ヘルプ コマンドのパスが通っていないので、プロファイルを修正

# .bash_profile

# Get the aliases and functions
if [ -f ~/.bashrc ]; then
        . ~/.bashrc
fi

# User specific environment and startup programs

export GOPATH=$HOME/go
PATH=$PATH:$HOME/bin:$GOPATH/bin

export PATH

cd
source .bash_profile

とりあえず実行してみる

mtail --logs /var/log/syslog
F0202 06:58:49.250055    1079 main.go:118] No mtail program directory specified; use -progs

プログラムファイルがないぞと起こられたので、 mtailようプログラムファイルの作成

mkdir /etc/mtail

vim test.mtail
# simple line counter
counter line_count
/$/ {
  line_count++
}

mtail -prog /etc/mtail/test.mtail -logs /var/log/messages & プログラムファイルは動的反映される

プロメテウスサーバを起動 監視対象を追加 mtailのエンドポイントを追加した static_configs: - targets: ['localhost:9090'] - targets: ['localhost:3903']

プ

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment