Created
July 31, 2016 11:16
-
-
Save yukiyan/3c5dd521bcfe0be74ec629297c4e3d5e to your computer and use it in GitHub Desktop.
use typecast
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
[ | |
{ | |
"name": "boolean_value1", | |
"type": "BOOLEAN" | |
}, | |
{ | |
"name": "boolean_value2", | |
"type": "BOOLEAN" | |
} | |
] |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
version: '2' | |
services: | |
fluentd: | |
build: . | |
container_name: fluentd | |
volumes: | |
- ./:/fluentd/etc/ |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
FROM fluent/fluentd:v0.12.26-onbuild | |
USER root | |
RUN apk --update add build-base ruby-dev | |
USER fluent | |
RUN gem install \ | |
fluent-plugin-bigquery \ | |
fluent-plugin-typecast \ | |
fluent-plugin-parser &&\ | |
fluent-gem install bigdecimal |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
<source> | |
@type dummy | |
dummy {"message":"boolean_value1:true\tboolean_value2:false"} | |
rate 1 | |
tag bigquery | |
</source> | |
<filter bigquery> | |
@type parser | |
format ltsv | |
key_name message | |
</filter> | |
<match bigquery> | |
@type typecast | |
item_types boolean_value1:bool,boolean_value2:bool | |
prefix typed | |
</match> | |
<match typed.bigquery> | |
@type bigquery | |
method insert | |
auth_method private_key | |
email xxxxxxxxxxxx-xxxxxxxxxxxxxxxxxxxxxx@developer.gserviceaccount.com | |
private_key_path 00000000000000000000000000000000-privatekey.p12 | |
project yukiyan-development | |
dataset sample | |
table sample | |
auto_create_table true | |
schema_path /fluentd/etc/bq_schema.json | |
</match> |
Author
yukiyan
commented
Jul 31, 2016
apk --update add build-base ruby-dev
とか fluent-gem install bigdecimal
はコンテナでfluent-plugin-bigquery
動かす上で必要なやつなのであまり気にしなくて良い。
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment