Skip to content

Instantly share code, notes, and snippets.

@hiroyuki-sato
Created December 10, 2015 13:23
Show Gist options
  • Star 1 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save hiroyuki-sato/5ae82c1ddcb8d84bd07c to your computer and use it in GitHub Desktop.
Save hiroyuki-sato/5ae82c1ddcb8d84bd07c to your computer and use it in GitHub Desktop.
Embulk cookie sample

sample.txt

id_field,key1=aaa; key2=bbb; key3=ccc

config.yml

in:
  type: file
  path_prefix: sample.txt
  parser:
    type: csv
    columns: 
      - { name: id_field, type: string }
      - { name: cookie, type: string }
filters:
  - type: split
    delimiter: '; '
    keep_input: true 
    target_key: cookie
    output_key: cookie_val
out:
  type: stdout

embulk run config.yml

+-----------------+------------------------------+-------------------+
| id_field:string |                cookie:string | cookie_val:string |
+-----------------+------------------------------+-------------------+
|        id_field | key1=aaa; key2=bbb; key3=ccc |          key1=aaa |
|        id_field | key1=aaa; key2=bbb; key3=ccc |          key2=bbb |
|        id_field | key1=aaa; key2=bbb; key3=ccc |          key3=ccc |
+-----------------+------------------------------+-------------------+
@yoshikaw
Copy link

ESに格納できる形式を考えずに、以下にようにEmbulkで加工したのがインポートできればいいなと思っていました。

+-----------------+------------------------------+
| id_field:string |                cookie:string |
+-----------------+------------------------------+
|        id_field | key1=aaa; key2=bbb; key3=ccc |
+-----------------+------------------------------+

↓ embulk-filter-column で複製

+-----------------+------------------------------+------------------------------+------------------------------+------------------------------+
| id_field:string |                cookie:string |           cookie.key1:string |           cookie.key2:string |           cookie.key3:string |
+-----------------+------------------------------+------------------------------+------------------------------+------------------------------+
|        id_field | key1=aaa; key2=bbb; key3=ccc | key1=aaa; key2=bbb; key3=ccc | key1=aaa; key2=bbb; key3=ccc | key1=aaa; key2=bbb; key3=ccc |
+-----------------+------------------------------+------------------------------+------------------------------+------------------------------+

↓ embulk-filter-eval で必要な値を絞り込み

+-----------------+------------------------------+--------------------+--------------------+--------------------+
| id_field:string |                cookie:string | cookie.key1:string | cookie.key2:string | cookie.key3:string |
+-----------------+------------------------------+--------------------+--------------------+--------------------+
|        id_field | key1=aaa; key2=bbb; key3=ccc |                aaa |                bbb |                ccc |
+-----------------+------------------------------+--------------------+--------------------+--------------------+

↓ embulk-filter-column で不要な列を削除

+-----------------+--------------------+--------------------+--------------------+
| id_field:string | cookie.key1:string | cookie.key2:string | cookie.key3:string |
+-----------------+--------------------+--------------------+--------------------+
|        id_field |                aaa |                bbb |                ccc |
+-----------------+--------------------+--------------------+--------------------+

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment