Skip to content

Instantly share code, notes, and snippets.

View civitaspo's full-sized avatar
❄️

Takahiro Nakayama civitaspo

❄️
31 followers · 12 following
View GitHub Profile
@civitaspo
civitaspo / magic_comment_frozen_string_literal_after_encoding.log
Created July 10, 2017 04:48
if `frozen_string_literal: true` is before `encoding: cp932`, `encoding: cp932` doesn't work.
#<Encoding:Windows-31J>
@civitaspo
civitaspo / log
Created July 7, 2017 12:56
$ ruby test.rb
UTF-8
test_b.rb:8:in `include?': incompatible character encodings: UTF-8 and Windows-31J (Encoding::CompatibilityError)
from test_b.rb:8:in `block in <main>'
from test_b.rb:5:in `open'
from test_b.rb:5:in `<main>'
@civitaspo
civitaspo / embulk.log
Last active May 26, 2017 09:59
https://medjed-staging.analytics.mbga.local/app/project/210/taskset/485/task/10890?result_id=600012608 のログ
This file has been truncated, but you can view the full file.
2017-05-26 18:37:42.191 +0900 [DEBUG] (0062:task-0018): embulk-input-hdfs: createDecompressedInputStream: hdfs://main/data/current/pf/promo/launch_tracking/2016-03-14/m/merged-m-00018.gz
2017-05-26 18:37:42.239 +0900 [DEBUG] (0062:task-0018): embulk-input-hdfs: CompressionCodec: org.apache.hadoop.io.compress.GzipCodec@41fb0731: hdfs://main/data/current/pf/promo/launch_tracking/2016-03-14/m/merged-m-00018.gz
2017-05-26 18:37:42.254 +0900 [DEBUG] (0062:task-0018): BlockReaderFactory(fileName=/data/current/pf/promo/launch_tracking/2016-03-14/m/merged-m-00018.gz, block=BP-1490112602-10.54.10.205-1412742243656:blk_3118474030_1104363250399): PathInfo{path=, state=UNUSABLE} is not usable for short circuit; giving up on BlockReaderLocal.
2017-05-26 18:37:42.270 +0900 [DEBUG] (0062:task-0018): SASL client skipping handshake in unsecured configuration for addr = /10.54.10.149, datanodeId = 10.54.10.149:50010
2017-05-26 18:37:42.367 +0900 [WARN] (0062:task-0018): doFlush:start:buffer: org.embulk.exec.PooledBufferAllocat
@civitaspo
civitaspo / exception_exp.rb
Created May 23, 2017 00:53
require 'java'
class Error < Java::JavaLang::RuntimeException; end
class Error1 < Error; end
class Error2 < Error; end
class Error3 < Error; end
def err1
raise Error1.new("err")
end
@civitaspo
civitaspo / newGemPush.gradle
Last active May 18, 2017 10:20
modified after `embulk new java-filter test`
plugins {
id "com.jfrog.bintray" version "1.1"
id "com.github.jruby-gradle.base" version "0.1.5"
id "java"
id "checkstyle"
}
import com.github.jrubygradle.JRubyExec
repositories {
mavenCentral()
jcenter()
@civitaspo
civitaspo / optimist_README.md
Last active February 9, 2017 14:42

optimist

optimist is a simple option parser for golang which is inspired by minimist, Getopt::Casual.

Usage

The below code returns optimist.Args that is the result of parsing arguments and structuring them.




  


  




    


          
    

      

        
        

          

              @civitaspo
          

          

            
                civitaspo
                / .rubocop.yml
            
            

              Created
              October 18, 2016 23:50
            

          

        

      

        

  

    
    


        



  


  

        

          
          # This configuration was generated by
        

        

          
          # `rubocop --auto-gen-config`
        

        

          
          # on 2016-01-24 14:28:14 +0900 using RuboCop version 0.36.0.
        

        

          
          # The point is for the user to remove these configuration records
        

        

          
          # one by one as the offenses are removed from the code base.
        

        

          
          # Note that changes in the inspected code, or installation of new
        

        

          
          # versions of RuboCop, may require this file to be generated again.
        

        

          
          

        

        

          
          AllCops:
        

        

          
            Exclude:
        

  





    


  




    


          
    

      

        
        

          

              @civitaspo
          

          

            
                civitaspo
                / rescue_different_error.rb
            
            

              Last active
              July 27, 2016 09:34
            

          

        

      

        

  

    
    


        



  


  

        

          
          require 'embulk'
        

        

          
          

        

        

          
          Embulk.setup
        

        

          
          

        

        

          
          class MyError1 < ::Embulk::DataError; end
        

        

          
          class MyError2 < ::Embulk::DataError; end
        

        

          
          

        

        

          
          begin
        

        

          
            raise MyError1.new("hoge")
        

        

          
          rescue MyError2 => e
        

  





    


  




    


          
    

      

        
        

          

              @civitaspo
          

          

            
                civitaspo
                / config.yml
            
            

              Last active
              March 1, 2016 03:56
            

              
                embulk-input-command is not built for java7?
              
          

        

      

        

  

    
    


        



  


  

        

          
          [embulk-test] cat config.yml
        

        

          
          in:
        

        

          
            type: command
        

        

          
            command: 'cat /Users/takahiro.nakayama/work/embulk-test/data/data*'
        

        

          
            parser:
        

        

          
              type: csv
        

        

          
              columns:
        

        

          
                - {name: t, type: string}
        

        

          
          out:
        

        

          
            type: file
        

  





    


  




    


          
    

      

        
        

          

              @civitaspo
          

          

            
                civitaspo
                / s3_to_file.yml.liquid
            
            

              Created
              February 4, 2016 01:38
            

              
                embulkでデータ欠損が発生した際の全ログとconfig
              
          

        

      

        

  

    
    


        



  


  

        

          
          in:
        

        

          
            type: s3
        

        

          
            access_key_id: "{{ env.AWS_ACCESS_KEY_ID }}"
        

        

          
            secret_access_key: "{{ env.AWS_SECRET_ACCESS_KEY }}"
        

        

          
            bucket: "{{ env.S3_BUCKET }}"
        

        

          
            path_prefix: "{{ env.INPUT_PREFIX }}"
        

        

          
            decoders:
        

        

          
              - type: gzip
        

        

          
            parser:
        

        

          
              type: csv