myui

Hivemall v0.6.0 update

Hivemall version is updated to v0.6.0-201912-r03 based on Apache Hivemall v0.6.0.

Major new features in this release includes:

• xgboost (v0.90) support (find usage)
• Improvements on tokenize_ja
  • Part-of-Speech (PoS) support in tokenze_ja (find usage)
  • new stoptag_excludes UDF (find usage)

myui

Followed this document for release verification.

Install required softwares

brew install gpg gpg-agent pinentry-mac
brew install maven md5sha1sum

Import GPG KEYS for verification

myui

WITH data as (
  select 'jake' as name, 18 as age
  union all
  select 'tom' as name, 64 as age
  union all
  select 'lisa' as name, 32 as age
  union all
  select 'justin' as name, 43 as age
)

myui

Download dataset from https://www.kaggle.com/tomlisankie/blog-posts-labeled-with-age-and-gender/download

pip install kaggle

Please set your kaggle API credentials in ~/.kaggle/kaggle.json following the instruction: https://github.com/Kaggle/kaggle-api#api-credentials

myui

xgboost binary classification example

prepare data

prepare iris data https://support.treasuredata.com/hc/en-us/articles/360001260787-Iris-Multiclass-Classification-by-RandomForest

-- create xgboost input format (see https://xgboost.readthedocs.io/en/latest/tutorials/input_format.html)
INSERT OVERWRITE TABLE input

myui

survived,pclass,name,sex,age,sibsp,parch,ticket,fare,cabin,embarked
0,3,0,0,22,1,0,0,7.25,0,0
1,3,1,1,26,0,0,1,7.925,0,0
0,3,2,0,35,0,0,2,8.05,0,0
0,3,3,0,0,0,0,3,8.4583,0,1
0,3,4,0,2,3,1,4,21.075,0,0
1,3,5,1,27,0,2,5,11.1333,0,0
1,2,6,1,14,1,0,6,30.0708,0,2
0,3,7,0,20,0,0,7,8.05,0,0
0,3,8,0,39,1,5,8,31.275,0,0

myui

myui

WITH tmp1 as (
  select
      userid,
      array[ 
        probabilities[1],
        probabilities[2],
        probabilities[3],
        probabilities[4],
        probabilities[5],
        probabilities[6],

myui

CREATE TABLE users (
  rowid int, name string, age int, gender string
);
INSERT INTO users VALUES
  (1, 'Jacob', 20, 'Male'),
  (2, 'Mason', 22, 'Male'),
  (3, 'Sophia', 35, 'Female'),
  (4, 'Ethan', 55, 'Male'),
  (5, 'Emma', 15, 'Female'),
  (6, 'Noah', 46, 'Male'),

myui

set my_coment to "#" --コメント文字

tell application "mi"
	tell document 1
		set p_count to (count paragraphs of selection object 1)
		set start_p to index of paragraph 1 of selection object 1
		set end_p to start_p + p_count - 1
		
		repeat with i from start_p to end_p
			set tmp_str to quoted form of (paragraph i as Unicode text)