Skip to content

Instantly share code, notes, and snippets.

@masuwo3
Last active March 29, 2017 19:08
  • Star 2 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
Star You must be signed in to star a gist
Save masuwo3/46cddc8f07c7d6938d3e to your computer and use it in GitHub Desktop.
AML Tutorial with aws-sdk
require 'aws-sdk'
require 'securerandom'
ml = Aws::MachineLearning::Client.new(region: 'us-east-1')
hash_value = SecureRandom.hex(8)
# 訓練用データセットの設定
ds_train = {
data_source_id: 'ds-train-' + hash_value,
data_source_name: 'ds-train-banking',
data_spec: {
data_location_s3: "s3://aml-sample-data/banking.csv",
data_schema_location_s3: "s3://aml-sample-data/banking.csv.schema",
data_rearrangement: "{\"randomSeed\":\"#{hash_value}\", \"splitting\":{\"percentBegin\":0,\"percentEnd\":70}}"
},
compute_statistics: true
}
# 評価用データセットの設定
ds_eval = {
data_source_id: 'ds-eval-' + hash_value,
data_source_name: 'ds-eval-banking',
data_spec: {
data_location_s3: "s3://aml-sample-data/banking.csv",
data_schema_location_s3: "s3://aml-sample-data/banking.csv.schema",
data_rearrangement: "{\"randomSeed\":\"#{hash_value}\", \"splitting\":{\"percentBegin\":70,\"percentEnd\":100}}"
},
compute_statistics: true
}
# バッチ予測用データセットの設定
ds_bp = {
data_source_id: 'ds-bp-' + hash_value,
data_source_name: 'ds-bp-banking',
data_spec: {
data_location_s3: "s3://aml-sample-data/banking-batch.csv",
data_schema_location_s3: "s3://aml-sample-data/banking.csv.schema"
},
compute_statistics: true
}
# モデルの設定
model = {
ml_model_id: 'model-' + hash_value,
ml_model_name: 'model-banking',
ml_model_type: 'BINARY',
training_data_source_id: ds_train[:data_source_id]
}
# 評価の設定
eval = {
evaluation_id: 'eval-' + hash_value,
evaluation_name: 'eval-banking',
ml_model_id: model[:ml_model_id], # required
evaluation_data_source_id: ds_eval[:data_source_id]
}
# バッチ予測の設定
bp = {
batch_prediction_id: 'bp-' + hash_value,
batch_prediction_name: "bp-banking",
ml_model_id: model[:ml_model_id],
batch_prediction_data_source_id: ds_bp[:data_source_id],
output_uri: "s3://cm-aml-test/result",
}
# データセット作成
ml.create_data_source_from_s3(ds_train)
ml.create_data_source_from_s3(ds_eval)
ml.create_data_source_from_s3(ds_bp)
print 'DataSource creating'
loop do
status = ml.get_data_source(data_source_id: ds_train[:data_source_id]).status
case status
when 'COMPLETED' then
puts status
break
when 'FAILED' then
raise 'datasource creation failed'
else
print '.'
sleep 10;
end
end
# モデル構築
ml.create_ml_model(model)
print 'Model Creating'
loop do
status = ml.get_ml_model(ml_model_id: model[:ml_model_id]).status
case status
when 'COMPLETED' then
puts status
break
when 'FAILED' then
raise 'model creation failed'
else
print '.'
sleep 10;
end
end
# 評価作成
ml.create_evaluation(eval)
print 'Evaluation creating'
loop do
status = ml.get_evaluation(evaluation_id: eval[:evaluation_id]).status
case status
when 'COMPLETED' then
puts status
break
when 'FAILED' then
raise 'evaluation creation failed'
else
print '.'
sleep 10;
end
end
# バッチ予測
ml.create_batch_prediction(bp)
print 'Batch-Prediction creating'
loop do
status = ml.get_batch_prediction(batch_prediction_id: bp[:batch_prediction_id]).status
case status
when 'COMPLETED' then
puts status
break
when 'FAILED' then
raise 'batch-prediction creation failed'
else
print '.'
sleep 10;
end
end
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment