Skip to content

Instantly share code, notes, and snippets.

@trotter

trotter/master.log Secret

Created October 29, 2012 02:31
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save trotter/3e4547d73efc09a045f3 to your computer and use it in GitHub Desktop.
Save trotter/3e4547d73efc09a045f3 to your computer and use it in GitHub Desktop.
Download ZIP
druid batch ingestion issue
Raw
master.log
2012-10-29 11:18:11,908 DEBUG [Master-ZKYP--0] com.metamx.druid.client.ZKPhoneBook - New child[localhost], for peon[com.metamx.druid.client.InventoryManager$MasterPeon@4907dcfe]
2012-10-29 11:18:11,909 INFO [DatabaseSegmentManager-Exec--0] com.metamx.druid.db.DatabaseSegmentManager - Polled and found 2 segments in the database
2012-10-29 11:18:11,936 DEBUG [main] com.metamx.common.concurrent.ScheduledExecutors - Scheduling repeatedly: com.metamx.druid.master.DruidMaster$3@cb8d09d with delay PT60S
2012-10-29 11:18:11,937 DEBUG [main] com.metamx.common.concurrent.ScheduledExecutors - Scheduling repeatedly: com.metamx.druid.master.DruidMaster$3@7eaa2ef2 with delay PT1800S
2012-10-29 11:18:11,971 INFO [Master-ZKYP--0] com.metamx.druid.client.ServerInventoryManager./druid/servedSegmentsPath - New inventory container[localhost]!
2012-10-29 11:18:11,972 INFO [Master-ZKYP--0] com.metamx.druid.client.ServerInventoryManager./druid/servedSegmentsPath - Putting watch on [/druid/servedSegmentsPath/localhost]
2012-10-29 11:18:11,974 DEBUG [Master-ZKYP--0] com.metamx.druid.client.ZKPhoneBook - New child[_MASTER], for peon[com.metamx.druid.master.DruidMaster$MasterListeningPeon@56f052d5]
2012-10-29 11:18:11,979 INFO [Master-ZKYP--0] com.metamx.druid.master.DruidMaster - I really am the master!
Raw
raw data
2012-10-20T20:21:59Z billy california 3
2012-10-20T20:22:59Z ann new york 5
2012-10-20T20:23:59Z tom mississippi 2
2012-10-20T20:24:59Z jill arkansas 7
2012-10-20T20:25:59Z beaux california 8
2012-10-20T20:26:59Z jamie california 9
2012-10-20T20:27:59Z willy california 1
Raw
run_query.sh
curl -X POST http://localhost:8081/druid/v2/ -H 'content-type: application/json' -d @sample_query.body
Raw
sample_query.body
{
"queryType": "groupBy",
"dataSource": "trotter-test",
"granularity": "all",
"dimensions": ["state"],
"aggregations":[
{ "type": "count", "name": "event_count"},
{ "type": "doubleSum", "fieldName": "column_4", "name": "revenue" }
],
"intervals":["2010-10-01T00:00/2020-01-01T00"]
}
Raw
segment table
mysql> select * from test_segments;
+-------------------------------------------------------------------------------------------------------------+-------------------+-------------------------------+-------------------------------+-------------------------------+-------------+-------------------------------+------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| id | dataSource | created_date | start | end | partitioned | version | used | payload |
+-------------------------------------------------------------------------------------------------------------+-------------------+-------------------------------+-------------------------------+-------------------------------+-------------+-------------------------------+------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| trotter-test_2012-10-20T00:00:00.000-07:00_2012-10-21T00:00:00.000-07:00_2012-10-28T18:46:00.548-07:00 | trotter-test | 2012-10-28T18:47:02.255-07:00 | 2012-10-20T00:00:00.000-07:00 | 2012-10-21T00:00:00.000-07:00 | 0 | 2012-10-28T18:46:00.548-07:00 | 1 | {"dataSource":"trotter-test","interval":"2012-10-20T00:00:00.000/2012-10-21T00:00:00.000","version":"2012-10-28T18:46:00.548-07:00","loadSpec":{"type":"s3_zip","bucket":"trotter-test","key":"output/2012-10-20T00:00:00.000-07:00_2012-10-21T00:00:00.000-07:00/2012-10-28T18:46:00.548-07:00/0/index.zip"},"dimensions":"clicks,state","metrics":"event_count,revenue,clicks","shardSpec":{"type":"none"},"size":1717,"identifier":"trotter-test_2012-10-20T00:00:00.000-07:00_2012-10-21T00:00:00.000-07:00_2012-10-28T18:46:00.548-07:00"} |
+-------------------------------------------------------------------------------------------------------------+-------------------+-------------------------------+-------------------------------+-------------------------------+-------------+-------------------------------+------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment