Created
June 14, 2011 15:10
-
-
Save marblejenka/1025089 to your computer and use it in GitHub Desktop.
ぶたっちべーす
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| cloudera@cloudera-vm:~$ pig | |
| 2011-06-02 07:26:54,093 [main] INFO org.apache.pig.Main - Logging error messages to: /home/cloudera/pig_1307024814091.log | |
| 2011-06-02 07:26:54,559 [main] INFO org.apache.pig.backend.hadoop.executionengine.HExecutionEngine - Connecting to hadoop file system at: hdfs://localhost:8020 | |
| 2011-06-02 07:26:54,885 [main] INFO org.apache.pig.backend.hadoop.executionengine.HExecutionEngine - Connecting to map-reduce job tracker at: localhost:8021 | |
| grunt> result = load 'test' using org.apache.pig.backend.hadoop.hbase.HBaseStorage('family:a family:b') as (a:chararray, b:chararray); | |
| grunt> dump result | |
| 2011-06-02 07:27:37,041 [main] INFO org.apache.pig.tools.pigstats.ScriptState - Pig features used in the script: UNKNOWN | |
| 2011-06-02 07:27:37,042 [main] INFO org.apache.pig.backend.hadoop.executionengine.HExecutionEngine - pig.usenewlogicalplan is set to true. New logical plan will be used. | |
| 2011-06-02 07:27:37,318 [main] INFO org.apache.pig.backend.hadoop.executionengine.HExecutionEngine - (Name: result: Store(hdfs://localhost/tmp/temp831782254/tmp1008601629:org.apache.pig.impl.io.InterStorage) - scope-8 Operator Key: scope-8) | |
| 2011-06-02 07:27:37,338 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MRCompiler - File concatenation threshold: 100 optimistic? false | |
| 2011-06-02 07:27:37,378 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MultiQueryOptimizer - MR plan size before optimization: 1 | |
| 2011-06-02 07:27:37,378 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MultiQueryOptimizer - MR plan size after optimization: 1 | |
| 2011-06-02 07:27:37,468 [main] INFO org.apache.pig.tools.pigstats.ScriptState - Pig script settings are added to the job | |
| 2011-06-02 07:27:37,478 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler - mapred.job.reduce.markreset.buffer.percent is not set, set to default 0.3 | |
| 2011-06-02 07:27:37,528 [main] INFO org.apache.zookeeper.ZooKeeper - Client environment:zookeeper.version=3.3.3-1073969, built on 02/23/2011 22:27 GMT | |
| 2011-06-02 07:27:37,528 [main] INFO org.apache.zookeeper.ZooKeeper - Client environment:host.name=cloudera-vm | |
| 2011-06-02 07:27:37,528 [main] INFO org.apache.zookeeper.ZooKeeper - Client environment:java.version=1.6.0_24 | |
| 2011-06-02 07:27:37,528 [main] INFO org.apache.zookeeper.ZooKeeper - Client environment:java.vendor=Sun Microsystems Inc. | |
| 2011-06-02 07:27:37,528 [main] INFO org.apache.zookeeper.ZooKeeper - Client environment:java.home=/usr/lib/jvm/java-6-sun-1.6.0.24/jre | |
| 2011-06-02 07:27:37,528 [main] INFO org.apache.zookeeper.ZooKeeper - Client environment:java.class.path=/etc/hbase/conf:/usr/lib/pig/bin/../conf:/usr/lib/jvm/java-6-sun/lib/tools.jar:/usr/lib/pig/bin/../pig-0.8.0-cdh3u0-core.jar:/usr/lib/pig/bin/../build/pig-*-SNAPSHOT.jar:/usr/lib/pig/bin/../lib/ant-contrib-1.0b3.jar:/usr/lib/pig/bin/../lib/automaton.jar:/usr/lib/pig/bin/../build/ivy/lib/Pig/*.jar:/usr/lib/hadoop/hadoop-core-0.20.2-cdh3u0.jar:/usr/lib/hadoop/lib/ant-contrib-1.0b3.jar:/usr/lib/hadoop/lib/aspectjrt-1.6.5.jar:/usr/lib/hadoop/lib/aspectjtools-1.6.5.jar:/usr/lib/hadoop/lib/commons-cli-1.2.jar:/usr/lib/hadoop/lib/commons-codec-1.4.jar:/usr/lib/hadoop/lib/commons-daemon-1.0.1.jar:/usr/lib/hadoop/lib/commons-el-1.0.jar:/usr/lib/hadoop/lib/commons-httpclient-3.0.1.jar:/usr/lib/hadoop/lib/commons-logging-1.0.4.jar:/usr/lib/hadoop/lib/commons-logging-api-1.0.4.jar:/usr/lib/hadoop/lib/commons-net-1.4.1.jar:/usr/lib/hadoop/lib/core-3.1.1.jar:/usr/lib/hadoop/lib/hadoop-fairscheduler-0.20.2-cdh3u0.jar:/usr/lib/hadoop/lib/hsqldb-1.8.0.10.jar:/usr/lib/hadoop/lib/hue-plugins-1.2.0.jar:/usr/lib/hadoop/lib/jackson-core-asl-1.5.2.jar:/usr/lib/hadoop/lib/jackson-mapper-asl-1.5.2.jar:/usr/lib/hadoop/lib/jasper-compiler-5.5.12.jar:/usr/lib/hadoop/lib/jasper-runtime-5.5.12.jar:/usr/lib/hadoop/lib/jdiff:/usr/lib/hadoop/lib/jets3t-0.6.1.jar:/usr/lib/hadoop/lib/jetty-6.1.26.jar:/usr/lib/hadoop/lib/jetty-servlet-tester-6.1.26.jar:/usr/lib/hadoop/lib/jetty-util-6.1.26.jar:/usr/lib/hadoop/lib/jsch-0.1.42.jar:/usr/lib/hadoop/lib/jsp-2.1:/usr/lib/hadoop/lib/junit-4.5.jar:/usr/lib/hadoop/lib/kfs-0.2.2.jar:/usr/lib/hadoop/lib/log4j-1.2.15.jar:/usr/lib/hadoop/lib/mockito-all-1.8.2.jar:/usr/lib/hadoop/lib/native:/usr/lib/hadoop/lib/oro-2.0.8.jar:/usr/lib/hadoop/lib/servlet-api-2.5-20081211.jar:/usr/lib/hadoop/lib/servlet-api-2.5-6.1.14.jar:/usr/lib/hadoop/lib/slf4j-api-1.4.3.jar:/usr/lib/hadoop/lib/slf4j-log4j12-1.4.3.jar:/usr/lib/hadoop/lib/xmlenc-0.52.jar:/usr/lib/hadoop/conf | |
| 2011-06-02 07:27:37,528 [main] INFO org.apache.zookeeper.ZooKeeper - Client environment:java.library.path=/usr/lib/jvm/java-6-sun-1.6.0.24/jre/lib/i386/client:/usr/lib/jvm/java-6-sun-1.6.0.24/jre/lib/i386:/usr/lib/jvm/java-6-sun-1.6.0.24/jre/../lib/i386:/usr/java/packages/lib/i386:/lib:/usr/lib | |
| 2011-06-02 07:27:37,528 [main] INFO org.apache.zookeeper.ZooKeeper - Client environment:java.io.tmpdir=/tmp | |
| 2011-06-02 07:27:37,528 [main] INFO org.apache.zookeeper.ZooKeeper - Client environment:java.compiler=<NA> | |
| 2011-06-02 07:27:37,528 [main] INFO org.apache.zookeeper.ZooKeeper - Client environment:os.name=Linux | |
| 2011-06-02 07:27:37,528 [main] INFO org.apache.zookeeper.ZooKeeper - Client environment:os.arch=i386 | |
| 2011-06-02 07:27:37,528 [main] INFO org.apache.zookeeper.ZooKeeper - Client environment:os.version=2.6.35-28-generic | |
| 2011-06-02 07:27:37,528 [main] INFO org.apache.zookeeper.ZooKeeper - Client environment:user.name=cloudera | |
| 2011-06-02 07:27:37,528 [main] INFO org.apache.zookeeper.ZooKeeper - Client environment:user.home=/home/cloudera | |
| 2011-06-02 07:27:37,529 [main] INFO org.apache.zookeeper.ZooKeeper - Client environment:user.dir=/home/cloudera | |
| 2011-06-02 07:27:37,531 [main] INFO org.apache.zookeeper.ZooKeeper - Initiating client connection, connectString=localhost:2181 sessionTimeout=180000 watcher=hconnection | |
| 2011-06-02 07:27:37,550 [main-SendThread()] INFO org.apache.zookeeper.ClientCnxn - Opening socket connection to server localhost/127.0.0.1:2181 | |
| 2011-06-02 07:27:37,560 [main-SendThread(localhost:2181)] INFO org.apache.zookeeper.ClientCnxn - Socket connection established to localhost/127.0.0.1:2181, initiating session | |
| 2011-06-02 07:27:37,574 [main-SendThread(localhost:2181)] INFO org.apache.zookeeper.ClientCnxn - Session establishment complete on server localhost/127.0.0.1:2181, sessionid = 0x13021f8249f0091, negotiated timeout = 40000 | |
| 2011-06-02 07:27:37,637 [main] DEBUG org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation - Lookedup root region location, connection=org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@1687e7c; hsa=cloudera-vm:59205 | |
| 2011-06-02 07:27:37,716 [main] DEBUG org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation - Cached location for .META.,,1.1028785192 is cloudera-vm:59205 | |
| 2011-06-02 07:27:37,724 [main] DEBUG org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation - Cache hit for row <> in tableName .META.: location server cloudera-vm:59205, location region name .META.,,1.1028785192 | |
| 2011-06-02 07:27:37,730 [main] DEBUG org.apache.hadoop.hbase.client.MetaScanner - Scanning .META. starting at row=test,,00000000000000 for max=10 rows | |
| 2011-06-02 07:27:37,735 [main] DEBUG org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation - Cached location for test,,1307024773728.5f37a8307fd4f5b1a35bce12bc72eab2. is cloudera-vm:59205 | |
| 2011-06-02 07:27:37,736 [main] DEBUG org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation - Cache hit for row <> in tableName test: location server cloudera-vm:59205, location region name test,,1307024773728.5f37a8307fd4f5b1a35bce12bc72eab2. | |
| 2011-06-02 07:27:41,516 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler - Setting up single store job | |
| 2011-06-02 07:27:41,571 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - 1 map-reduce job(s) waiting for submission. | |
| 2011-06-02 07:27:42,079 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - 0% complete | |
| 2011-06-02 07:27:42,268 [Thread-7] INFO org.apache.zookeeper.ZooKeeper - Initiating client connection, connectString=localhost:2181 sessionTimeout=180000 watcher=hconnection | |
| 2011-06-02 07:27:42,270 [Thread-7-SendThread()] INFO org.apache.zookeeper.ClientCnxn - Opening socket connection to server localhost/0:0:0:0:0:0:0:1:2181 | |
| 2011-06-02 07:27:42,271 [Thread-7-SendThread(localhost:2181)] INFO org.apache.zookeeper.ClientCnxn - Socket connection established to localhost/0:0:0:0:0:0:0:1:2181, initiating session | |
| 2011-06-02 07:27:42,274 [Thread-7-SendThread(localhost:2181)] INFO org.apache.zookeeper.ClientCnxn - Session establishment complete on server localhost/0:0:0:0:0:0:0:1:2181, sessionid = 0x13021f8249f0092, negotiated timeout = 40000 | |
| 2011-06-02 07:27:42,282 [Thread-7] DEBUG org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation - Lookedup root region location, connection=org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@2d59a3; hsa=cloudera-vm:59205 | |
| 2011-06-02 07:27:42,285 [Thread-7] DEBUG org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation - Cached location for .META.,,1.1028785192 is cloudera-vm:59205 | |
| 2011-06-02 07:27:42,285 [Thread-7] DEBUG org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation - Cache hit for row <> in tableName .META.: location server cloudera-vm:59205, location region name .META.,,1.1028785192 | |
| 2011-06-02 07:27:42,287 [Thread-7] DEBUG org.apache.hadoop.hbase.client.MetaScanner - Scanning .META. starting at row=test,,00000000000000 for max=10 rows | |
| 2011-06-02 07:27:42,289 [Thread-7] DEBUG org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation - Cached location for test,,1307024773728.5f37a8307fd4f5b1a35bce12bc72eab2. is cloudera-vm:59205 | |
| 2011-06-02 07:27:42,291 [Thread-7] DEBUG org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation - Cache hit for row <> in tableName test: location server cloudera-vm:59205, location region name test,,1307024773728.5f37a8307fd4f5b1a35bce12bc72eab2. | |
| 2011-06-02 07:27:42,320 [Thread-7] INFO org.apache.zookeeper.ZooKeeper - Initiating client connection, connectString=localhost:2181 sessionTimeout=180000 watcher=hconnection | |
| 2011-06-02 07:27:42,321 [Thread-7-SendThread()] INFO org.apache.zookeeper.ClientCnxn - Opening socket connection to server localhost/0:0:0:0:0:0:0:1:2181 | |
| 2011-06-02 07:27:42,330 [Thread-7-SendThread(localhost:2181)] INFO org.apache.zookeeper.ClientCnxn - Socket connection established to localhost/0:0:0:0:0:0:0:1:2181, initiating session | |
| 2011-06-02 07:27:42,333 [Thread-7-SendThread(localhost:2181)] INFO org.apache.zookeeper.ClientCnxn - Session establishment complete on server localhost/0:0:0:0:0:0:0:1:2181, sessionid = 0x13021f8249f0093, negotiated timeout = 40000 | |
| 2011-06-02 07:27:42,343 [Thread-7] DEBUG org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation - Lookedup root region location, connection=org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@1128f5a; hsa=cloudera-vm:59205 | |
| 2011-06-02 07:27:42,346 [Thread-7] DEBUG org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation - Cached location for .META.,,1.1028785192 is cloudera-vm:59205 | |
| 2011-06-02 07:27:42,346 [Thread-7] DEBUG org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation - Cache hit for row <> in tableName .META.: location server cloudera-vm:59205, location region name .META.,,1.1028785192 | |
| 2011-06-02 07:27:42,349 [Thread-7] DEBUG org.apache.hadoop.hbase.client.MetaScanner - Scanning .META. starting at row=test,,00000000000000 for max=10 rows | |
| 2011-06-02 07:27:42,353 [Thread-7] DEBUG org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation - Cached location for test,,1307024773728.5f37a8307fd4f5b1a35bce12bc72eab2. is cloudera-vm:59205 | |
| 2011-06-02 07:27:42,356 [Thread-7] DEBUG org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation - Cache hit for row <> in tableName test: location server cloudera-vm:59205, location region name test,,1307024773728.5f37a8307fd4f5b1a35bce12bc72eab2. | |
| 2011-06-02 07:27:42,358 [Thread-7] DEBUG org.apache.hadoop.hbase.client.MetaScanner - Scanning .META. starting at row=test,,00000000000000 for max=2147483647 rows | |
| 2011-06-02 07:27:42,366 [Thread-7] DEBUG org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation - Cache hit for row <> in tableName test: location server cloudera-vm:59205, location region name test,,1307024773728.5f37a8307fd4f5b1a35bce12bc72eab2. | |
| 2011-06-02 07:27:42,367 [Thread-7] DEBUG org.apache.hadoop.hbase.mapreduce.TableInputFormatBase - getSplits: split -> 0 -> cloudera-vm:, | |
| 2011-06-02 07:27:43,094 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - HadoopJobId: job_201105240525_0157 | |
| 2011-06-02 07:27:43,095 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - More information at: http://localhost:50030/jobdetails.jsp?jobid=job_201105240525_0157 | |
| 2011-06-02 07:27:48,661 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - 50% complete | |
| 2011-06-02 07:27:52,706 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - 100% complete | |
| 2011-06-02 07:27:52,708 [main] INFO org.apache.pig.tools.pigstats.PigStats - Script Statistics: | |
| HadoopVersion PigVersion UserId StartedAt FinishedAt Features | |
| 0.20.2-cdh3u0 0.8.0-cdh3u0 cloudera 2011-06-02 07:27:37 2011-06-02 07:27:52 UNKNOWN | |
| Success! | |
| Job Stats (time in seconds): | |
| JobId Maps Reduces MaxMapTime MinMapTIme AvgMapTime MaxReduceTime MinReduceTime AvgReduceTime Alias Feature Outputs | |
| job_201105240525_0157 1 0 3 3 3 0 0 0 result MAP_ONLY hdfs://localhost/tmp/temp831782254/tmp1008601629, | |
| Input(s): | |
| Successfully read 2 records (317 bytes) from: "test" | |
| Output(s): | |
| Successfully stored 2 records (46 bytes) in: "hdfs://localhost/tmp/temp831782254/tmp1008601629" | |
| Counters: | |
| Total records written : 2 | |
| Total bytes written : 46 | |
| Spillable Memory Manager spill count : 0 | |
| Total bags proactively spilled: 0 | |
| Total records proactively spilled: 0 | |
| Job DAG: | |
| job_201105240525_0157 | |
| 2011-06-02 07:27:52,716 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - Success! | |
| 2011-06-02 07:27:52,733 [main] INFO org.apache.hadoop.mapreduce.lib.input.FileInputFormat - Total input paths to process : 1 | |
| 2011-06-02 07:27:52,733 [main] INFO org.apache.pig.backend.hadoop.executionengine.util.MapRedUtil - Total input paths to process : 1 | |
| (valuea,valueb) | |
| (valuea,valueb) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| result = load 'test' using org.apache.pig.backend.hadoop.hbase.HBaseStorage('family:a family:b') as (a:chararray, b:chararray); | |
| dump result |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| cloudera@cloudera-vm:~$ hbase shell | |
| HBase Shell; enter 'help<RETURN>' for list of supported commands. | |
| Type "exit<RETURN>" to leave the HBase Shell | |
| Version 0.90.1-cdh3u0, r, Fri Mar 25 16:10:51 PDT 2011 | |
| hbase(main):001:0> create 'test', 'family' | |
| 0 row(s) in 0.7190 seconds | |
| hbase(main):002:0> | |
| hbase(main):003:0* put 'test', '1', 'family:a', 'valuea' | |
| 0 row(s) in 0.1180 seconds | |
| hbase(main):004:0> put 'test', '1', 'family:b', 'valueb' | |
| 0 row(s) in 0.0290 seconds | |
| hbase(main):005:0> put 'test', '2', 'family:a', 'valuea' | |
| 0 row(s) in 0.0300 seconds | |
| hbase(main):006:0> put 'test', '2', 'family:b', 'valueb' | |
| 0 row(s) in 0.0220 seconds | |
| hbase(main):007:0> | |
| hbase(main):008:0* scan 'test' | |
| ROW COLUMN+CELL | |
| 1 column=family:a, timestamp=1307024774090, value=valuea | |
| 1 column=family:b, timestamp=1307024774169, value=valueb | |
| 2 column=family:a, timestamp=1307024774246, value=valuea | |
| 2 column=family:b, timestamp=1307024774333, value=valueb | |
| 2 row(s) in 0.0890 seconds | |
| hbase(main):009:0> quit | |
| cloudera@cloudera-vm:~$ |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| create 'test', 'family' | |
| put 'test', '1', 'family:a', 'valuea' | |
| put 'test', '1', 'family:b', 'valueb' | |
| put 'test', '2', 'family:a', 'valuea' | |
| put 'test', '2', 'family:b', 'valueb' | |
| scan 'test' |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment