This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1002: Unable to store alias XYZ_active_users | |
at org.apache.pig.PigServer$Graph.registerQuery(PigServer.java:1552) | |
at org.apache.pig.PigServer.registerQuery(PigServer.java:540) | |
at org.apache.pig.tools.grunt.GruntParser.processPig(GruntParser.java:970) | |
at org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:386) | |
at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:189) | |
at org.apache.pig.PigServer.registerScript(PigServer.java:614) | |
at org.apache.pig.PigServer.registerScript(PigServer.java:716) | |
at org.apache.pig.PigServer.registerScript(PigServer.java:689) | |
at com.bsb.hike.analytics.pig.BaseHikePigTask.run(BaseHikePigTask.java:148) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
2012-11-26 18:46:14,525 WARN ActionStartXCommand:542 - USER[hduser] GROUP[-] TOKEN[] APP[map-reduce-wf] JOB[0000000-121126183401740-oozie-hdus-W] ACTION[0000000-121126183401740-oozie-hdus-W@mr-node] Error starting action [mr-node]. ErrorType [TRANSIENT], ErrorCode [JA009], Message [JA009: java.io.IOException: job_201211260754_0024(-1 memForMapTasks -1 memForReduceTasks): Invalid job requirements. | |
at org.apache.hadoop.mapred.JobTracker.checkMemoryRequirements(JobTracker.java:4992) | |
at org.apache.hadoop.mapred.JobTracker.submitJob(JobTracker.java:3752) | |
at org.apache.hadoop.mapred.JobTracker.submitJob(JobTracker.java:3695) | |
at sun.reflect.GeneratedMethodAccessor11.invoke(Unknown Source) | |
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) | |
at java.lang.reflect.Method.invoke(Method.java:601) | |
at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:578) | |
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1393) | |
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1389) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
#!/usr/bin/perl | |
use strict; | |
use warnings; | |
use File::Copy; | |
my $usage = <<'END'; | |
# Usage: perl $0 <input_logs_dir> <output_logs_dir> <suffix_regex?> | |
# Example: perl bucket_mqtt_logs.pl ./snapshots/ ./snapshots/ 'log\.((\d{4})-(\d{2})-(\d{2}))-(\d{2})' | |
END | |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Specify the configuration in a file and test the same via the "-t" option. | |
root:~/stud# ./stud -t --config=ss_default_config.conf --ssl | |
Trying to initialize SSL contexts with your certificates{core} | |
Note: no DH parameters found in /root/stud/saggar.in.includesprivatekey.pem | |
stud configuration looks ok. | |
STUD creates two separate process i.e. do a ps from a different screen and you will see 2 pids: | |
root:~/stud# ps -ef| grep stud |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
/* | |
* Pending - Covert _id to uid or vice versa | |
*/ | |
// Collection 01 - users | |
users_map = function() { | |
// Simply emit the msisdn and 0 for the file length. | |
// The file length will come from the other collection. | |
emit(this._id, { msisdn: this.msisdn, file_length: 0 }); | |
} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
hdfs@hadoop-prod-growthui:~$ cat /var/lib/hdfs/pig_1364556796933.log | |
Pig Stack Trace | |
--------------- | |
ERROR 1066: Unable to open iterator for alias log | |
org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1066: Unable to open iterator for alias log | |
at org.apache.pig.PigServer.openIterator(PigServer.java:836) | |
at org.apache.pig.tools.grunt.GruntParser.processDump(GruntParser.java:696) | |
at org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:320) | |
at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:194) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
redis 127.0.0.1:6379> hgetall linearregression | |
1) "1367373828" <<<< timestamp | |
2) "30860" <<<< visitors count | |
3) "1367473828" | |
4) "32860" | |
5) "1367273828" | |
6) "28860" | |
7) "1367073828" | |
8) "27060" | |
..... |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
curl -X GET http://0.0.0.0:6600/rpy/linearregression | |
{ | |
"stat": "pass", | |
"future": { | |
"c": -19581061.761599176, | |
"points": 4, | |
"m": 0.01434285714285654, | |
"predictions": { | |
"2013-05-05": 36307, | |
"2013-05-04": 35068, |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
hdfs@hadoop-prod-growthui:~$ hadoop distcp -i hdfs://hadoop-prod-master.vpc:8020/data/analytics/smsrecords hdfs://10.0.0.144:8020/data/analytics/smsrecords | |
13/06/07 07:18:19 INFO tools.DistCp: Input Options: DistCpOptions{atomicCommit=false, syncFolder=false, deleteMissing=false, ignoreFailures=true, maxMaps=20, sslConfigurationFile='null', copyStrategy='uniformsize', sourceFileListing=null, sourcePaths=[hdfs://hadoop-prod-master.vpc:8020/data/analytics/smsrecords], targetPath=hdfs://10.0.0.144:8020/data/analytics/smsrecords} | |
13/06/07 07:18:22 INFO service.AbstractService: Service:org.apache.hadoop.yarn.client.YarnClientImpl is inited. | |
13/06/07 07:18:23 INFO service.AbstractService: Service:org.apache.hadoop.yarn.client.YarnClientImpl is started. | |
13/06/07 07:18:26 ERROR tools.DistCp: Exception encountered | |
java.io.IOException: Failed on local exception: java.io.IOException: Broken pipe; Host Details : local host is: "hadoop-prod-growthui.vpc/10.0.0.230"; destination host is: "ip-10-0-0-144.ap-southeast-1.comp |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
''' | |
USAGE: ubuntu@mysql-ab1:~$ python mysqltail.py | |
''' | |
from pymysqlreplication import BinLogStreamReader | |
from pymysqlreplication.row_event import (DeleteRowsEvent, UpdateRowsEvent, WriteRowsEvent) | |
import types | |
import sys | |
from datetime import date | |
import redis | |
import time |