Skip to content

Instantly share code, notes, and snippets.

View fs111's full-sized avatar

André Kelpe fs111

View GitHub Profile
$ gradle test
The TaskContainer.add() method has been deprecated and is scheduled to be removed in Gradle 2.0. Please use the create() method instead.
Using Apache Hadoop Stable [1.2.1]
:compileJava UP-TO-DATE
:processResources UP-TO-DATE
:classes UP-TO-DATE
:compileTestJava UP-TO-DATE
:processTestResources UP-TO-DATE
:testClasses UP-TO-DATE
:test
public class Main
{
public static void main( String[] args )
{
String outPath = args[ 0 ];
Properties properties = new Properties();
AppProps.setApplicationJarClass( properties, Main.class );
HadoopFlowConnector flowConnector = new HadoopFlowConnector( properties );
$ hadoop jar build/libs/es-cascading-tryout.jar foo
14/02/28 13:32:41 INFO util.HadoopUtil: resolving application jar from found main method on: impatient.Main
14/02/28 13:32:41 INFO planner.HadoopPlanner: using application jar: /home/fs111/code/concurrent/es-test/build/libs/es-cascading-tryout.jar
14/02/28 13:32:41 INFO property.AppProps: using app.id: BB3B7C8730DC470D8E6ED817EFFC2E5B
14/02/28 13:32:41 INFO util.Version: Concurrent, Inc - Cascading 2.5.2
14/02/28 13:32:41 INFO flow.Flow: [] starting
14/02/28 13:32:41 INFO flow.Flow: [] source: EsHadoopTap["EsHadoopScheme[[UNKNOWN]->[ALL]]"]["employees/titles"]
14/02/28 13:32:41 INFO flow.Flow: [] sink: Hfs["TextDelimited[[UNKNOWN]->[ALL]]"]["foo"]
14/02/28 13:32:41 INFO flow.Flow: [] parallel execution is enabled: false
14/02/28 13:32:41 INFO flow.Flow: [] starting jobs: 1
public class Main
{
public static void main( String[] args )
{
String outPath = args[ 0 ];
Map<Object, Object> properties = new HashMap<Object, Object>();
properties.put("es.http.timeout", "10m");
properties.put("es.scroll.size", "10");
AppProps.setApplicationJarClass( properties, Main.class );
$ curl 'http://elastic.local:9200/foo/bar/_search?pretty=true&q=*:*'
{
"took" : 1,
"timed_out" : false,
"_shards" : {
"total" : 5,
"successful" : 5,
"failed" : 0
},
"hits" : {
14/02/28 14:54:31 INFO planner.HadoopPlanner: using application jar: /home/fs111/code/concurrent/es-test/build/libs/es-cascading-tryout.jar
14/02/28 14:54:31 INFO property.AppProps: using app.id: FD14E98AD3D446AF9912A2DBAAE9F14E
14/02/28 14:54:32 DEBUG cascading.EsTap: Detected Hadoop environment; initializing [EsHadoopTap]
14/02/28 14:54:32 TRACE cascading.EsTap: Registered Cascading serialization token 140 for org.elasticsearch.hadoop.mr.LinkedMapWritable
14/02/28 14:54:32 TRACE cascading.EsHadoopScheme: Initialized configuration {fs.s3n.impl=org.apache.hadoop.fs.s3native.NativeS3FileSystem, mapreduce.job.counters.max=120, mapred.task.cache.levels=2, mapreduce.job.restart.recover=true, map.sort.class=org.apache.hadoop.util.QuickSort, hadoop.tmp.dir=/tmp/hadoop-${user.name}, hadoop.native.lib=true, ipc.client.idlethreshold=4000, mapred.system.dir=${hadoop.tmp.dir}/mapred/system, mapred.job.tracker.persist.jobstatus.hours=0, io.skip.checksum.errors=false, fs.default.name=file:///, mapred.cluster.reduce.memory
lingual catalog --provider --add cascading:cascading-jdbc-mysql:2.5.1:provider
:: loading settings :: url = jar:file:/home/fs111/code/vagrant-cascading-hadoop-cluster/single-node/lingual-client/lib/ivy-2.3.0.jar!/org/apache/ivy/core/settings/ivysettings.xml
using ivy parser to parse file:/tmp/ivy4532929233272986986.xml
:: resolving dependencies :: cascading#cascading-jdbc-mysql-caller;working [not transitive]
confs: [default]
validate = true
refresh = false
resolving dependencies for configuration 'default'
== resolving dependencies for cascading#cascading-jdbc-mysql-caller;working [default]
== resolving dependencies cascading#cascading-jdbc-mysql-caller;working->cascading#cascading-jdbc-mysql;2.5.1 [default->*]
vagrant ssh master --command 'sudo rm -rf /srv/hadoop'
TupleTypeableInstances$$anon$8$$anonfun$cast$7$$anonfun$apply$16$$anonfun$apply$17$$anonfun$apply$18$$anonfun$apply$19$$anonfun$apply$20$$anonfun$apply$21.class
libreoffice-core:
Installiert: 1:4.3.1~rc2-0ubuntu1~precise1
Kandidat: 1:4.3.1~rc2-0ubuntu1~precise1
Versionstabelle:
*** 1:4.3.1~rc2-0ubuntu1~precise1 0
500 http://ppa.launchpad.net/libreoffice/libreoffice-4-3/ubuntu/ precise/main amd64 Packages
100 /var/lib/dpkg/status
1:3.5.7-0ubuntu6.1 0
500 http://archive.ubuntu.com/ubuntu/ precise-updates/main amd64 Packages
1:3.5.4-0ubuntu1.1 0