This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
#!/bin/bash | |
## http://sujee.net/tech/articles/amazon-emr-beyond-basics/ | |
# config | |
# if changing machine type, also change mapred config file | |
MASTER_INSTANCE_TYPE="m1.large" | |
SLAVE_INSTANCE_TYPE="c1.xlarge" | |
INSTANCES=5 | |
export JOBNAME="MyMR" | |
export TIMESTAMP=$(date +%Y%m%d-%H%M%S) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
/** | |
* http://sujee.net/tech/articles/amazon-emr-beyond-basics/ | |
* takes one argument for input path | |
*/ | |
public class TestMR extends Configured implements Tool | |
{ | |
public static void main(String[] args) throws Exception | |
{ | |
int res = ToolRunner.run(new Configuration(), new TestMR(), args); |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
#!/bin/bash | |
## http://sujee.net/tech/articles/amazon-emr-beyond-basics/ | |
CLASS_DIR=classes | |
mkdir -p $CLASS_DIR | |
rm -rf $CLASS_DIR/* | |
# compile |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
... | |
... | |
long t2 = System.currentTimeMillis(); | |
cache.shutdown(10, TimeUnit.SECONDS); // graceful shutdown | |
System.out.println("Time for " + MAX + " puts is " + (t2 - t1) + " ms"); | |
// open another connection | |
cache = new MemcachedClient(new InetSocketAddress(server, port)); | |
... | |
... |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
// MembaseTest2 | |
package tutorial; | |
import java.net.InetSocketAddress; | |
import net.spy.memcached.MemcachedClient; | |
/** | |
* simulates writing / reading from two different clients | |
*/ | |
public class MembaseTest2 |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
// MembaseTest1 | |
package tutorial; | |
import java.net.InetSocketAddress; | |
import net.spy.memcached.MemcachedClient; | |
/** | |
* Write / Read from Membase | |
* | |
* @author sujee | |
* |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
// MembaseTest3 | |
package tutorial; | |
import java.net.InetSocketAddress; | |
import java.util.concurrent.TimeUnit; | |
import net.spy.memcached.MemcachedClient; | |
/** | |
* simulates writing / reading from two different clients |
NewerOlder