Skip to content

Instantly share code, notes, and snippets.

View aajisaka's full-sized avatar

Akira Ajisaka aajisaka

View GitHub Profile
@aajisaka
aajisaka / BlockManager.java
Created August 27, 2014 05:43
BlockManager#removeStoredBlock(CDH4.4.0)
/**
* Modify (block-->datanode) map. Possibly generate replication tasks, if the
* removed block is still valid.
*/
public void removeStoredBlock(Block block, DatanodeDescriptor node) {
if(blockLog.isDebugEnabled()) {
blockLog.debug("BLOCK* removeStoredBlock: "
+ block + " from " + node);
}
assert (namesystem.hasWriteLock());
@aajisaka
aajisaka / HDP-CHANGES.txt
Created December 3, 2014 06:02
HDP 2.2.0.0-2041 Change log
Changes only in Hortonworks github and not in hadoop-2.6.0 and hence not covered by CHANGES.txt.
INCOMPATIBLE CHANGES
NEW FEATURES
HDFS-6581. Support for writing to single replica in RAM.
Applied HDFS-6581.merge.15.patch to Champlain. This is a consolidated
patch containing all feature changes and was applied to trunk.
(Multiple contributors via Arpit Agarwal)
@aajisaka
aajisaka / bugspots_hadoop-trunk.out
Created October 27, 2015 14:31
Applied bugspots for Apache Hadoop (trunk)
Scanning . repo
Found 1664 bugfix commits, with 3682 hotspots:
Fixes:
- Fixed basic hadoop core targets
- fixed test test-core and test-contrib targets
- Fix jdiff and javadoc target
- To fix test targets
- Fix test-patch.sh script for eclipse classpath verification
- HADOOP-4687. Fix some of the remaining javadoc warnings.
@aajisaka
aajisaka / BlockPlacementPolicyDefault.java
Created November 4, 2015 17:16
Created gist to add comments
/**
* Licensed to the Apache Software Foundation (ASF) under one
* or more contributor license agreements. See the NOTICE file
* distributed with this work for additional information
* regarding copyright ownership. The ASF licenses this file
* to you under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
@aajisaka
aajisaka / WordCount2.java
Created October 26, 2013 00:49
WordCount v2.0 for MapReduce Tutorial
import java.io.BufferedReader;
import java.io.FileReader;
import java.io.IOException;
import java.net.URI;
import java.util.ArrayList;
import java.util.HashSet;
import java.util.List;
import java.util.Set;
import java.util.StringTokenizer;
@aajisaka
aajisaka / jar_list_hadoop-2.7.2
Created April 7, 2016 04:58
find . -name "*.jar" | xargs -n1 basename | sort | uniq
activation-1.1.jar
annotations-api.jar
aopalliance-1.0.jar
apacheds-i18n-2.0.0-M15.jar
apacheds-kerberos-codec-2.0.0-M15.jar
api-asn1-api-1.0.0-M20.jar
api-i18n-1.0.0-M20.jar
api-util-1.0.0-M20.jar
asm-3.2.jar
avro-1.7.4.jar
@aajisaka
aajisaka / THIRD-PARTY.txt
Created April 7, 2016 09:38
Output of "mvn license:aggregate-add-third-party" using license-maven-plugin
Lists of 253 third-party dependencies.
(BSD License) AntLR Parser Generator (antlr:antlr:2.7.7 - http://www.antlr.org/)
(Public Domain) AOP alliance (aopalliance:aopalliance:1.0 - http://aopalliance.sourceforge.net)
(BSD) ASM Core (asm:asm:3.2 - http://asm.objectweb.org/asm/)
(Unknown license) bcprov-jdk15 (bouncycastle:bcprov-jdk15:140 - no url defined)
(ASF 2.0) Code Generation Library (cglib:cglib:2.2 - http://cglib.sourceforge.net/)
(Eclipse Public License - v 1.0) (GNU Lesser General Public License) Logback Classic Module (ch.qos.logback:logback-classic:1.1.2 - http://logback.qos.ch)
(Eclipse Public License - v 1.0) (GNU Lesser General Public License) Logback Core Module (ch.qos.logback:logback-core:1.1.2 - http://logback.qos.ch)
(The Apache Software License, Version 2.0) ZkClient (com.101tec:zkclient:0.3 - https://github.com/sgroschupf/zkclient)
@aajisaka
aajisaka / THIRD-PARTY.txt
Created April 25, 2016 13:50
(branch-2) $ mvn license:aggregate-add-third-party
Lists of 245 third-party dependencies.
(BSD License) AntLR Parser Generator (antlr:antlr:2.7.7 - http://www.antlr.org/)
(Public Domain) AOP alliance (aopalliance:aopalliance:1.0 - http://aopalliance.sourceforge.net)
(BSD) ASM Core (asm:asm:3.2 - http://asm.objectweb.org/asm/)
(Unknown license) bcprov-jdk15 (bouncycastle:bcprov-jdk15:140 - no url defined)
(ASF 2.0) Code Generation Library (cglib:cglib:2.2 - http://cglib.sourceforge.net/)
(Eclipse Public License - v 1.0) (GNU Lesser General Public License) Logback Classic Module (ch.qos.logback:logback-classic:1.1.2 - http://logback.qos.ch)
(Eclipse Public License - v 1.0) (GNU Lesser General Public License) Logback Core Module (ch.qos.logback:logback-core:1.1.2 - http://logback.qos.ch)
(Apache License, Version 2.0) AWS SDK for Java - Core (com.amazonaws:aws-java-sdk-core:1.10.6 - https://aws.amazon.com/sdkforjava)
(Apache License, Version 2.0) AWS Java SDK for AWS KMS (com.amazonaws:aws-java-sdk-kms:1.10.6 - https:/
@aajisaka
aajisaka / license-maven-plugin.patch
Created April 25, 2016 14:07
Patch used for aggregating licenses by license-maven-plugin
diff --git a/pom.xml b/pom.xml
index e4623b6..823b4c2 100644
--- a/pom.xml
+++ b/pom.xml
@@ -285,6 +285,11 @@ xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xs
<artifactId>maven-checkstyle-plugin</artifactId>
<version>${maven-checkstyle-plugin.version}</version>
</plugin>
+ <plugin>
+ <groupId>org.codehaus.mojo</groupId>
@aajisaka
aajisaka / GitParse.java
Last active April 25, 2016 14:12
Parse git log of Apache Hadoop for analysis
package main;
import java.io.BufferedReader;
import java.io.File;
import java.io.FileOutputStream;
import java.io.FileReader;
import java.io.IOException;
import java.nio.charset.Charset;
import java.text.ParseException;
import java.text.SimpleDateFormat;