Skip to content

Instantly share code, notes, and snippets.

@ijaureguialzo

ijaureguialzo/ejemplo_mapreduce.md

Last active June 11, 2020 15:18
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save ijaureguialzo/6dc1b31f8e0f7d4b323fe2240a58f11f to your computer and use it in GitHub Desktop.
Save ijaureguialzo/6dc1b31f8e0f7d4b323fe2240a58f11f to your computer and use it in GitHub Desktop.
Download ZIP
Ejemplo MapReduce
Raw
ejemplo_mapreduce.md 
make hadoop
wc -l /ficheros/GlobalLandTemperaturesByCity.csv
awk 'NR != 1 {print}' \
    /ficheros/GlobalLandTemperaturesByCity.csv | \
    hdfs dfs -put - \
    hdfs://namenode/user/egibide/temperaturas.csv
../ficheros/GlobalLandTemperaturesByCity.csv ../ficheros/salida

hadoop jar /ficheros/EjemploMapReduce.jar \
    com.jaureguialzo.Main \
    -Dnum-reducers=10 \
    /user/egibide/temperaturas.csv \
    /user/egibide/salida
hdfs dfs -getmerge \
    /user/egibide/salida \
    /ficheros/resultado.txt
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment