Justin Zhang schnell18

Think and tinker in an infinite loop

38 followers · 11 following

UCloud
Shanghai
https://www.tinkerit.cf

View GitHub Profile

schnell18 / ncdc.sh

Created December 21, 2016 12:10 — forked from aligusnet/ncdc.sh

Download a weather dataset from the National Climatic Data Center (NCDC, http://www .ncdc.noaa.gov/). Prepare it for examples of "Hadoop: The Definitive Guide" book by Tom White. http://www.amazon.com/Hadoop-Definitive-Guide-Tom-White/dp/1449311520 Usage: ./ncdc.sh 1901 1930 # download wheather datasets for period from 1901 to 1930.

	#!/usr/bin/env bash

	# global parameters
	g_tmp_folder="ncdc_tmp";
	g_output_folder="ncdc_data";

	g_remote_host="ftp3.ncdc.noaa.gov";
	g_remote_path="pub/data/noaa";