Skip to content

Instantly share code, notes, and snippets.

@nickmancol
Last active December 6, 2021 20:46
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save nickmancol/4b8ef22814cb706c00fc7cfe87acb8f3 to your computer and use it in GitHub Desktop.
Save nickmancol/4b8ef22814cb706c00fc7cfe87acb8f3 to your computer and use it in GitHub Desktop.
InfluxDB Write of global CO2 emissions 1751-2014, Source: Tom Boden (Oak Ridge National Laboratory), Gregg Marland (Appalachian State University), and Bob Andres (Oak Ridge National Laboratory)
We can make this file beautiful and searchable if this error is corrected: It looks like row 2 should actually have 8 columns, instead of 7. in line 1.
"Year","Total carbon emissions from fossil fuel consumption and cement production (million metric tons of C)","Carbon emissions from gas fuel consumption","Carbon emissions from liquid fuel consumption","Carbon emissions from solid fuel consumption","Carbon emissions from cement production","Carbon emissions from gas flaring","Per capita carbon emissions (metric tons of carbon; after 1949 only)"
1751,3,0,0,3,0,0
1752,3,0,0,3,0,0
1753,3,0,0,3,0,0
1754,3,0,0,3,0,0
1755,3,0,0,3,0,0
1756,3,0,0,3,0,0
1757,3,0,0,3,0,0
1758,3,0,0,3,0,0
1759,3,0,0,3,0,0
1760,3,0,0,3,0,0
1761,3,0,0,3,0,0
1762,3,0,0,3,0,0
1763,3,0,0,3,0,0
1764,3,0,0,3,0,0
1765,3,0,0,3,0,0
1766,3,0,0,3,0,0
1767,3,0,0,3,0,0
1768,3,0,0,3,0,0
1769,3,0,0,3,0,0
1770,3,0,0,3,0,0
1771,4,0,0,4,0,0
1772,4,0,0,4,0,0
1773,4,0,0,4,0,0
1774,4,0,0,4,0,0
1775,4,0,0,4,0,0
1776,4,0,0,4,0,0
1777,4,0,0,4,0,0
1778,4,0,0,4,0,0
1779,4,0,0,4,0,0
1780,4,0,0,4,0,0
1781,5,0,0,5,0,0
1782,5,0,0,5,0,0
1783,5,0,0,5,0,0
1784,5,0,0,5,0,0
1785,5,0,0,5,0,0
1786,5,0,0,5,0,0
1787,5,0,0,5,0,0
1788,5,0,0,5,0,0
1789,5,0,0,5,0,0
1790,5,0,0,5,0,0
1791,6,0,0,6,0,0
1792,6,0,0,6,0,0
1793,6,0,0,6,0,0
1794,6,0,0,6,0,0
1795,6,0,0,6,0,0
1796,6,0,0,6,0,0
1797,7,0,0,7,0,0
1798,7,0,0,7,0,0
1799,7,0,0,7,0,0
1800,8,0,0,8,0,0
1801,8,0,0,8,0,0
1802,10,0,0,10,0,0
1803,9,0,0,9,0,0
1804,9,0,0,9,0,0
1805,9,0,0,9,0,0
1806,10,0,0,10,0,0
1807,10,0,0,10,0,0
1808,10,0,0,10,0,0
1809,10,0,0,10,0,0
1810,10,0,0,10,0,0
1811,11,0,0,11,0,0
1812,11,0,0,11,0,0
1813,11,0,0,11,0,0
1814,11,0,0,11,0,0
1815,12,0,0,12,0,0
1816,13,0,0,13,0,0
1817,14,0,0,14,0,0
1818,14,0,0,14,0,0
1819,14,0,0,14,0,0
1820,14,0,0,14,0,0
1821,14,0,0,14,0,0
1822,15,0,0,15,0,0
1823,16,0,0,16,0,0
1824,16,0,0,16,0,0
1825,17,0,0,17,0,0
1826,17,0,0,17,0,0
1827,18,0,0,18,0,0
1828,18,0,0,18,0,0
1829,18,0,0,18,0,0
1830,24,0,0,24,0,0
1831,23,0,0,23,0,0
1832,23,0,0,23,0,0
1833,24,0,0,24,0,0
1834,24,0,0,24,0,0
1835,25,0,0,25,0,0
1836,29,0,0,29,0,0
1837,29,0,0,29,0,0
1838,30,0,0,30,0,0
1839,31,0,0,31,0,0
1840,33,0,0,33,0,0
1841,34,0,0,34,0,0
1842,36,0,0,36,0,0
1843,37,0,0,37,0,0
1844,39,0,0,39,0,0
1845,43,0,0,43,0,0
1846,43,0,0,43,0,0
1847,46,0,0,46,0,0
1848,47,0,0,47,0,0
1849,50,0,0,50,0,0
1850,54,0,0,54,0,0
1851,54,0,0,54,0,0
1852,57,0,0,57,0,0
1853,59,0,0,59,0,0
1854,69,0,0,69,0,0
1855,71,0,0,71,0,0
1856,76,0,0,76,0,0
1857,77,0,0,77,0,0
1858,78,0,0,78,0,0
1859,83,0,0,83,0,0
1860,91,0,0,91,0,0
1861,95,0,0,95,0,0
1862,97,0,0,96,0,0
1863,104,0,0,103,0,0
1864,112,0,0,112,0,0
1865,119,0,0,119,0,0
1866,122,0,0,122,0,0
1867,130,0,0,130,0,0
1868,135,0,0,134,0,0
1869,142,0,0,142,0,0
1870,147,0,1,146,0,0
1871,156,0,1,156,0,0
1872,173,0,1,173,0,0
1873,184,0,1,183,0,0
1874,174,0,1,173,0,0
1875,188,0,1,187,0,0
1876,191,0,1,190,0,0
1877,194,0,2,192,0,0
1878,196,0,2,194,0,0
1879,210,0,3,207,0,0
1880,236,0,3,233,0,0
1881,243,0,4,239,0,0
1882,256,0,4,252,0,0
1883,272,0,3,269,0,0
1884,275,0,4,271,0,0
1885,277,1,4,273,0,0
1886,281,2,5,275,0,0
1887,295,3,5,287,0,0
1888,327,5,5,317,0,0
1889,327,3,6,318,0,0
1890,356,3,8,345,0,0
1891,372,2,9,360,0,0
1892,374,2,9,363,0,0
1893,370,2,10,358,0,0
1894,383,2,9,372,0,0
1895,406,2,11,393,0,0
1896,419,2,12,405,0,0
1897,440,2,13,425,0,0
1898,465,2,13,449,0,0
1899,507,3,14,491,0,0
1900,534,3,16,515,0,0
1901,552,4,18,531,0,0
1902,566,4,19,543,0,0
1903,617,4,20,593,0,0
1904,624,4,23,597,0,0
1905,663,5,23,636,0,0
1906,707,5,23,680,0,0
1907,784,5,28,750,0,0
1908,750,5,30,714,0,0
1909,785,6,32,747,0,0
1910,819,7,34,778,0,0
1911,836,7,36,792,0,0
1912,879,8,37,834,0,0
1913,943,8,41,895,0,0
1914,850,8,42,800,0,0
1915,838,9,45,784,0,0
1916,901,10,48,842,0,0
1917,955,11,54,891,0,0
1918,936,10,53,873,0,0
1919,806,10,61,735,0,0
1920,932,11,78,843,0,0
1921,803,10,84,709,0,0
1922,845,11,94,740,0,0
1923,970,14,111,845,0,0
1924,963,16,110,836,0,0
1925,975,17,116,842,0,0
1926,983,19,119,846,0,0
1927,1062,21,136,905,0,0
1928,1065,23,143,890,10,0
1929,1145,28,160,947,10,0
1930,1053,28,152,862,10,0
1931,940,25,147,759,8,0
1932,847,24,141,675,7,0
1933,893,25,154,708,7,0
1934,973,28,162,775,8,0
1935,1027,30,176,811,9,0
1936,1130,34,192,893,11,0
1937,1209,38,219,941,11,0
1938,1142,37,214,880,12,0
1939,1192,38,222,918,13,0
1940,1299,42,229,1017,11,0
1941,1334,42,236,1043,12,0
1942,1342,45,222,1063,11,0
1943,1391,50,239,1092,10,0
1944,1383,54,275,1047,7,0
1945,1160,59,275,820,7,0
1946,1238,61,292,875,10,0
1947,1392,67,322,992,12,0
1948,1469,76,364,1015,14,0
1949,1419,81,362,960,16,0
1950,1630,97,423,1070,18,23,0.65
1951,1767,115,479,1129,20,24,0.69
1952,1795,124,504,1119,22,26,0.69
1953,1841,131,533,1125,24,27,0.69
1954,1865,138,557,1116,27,27,0.69
1955,2042,150,625,1208,30,31,0.74
1956,2177,161,679,1273,32,32,0.78
1957,2270,178,714,1309,34,35,0.79
1958,2330,192,731,1336,36,35,0.80
1959,2454,206,789,1382,40,36,0.83
1960,2569,227,849,1410,43,39,0.85
1961,2580,240,904,1349,45,42,0.84
1962,2686,263,980,1351,49,44,0.86
1963,2833,286,1052,1396,51,47,0.89
1964,2995,316,1137,1435,57,51,0.92
1965,3130,337,1219,1460,59,55,0.94
1966,3288,364,1323,1478,63,60,0.97
1967,3393,392,1423,1448,65,66,0.98
1968,3566,424,1551,1448,70,73,1.01
1969,3780,467,1673,1486,74,80,1.05
1970,4053,493,1839,1556,78,87,1.10
1971,4208,530,1947,1559,84,88,1.12
1972,4376,560,2057,1576,89,95,1.14
1973,4614,588,2241,1581,95,110,1.18
1974,4623,597,2245,1579,96,107,1.16
1975,4596,604,2132,1673,95,92,1.13
1976,4864,630,2314,1710,103,108,1.18
1977,5016,650,2398,1756,108,104,1.19
1978,5074,680,2392,1780,116,106,1.18
1979,5357,721,2544,1875,119,98,1.23
1980,5301,737,2422,1935,120,86,1.19
1981,5138,755,2289,1908,121,65,1.14
1982,5094,738,2196,1976,121,64,1.11
1983,5075,739,2176,1977,125,58,1.08
1984,5258,807,2199,2074,128,51,1.10
1985,5417,835,2186,2216,131,49,1.12
1986,5583,830,2293,2277,137,46,1.13
1987,5725,892,2306,2339,143,44,1.14
1988,5936,935,2412,2387,152,50,1.16
1989,6066,982,2459,2428,156,41,1.16
1990,6074,1026,2492,2359,157,40,1.14
1991,6142,1051,2601,2284,161,45,1.14
1992,6078,1085,2499,2290,167,36,1.11
1993,6070,1117,2515,2225,176,37,1.09
1994,6174,1133,2539,2278,186,39,1.09
1995,6305,1151,2560,2359,197,39,1.10
1996,6448,1198,2626,2382,203,40,1.11
1997,6556,1197,2701,2409,209,40,1.11
1998,6576,1224,2763,2343,209,36,1.10
1999,6561,1258,2741,2310,217,35,1.08
2000,6733,1289,2845,2327,226,46,1.10
2001,6893,1316,2848,2445,237,47,1.11
2002,6994,1342,2832,2518,252,49,1.11
2003,7376,1397,2958,2695,276,48,1.16
2004,7743,1443,3043,2906,298,54,1.20
2005,8042,1485,3068,3108,320,60,1.23
2006,8336,1534,3091,3293,356,62,1.26
2007,8503,1562,3071,3422,382,66,1.27
2008,8776,1630,3103,3587,388,69,1.30
2009,8697,1584,3042,3590,415,66,1.27
2010,9128,1696,3107,3812,446,67,1.32
2011,9503,1756,3134,4055,494,64,1.36
2012,9673,1783,3200,4106,519,65,1.36
2013,9773,1806,3220,4126,554,68,1.36
2014,9855,1823,3280,4117,568,68,1.36
#! /bin/bash
#1. iterates over the global_emissions.csv file and reads each comma-separated column
while IFS="," read -r Year Total GasFuel LiquidFuel SolidFuel Cement GasFlaring PerCapita
do
echo "Year: $Year , total: $Total , Per capita $PerCapita"
#2. Converts the value of the year into a UNIX timestamp
ts=`date "+%s" -u -d "Dec 31 $Year 23:59:59"`
#3. Inserts the total value of co2 per year
curl -i -XPOST "http://localhost:8086/api/v2/write?precision=s&orgID=$1&bucket=$2" \
--header "Authorization: Token $3" \
--data-raw "total_co2,source=CDIAC value=$Total $ts"
#4. Checks if there is a value per capita
if [ -z "$PerCapita" ]; then
PerCapita=0
fi
#5. Inserts the per capita value of co2 emissions per year
curl -i -XPOST "http://localhost:8086/api/v2/write?precision=s&orgID=$1&bucket=$2" \
--header "Authorization: Token $3" \
--data-raw "per_capita_co2,source=CDIAC value=$PerCapita $ts"
done < <(tail -n +2 global_emissions.csv)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment