Skip to content

Instantly share code, notes, and snippets.

@mitsuhisaT
Created September 1, 2019 23:37
Show Gist options
  • Save mitsuhisaT/c343245851f05d96b625521cdcae3be9 to your computer and use it in GitHub Desktop.
Save mitsuhisaT/c343245851f05d96b625521cdcae3be9 to your computer and use it in GitHub Desktop.
about use converters matplotlib.dates.datestr2num in numpy.loadtxt
#!/usr/bin/env python
"""Draw line graph of 118 years August temperature line graph.
Past data of temperature gets from
https://www.data.jma.go.jp/gmd/risk/obsdl/index.php
Sample original data
2018,8,8,30.3,8,2,35.1,8,2,24.8,8,2
2018,8,9,31.2,8,2,37.3,8,2,26.7,8,2
2018,8,10,30.5,8,2,35.4,8,2,27.3,8,2
2018,8,11,29.8,8,2,34.1,8,2,26.1,8,2
2018,8,12,30.4,8,2,34.7,8,2,27.2,8,2
Replace two times
s/(\d{4}),8,(\d{1})/'$1-08-0$3'/
s/(\d{4}),8,(\d{2})/'$1-08-$3'/
Imput data
'2018-08-08',30.3,8,2,35.1,8,2,24.8,8,2
'2018-08-09',31.2,8,2,37.3,8,2,26.7,8,2
'2018-08-10',30.5,8,2,35.4,8,2,27.3,8,2
'2018-08-11',29.8,8,2,34.1,8,2,26.1,8,2
'2018-08-12',30.4,8,2,34.7,8,2,27.2,8,2
"""
from matplotlib import pyplot as plt
from matplotlib import dates as mdates
import numpy as np
dt = np.dtype({
'names': ['date', 'mean', 'max', 'min'],
'formats': ['f8', 'f4', 'f4', 'f4']
})
# Matplotlib represents dates using floating point numbers specifying the
# number of days since 0001-01-01 UTC, plus 1.
# For example, 0001-01-01, 06:00 is 1.25, not 0.25.
# Values < 1, i.e. dates before 0001-01-01 UTC are not supported.
# Reading data converting date string number.
# My sample data can't work `converters={0: mdates.datestr2num}`,
# becase '2018-08-01' is Bytes literals b'2018-08-01'.
# c.f. https://docs.python.org/3/reference/lexical_analysis.html#string-and-bytes-literals
datas = np.loadtxt('data/august-temperature.csv', dtype=dt, skiprows=1,
delimiter=',',
converters={0: lambda s: mdates.datestr2num(s.decode())},
usecols=(0, 1, 4, 7))
# print(datas)
# round to nearest years.
datemin = np.datetime64(mdates.num2date(datas['date'][0]), 'Y')
datemax = np.datetime64(mdates.num2date(
datas['date'][-1]), 'Y') + np.timedelta64(1, 'Y')
# print(datemax, datemin)
fig, ax = plt.subplots()
ax.set_xlim(datemin, datemax)
# format the cords message box
ax.format_xdata = mdates.DateFormatter('%Y')
ax.format_ydata = lambda x: '$%2.1f' % x # format the temperature.
ax.grid(True)
ax.plot('date', 'max', data=datas)
# rotates and right aligns the x labels, and moves the bottom of the
# axes up to make room for them
fig.autofmt_xdate()
plt.title('every August Max temperature at Tokyo from 1901 to 2018.')
plt.show()
@mitsuhisaT
Copy link
Author

mitsuhisaT commented Sep 1, 2019

過去の気温データを気象庁のサイトから CSVファイルをダウンロード
確認環境は
macOS High Sierra and Mojave
Python 3.7.4
必要なモジュールは pip でインストール

通常は、
converters={0: mdates.datestr2num} で Ok な様ですが、実行時エラーが発生
読み込んだデータを調べて、
converters={0: lambda s: mdates.datestr2num(s.decode())} で解決できた。

@mitsuhisaT
Copy link
Author

mitsuhisaT commented Sep 2, 2019

see numpy.loadtxt document

change line 46 to converters={0: mdates.datestr2num}
next error occurred.

Why? Because '2018-08-07' was read Bytes literal b'2018-08-07'

Traceback (most recent call last):
  File "./mean-temperature-graph_all.py", line 51, in <module>
    usecols=(0, 1, 4, 7))
  File "/Users/mitsu/.pyenv/versions/3.7.4/lib/python3.7/site-packages/numpy/lib/npyio.py", line 1146, in loadtxt
    for x in read_data(_loadtxt_chunksize):
  File "/Users/mitsu/.pyenv/versions/3.7.4/lib/python3.7/site-packages/numpy/lib/npyio.py", line 1074, in read_data
    items = [conv(val) for (conv, val) in zip(converters, vals)]
  File "/Users/mitsu/.pyenv/versions/3.7.4/lib/python3.7/site-packages/numpy/lib/npyio.py", line 1074, in <listcomp>
    items = [conv(val) for (conv, val) in zip(converters, vals)]
  File "/Users/mitsu/.pyenv/versions/3.7.4/lib/python3.7/site-packages/numpy/lib/npyio.py", line 1133, in tobytes_first
    return conv(x.encode("latin1"))
  File "/Users/mitsu/.pyenv/versions/3.7.4/lib/python3.7/site-packages/matplotlib/dates.py", line 387, in datestr2num
    return date2num(_dateutil_parser_parse_np_vectorized(d))
  File "/Users/mitsu/.pyenv/versions/3.7.4/lib/python3.7/site-packages/matplotlib/dates.py", line 418, in date2num
    return _to_ordinalf(d)
  File "/Users/mitsu/.pyenv/versions/3.7.4/lib/python3.7/site-packages/matplotlib/dates.py", line 226, in _to_ordinalf
    base = float(dt.toordinal())
AttributeError: 'numpy.ndarray' object has no attribute 'toordinal'

@mitsuhisaT
Copy link
Author

max-temperature

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment