Skip to content

Instantly share code, notes, and snippets.

@szaghi

szaghi/load_cvs_in_fortran.md

Last active February 1, 2016 09:41
Show Gist options
  • Save szaghi/adc94b90cd196c864de3 to your computer and use it in GitHub Desktop.
Save szaghi/adc94b90cd196c864de3 to your computer and use it in GitHub Desktop.
Download ZIP
Loading a formatted CVS in Fortran
Raw
load_cvs_in_fortran.md

A small snippet to load a formatted cvs

I assume that the date string is always 10 characters length.

The number of file records is assumed known at compile time: generalize it as you need.

program cvs_read
  implicit none

  character(len=10)  :: date
  integer, parameter :: file_records=11
  integer            :: year(file_records)
  integer            :: month(file_records)
  integer            :: day(file_records)
  real               :: a1(file_records)
  real               :: a2(file_records)
  real               :: a3(file_records)
  integer            :: file_unit
  integer            :: i

  open(newunit=file_unit, file='input.cvs')
  do i=1, file_records
    read(file_unit, *) date, a1(i), a2(i), a3(i)
    read(date(1:4), *) year(i)
    read(date(6:7), *) month(i)
    read(date(9:10), *) day(i)
  end do
  close(file_unit)
  do i=1, file_records
    print "(I4,1X,I2,1X,I2,1X,3(F5.1,1X))", year(i), month(i), day(i), a1(i), a2(i), a3(i)
  end do
  stop
end program cvs_read

Input cvs file test

1935-01-01 -5.5 -0.3 8.5 
1935-01-02 -6.0 -1.3 0.5 
1935-01-03 -9.0 -3.1 0.0 
1935-01-04 -10.7 -2.6 8.8 
1935-01-05 -8.1 -4.4 9.0 
1935-01-06 -10.9 -5.6 0.2 
1935-01-07 -12.3 -6.1 0.4 
1935-01-08 -11.8 -9.3 12.8 
1935-01-09 -15.3 -11.1 2.3 
1935-01-10 -17.2 -11.4 0.1 
1935-01-11 -16.6 -7.2 0.0

Ouput

Compiled with GNU gfortran 5.3 it gives

→ a.out 
1935  1  1  -5.5  -0.3   8.5 
1935  1  2  -6.0  -1.3   0.5 
1935  1  3  -9.0  -3.1   0.0 
1935  1  4 -10.7  -2.6   8.8 
1935  1  5  -8.1  -4.4   9.0 
1935  1  6 -10.9  -5.6   0.2 
1935  1  7 -12.3  -6.1   0.4 
1935  1  8 -11.8  -9.3  12.8 
1935  1  9 -15.3 -11.1   2.3 
1935  1 10 -17.2 -11.4   0.1 
1935  1 11 -16.6  -7.2   0.0

Seems to work as expected.

@szaghi
Copy link
Author

szaghi commented Jan 29, 2016

New version

As requested by hd

Code

program separate_date
implicit none
integer, parameter   :: unit_file=11
integer              :: records_number
character(len=1000)  :: line
real,    allocatable :: ms1(:)
real,    allocatable :: ms2(:)
real,    allocatable :: ms3(:)
integer, allocatable :: year(:)
integer, allocatable :: month(:)
integer, allocatable :: day(:)
character(6)         :: ms1str
character(6)         :: ms2str
character(6)         :: ms3str
integer              :: i
integer              :: index1
integer              :: index2
integer              :: index3

open(unit=unit_file, file="input.csv")
records_number = 0
count_records : do
  read(unit_file, *, end=10)
  records_number = records_number + 1
end do count_records
10 rewind(unit=unit_file)
records_number = records_number - 1 ! eliminate first line

allocate(ms1(1:records_number))
allocate(ms2(1:records_number))
allocate(ms3(1:records_number))
allocate(year(1:records_number))
allocate(month(1:records_number))
allocate(day(1:records_number))

ms1 = -999.
ms2 = -999.
ms3 = -999.

read(unit_file, '(A)') line
print "(A)", trim(line)
do i=1, records_number
  read(unit_file, '(A)') line
  ! parsing date
  read(line(1:4), *) year(i)
  read(line(6:7), *) month(i)
  read(line(9:10), *) day(i)
  ! parsing real values
  index1 = index(line, ',')
  index2 = index1 + index(line(index1+1:), ',')
  index3 = index2 + index(line(index2+1:), ',')

  if (index1+1<=index2-1) read(line(index1+1:index2-1), *) ms1(i)
  if (index2+1<=index3-1) read(line(index2+1:index3-1), *) ms2(i)
                          read(line(index3+1:),         *) ms3(i)
end do
close(unit=unit_file)

open(unit=unit_file, file="output.text")
do i=1, records_number
  if (int(ms1(i))==-999) then
    write(ms1str, "(I3)") int(ms1(i))
  else
     write(ms1str, "(F6.1)") ms1(i)
  end if
  if (int(ms2(i))==-999) then
    write(ms2str, "(I3)") int(ms2(i))
  else
     write(ms2str, "(F6.1)") ms2(i)
  end if
  if (int(ms3(i))==-999) then
    write(ms3str, "(I3)") int(ms3(i))
  else
     write(ms3str, "(F6.1)") ms3(i)
  end if
  write(unit_file, "(I4,1X,I2.2,1X,I2.2,1X,A)") year(i), month(i), day(i), ms1str//' '//ms2str//' '//ms3str
end do
close(unit=unit_file)

end program separate_date

input

first line: do with it what you want
1924-01-01,,,0.0 
1924-01-02,,,0.0 
1924-01-03,,,0.0 
1924-01-04,,,7.9 
1924-01-05,,,0.0 
1924-01-06,,,0.0 
1924-01-07,,,0.0 
1924-01-08,,,0.0 
1924-01-09,,,3.2 
1924-01-10,,,0.8 
1924-01-11,,,0.0 
1924-01-12,,,0.0 
1924-01-13,9.0,2.0,0.0 
1924-01-14,10.0,2.0,0.0 
1924-01-15,10.0,5.0,0.0
1924-01-16,12.0,5.0,0.0
1924-01-17,9.0,5.0,0.0
1924-01-18,12.0,4.0,0.0
1924-01-19,12.0,5.0,0.0
1924-01-20,8.0,3.0,0.0
1924-01-21,4.0,0.0,0.0
1924-01-22,4.0,2.0,0.0

Output

→ ifort test.f90
→ a.out                                        
first line: do with it what you want
→ cat output.text                              
1924 01 01    0.0   0.0   0.0
1924 01 02    0.0   0.0   0.0
1924 01 03    0.0   0.0   0.0
1924 01 04    0.0   0.0   7.9
1924 01 05    0.0   0.0   0.0
1924 01 06    0.0   0.0   0.0
1924 01 07    0.0   0.0   0.0
1924 01 08    0.0   0.0   0.0
1924 01 09    0.0   0.0   3.2
1924 01 10    0.0   0.0   0.8
1924 01 11    0.0   0.0   0.0
1924 01 12    0.0   0.0   0.0
1924 01 13    9.0   2.0   0.0
1924 01 14   10.0   2.0   0.0
1924 01 15   10.0   5.0   0.0
1924 01 16   12.0   5.0   0.0
1924 01 17    9.0   5.0   0.0
1924 01 18   12.0   4.0   0.0
1924 01 19   12.0   5.0   0.0
1924 01 20    8.0   3.0   0.0
1924 01 21    4.0   0.0   0.0
1924 01 22    4.0   2.0   0.0

This should be what you want 😄

Note

I have zero-padded month and day integer and I have made general the number of records parsed.

@szaghi
Copy link
Author

szaghi commented Feb 1, 2016

New versions with BLANKs...

Code

program separate_date
implicit none
integer, parameter   :: unit_file=11
real,    parameter   :: blank_value=-999.
integer              :: records_number
character(len=1000)  :: line
real,    allocatable :: ms1(:)
real,    allocatable :: ms2(:)
real,    allocatable :: ms3(:)
integer, allocatable :: year(:)
integer, allocatable :: month(:)
integer, allocatable :: day(:)
integer              :: i
integer              :: index1
integer              :: index2
integer              :: index3
character(6)         :: ms1str
character(6)         :: ms2str
character(6)         :: ms3str

open(unit=unit_file, file="input.csv")
records_number = 0
count_records : do
  read(unit_file, *, end=10)
  records_number = records_number + 1
end do count_records
10 rewind(unit=unit_file)
records_number = records_number - 1 ! eliminate first line

allocate(ms1(1:records_number))
allocate(ms2(1:records_number))
allocate(ms3(1:records_number))
allocate(year(1:records_number))
allocate(month(1:records_number))
allocate(day(1:records_number))

ms1 = blank_value
ms2 = blank_value
ms3 = blank_value

read(unit_file, '(A)') line
print "(A)", trim(line)
do i=1, records_number
  read(unit_file, '(A)') line
  ! parsing date
  read(line(1:4), *) year(i)
  read(line(6:7), *) month(i)
  read(line(9:10), *) day(i)
  ! parsing real values
  index1 = index(line, ',')
  index2 = index1 + index(line(index1+1:), ',')
  index3 = index2 + index(line(index2+1:), ',')

  if (index1+1<=index2-1) read(line(index1+1:index2-1), *) ms1(i)
  if (index2+1<=index3-1) read(line(index2+1:index3-1), *) ms2(i)
                          read(line(index3+1:),         *) ms3(i)
end do
close(unit=unit_file)

open(unit=unit_file, file="output.text")
do i=1, records_number
  ms1str = sanitized_value(ms1(i))
  ms2str = sanitized_value(ms2(i))
  ms3str = sanitized_value(ms3(i))
  write(unit_file, "(I4,1X,I2.2,1X,I2.2,1X,A)") year(i), month(i), day(i), adjustr(ms1str)//' '//&
                                                                           adjustr(ms2str)//' '//&
                                                                           adjustr(ms3str)
end do
close(unit=unit_file)

contains
  function sanitized_value(parsed_value)
    !< Sanitize the parsed value: return the pretty printed value if correctly parsed or
    !< a "BLANK" string otherwise.
    real, intent(in) :: parsed_value
    character(6)     :: sanitized_value

    if (int(parsed_value)==int(blank_value)) then
      write(sanitized_value, "(A)") 'BLANK'
    else
      write(sanitized_value, "(F6.1)") parsed_value
    end if
  end function sanitized_value
end program separate_date

Output

→ cat output.text
1924 01 01  BLANK  BLANK    0.0
1924 01 02  BLANK  BLANK    0.0
1924 01 03  BLANK  BLANK    0.0
1924 01 04  BLANK  BLANK    7.9
1924 01 05  BLANK  BLANK    0.0
1924 01 06  BLANK  BLANK    0.0
1924 01 07  BLANK  BLANK    0.0
1924 01 08  BLANK  BLANK    0.0
1924 01 09  BLANK  BLANK    3.2
1924 01 10  BLANK  BLANK    0.8
1924 01 11  BLANK  BLANK    0.0
1924 01 12  BLANK  BLANK    0.0
1924 01 13    9.0    2.0    0.0
1924 01 14   10.0    2.0    0.0
1924 01 15   10.0    5.0    0.0
1924 01 16   12.0    5.0    0.0
1924 01 17    9.0    5.0    0.0
1924 01 18   12.0    4.0    0.0
1924 01 19   12.0    5.0    0.0
1924 01 20    8.0    3.0    0.0
1924 01 21    4.0    0.0    0.0
1924 01 22    4.0    2.0    0.0

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment