Skip to content

Instantly share code, notes, and snippets.

@sloev
Created Apr 21, 2020
Embed
What would you like to do?
how to encode values to csv string in python 3 (args to csv string)
import csv
from io import StringIO
class ArgsToCsv:
def __init__(self, seperator=","):
self.seperator = seperator
self.buffer = StringIO()
self.writer = csv.writer(self.buffer)
def stringify(self, *args):
self.writer.writerow(args)
value = self.buffer.getvalue().strip("\r\n")
self.buffer.seek(0)
self.buffer.truncate(0)
return value + "\n"
from args_to_csv import ArgsToCsv
csv_formatter = ArgsToCsv()
output = ""
header = ["age", "text", "my_array"]
output += csv_formatter.stringify(*header)
output += csv_formatter.stringify(
10,
"""
lol i have some pretty
"freaky"
strings right here \' yo!
""",
[10, 20, 30],
)
output += csv_formatter.stringify(10, "hi there", None)
assert output.splitlines() == [
"age,text,my_array",
'10,"',
" lol i have some pretty",
' ""freaky""',
" strings right here ' yo!",
' ","[10, 20, 30]"',
"10,hi there,",
]
@rupestre-campos
Copy link

rupestre-campos commented Jul 17, 2020

Hello Sir, Thanks for your time doing this beauty string io csv writer!!

got your code from stack overflow.

I modified it and would like to share in the way I used at the end.
I don't know if thats something that you had in mind but, for me worked very fast.
Could create ios greater than hundreds of thousands lines.

file csv_string.py

from io import StringIO


class ArgsToCsv:
    def __init__(self, seperator=","):
        self.seperator = seperator
        self.buffer = StringIO()
        self.writer = csv.writer(self.buffer)

    def stringify(self):
        value = self.buffer.getvalue().strip("\r\n")
        self.buffer.seek(0)
        return value + "\n"

    def add_data(self, *args):
        self.writer.writerow(args)

so I used like this:

import csv_string

csv_formatter = csv_string.ArgsToCsv()
n = 0
for i in shapefile:
    csv_formatter.add_data(n,geom.ExportToWkb())

cur.copy_from(csv_formatter.buffer, table_name, sep=',')

I believe the stringify function could be separated from the add_data function as I was adding only to stringIO csv. But remains as a functionality to the class as you developed I believe. I did not use it at the end. I removed the truncate part to not get rid of the object content when printing the string.

If I am very wrong let me know!

@sloev
Copy link
Author

sloev commented Jul 20, 2020

thanks for collaboration.

i think our usecases are quite different

i needed a way to on-the-fly encode args to strings as i was continously writing to a http socket (streaming csv lines back to client)

you seem to "just" want to encode stuff to csv to afterwards take the resulting string buffer and use it for some db action.

i think you could get away with this cheaper with just:

buffer = StringIO()
writer = csv.writer(buffer)
writer.writerow(*column_names)
for row in x:
    writer.writerow(*row)

cur.copy_from(buffer, table_name, sep=',')

no need to do classy stuff here

@rupestre-campos
Copy link

rupestre-campos commented Jul 20, 2020

Hey man, thank's! You are right about the class, this way you propose to me is way more simple, and I'll sticky to it.

@harisreedhar
Copy link

harisreedhar commented Aug 28, 2021

@sloev Thanks for the snippet. I think you forgot to use separator.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment