Skip to content

Instantly share code, notes, and snippets.

@mmohiudd
Created September 17, 2012 16:17
Show Gist options
  • Star 1 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save mmohiudd/3738264 to your computer and use it in GitHub Desktop.
Save mmohiudd/3738264 to your computer and use it in GitHub Desktop.
Generic CSV parser
<?php // php csv_parser.php <table name> <csv target> <sql target>
ini_set('memory_limit', '8000M');
ini_set('max_execution_time', 3000);
if (empty($argc)) {
echo "Arguments missing";
exit(1);
}
if (!strstr($argv[0], basename(__FILE__))) {
echo "Permission denied";
exit(1);
}
$table_name = $argv[1];
$file = $argv[2];
$save_location = $argv[3];
$info = pathinfo($file);
$fp = @fopen($file, "r");
$line = fgets($fp, 4096); // read the first line, headers
$fields = explode(",", trim($line));
$sql_data = "";
$i=0;
$insert_values = array(); // will save all insert rows here
while (!feof($fp)) { // loop till end of file.
$line = fgets($fp, 4096); // read a line.
if (empty($line)) continue;
$field_values = array(); // will save all field values
$values = explode(",", trim($line));
// only to sanitize the values
foreach($values as $value){ // fetch all value
$field_values[] = "'" . addslashes(trim($value)) . "'"; // escape special characters
}
$insert_values[] = "(" . implode(",", $field_values) . ")";
}
$insert_values_chunks = array_chunk($insert_values, 30); // insert 30 rows at a time
foreach($insert_values_chunks as $insert_values){
$sql = "INSERT INTO " . $table_name ." (" . implode(",", $fields). ") VALUES \n";
$sql .= implode(",\n", $insert_values) . ";\n\n";
$sql_data .= $sql;
}
// if save locaiton passed
if( !empty($save_location) ){
file_put_contents($save_location, $sql_data);
} else {
echo $sql . "\n\n";
}
echo "\n\nDone\n\n";
<?php // php csv_parser.php <table name> <csv target> <sql target>
ini_set('memory_limit', '8000M');
ini_set('max_execution_time', 3000);
if (empty($argc)) {
echo "Arguments missing";
exit(1);
}
if (!strstr($argv[0], basename(__FILE__))) {
echo "Permission denied";
exit(1);
}
$table_name = $argv[1];
$file = $argv[2];
$save_location = $argv[3];
$info = pathinfo($file);
$fp = @fopen($file, "r");
$line = fgets($fp, 4096); // read the first line, headers
$fields = explode(",", trim($line));
$sql_data = "";
$i=0;
$insert_values = array(); // will save all insert rows here
while (!feof($fp)) { // loop till end of file.
$line = fgets($fp, 4096); // read a line.
if (empty($line)) continue;
$field_values = array(); // will save all field values
$values = explode(",", trim($line));
// only to sanitize the values
foreach($values as $value){ // fetch all value
$field_values[] = "'" . addslashes(trim($value)) . "'"; // escape special characters
}
$insert_values[] = "(" . implode(",", $field_values) . ")";
}
$insert_values_chunks = array_chunk($insert_values, 30); // insert 30 rows at a time
foreach($insert_values_chunks as $insert_values){
$sql = "INSERT INTO " . $table_name ." (" . implode(",", $fields). ") VALUES \n";
$sql .= implode(",\n", $insert_values) . ";\n\n";
$sql_data .= $sql;
}
// if save locaiton passed
if( !empty($save_location) ){
file_put_contents($save_location, $sql_data);
} else {
echo $sql . "\n\n";
}
echo "\n\nDone\n\n";
@BaraPellumb
Copy link

good stuff but i am doing some csv file parsing using flat files only(no db).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment