Skip to content

Instantly share code, notes, and snippets.

@davimacedo davimacedo/curl.sh
Last active Jan 27, 2020

Embed
What would you like to do?
Example of importing data with cloud functions. See a live example at https://www.back4app.com/database/davimacedo/parse-import-example
curl -X POST \
-H "X-Parse-Application-Id: LL9oIdzIkmwl5xyowQQu0fTmXyUWfet9RuAzwHfj" \
-H "X-Parse-REST-API-Key: R3S8PYQKuzeV4c8MUeO5ved46C50MEp56boDHW1O" \
-H "Content-Type: application/json" \
-d @data.json \
https://parseapi.back4app.com/functions/import
{
"className": "ExampleClass",
"rows": [
{ "ExampleColumnA": "row1columnA", "ExampleColumnB": "row1columnB" },
{ "ExampleColumnA": "row2columnA", "ExampleColumnB": "row2columnB"}
]
}
Parse.Cloud.define('import', async request => {
const className = request.params.className;
const rows = request.params.rows;
const MyClass = Parse.Object.extend(className);
var myClassObjects = [];
for (let i = 0; i < rows.length; i++) {
const myClassObject = new MyClass();
for (const column in rows[i]) {
myClassObject.set(column, rows[i][column]);
}
myClassObjects.push(myClassObject);
}
try {
await Parse.Object.saveAll(myClassObjects);
} catch (e) {
throw new Error(`Import failed: ${e}`);
}
return `Successfully imported ${myClassObjects.length} rows into ${className} class`;
});
@djarunn

This comment has been minimized.

Copy link

djarunn commented Jul 15, 2016

I just exported the data using your curl script and it exported the data correctly.But when I tried to import the same Json exported lately((by adding class info and changing result to rows), I found -

  1. If data is already existing in the row , unique ID passed will help to identify that row and will update the data.
    2)If the row is accidentally removed ( we don't have any row with 'UniqueID' being passed) , no data will be imported for that row.

So , this Import works well , if we want to update the existing data ,but for the cases when we lose data (most obvious reason for export/import) , this import functionality won't help , as it cannot add new rows with defined 'ObjectID'.

Is there anyway to fix this issue ( create new rows of data with object ID passed) ?

@davimacedo

This comment has been minimized.

Copy link
Owner Author

davimacedo commented Jul 25, 2016

This code example is only able to include new rows. This script would not be able to update rows. As I understood you are trying to upload data with ObejctId and that would require script to update this data. That's why it is failing. You will have to change script little bit to make it update.

@archiejudd

This comment has been minimized.

Copy link

archiejudd commented Aug 18, 2016

Hi Davi - this works great, except after importing a json file with 8 columns, 3 of them are missing at the parse dashboard. Any ideas on why this might be the case?

@archiejudd

This comment has been minimized.

Copy link

archiejudd commented Aug 18, 2016

I have now worked out my mistake - I was using spaces and dashes in the column names! Thanks

@mpc20001

This comment has been minimized.

Copy link

mpc20001 commented Aug 24, 2016

I keep getting this error - {"code":141,"error":"Import failed: undefined"} after trying to import through curl.

@mpc20001

This comment has been minimized.

Copy link

mpc20001 commented Aug 24, 2016

nevermind, i just wasn't using my master key. It works great.

@mpc20001

This comment has been minimized.

Copy link

mpc20001 commented Aug 24, 2016

Actually I tried importing a 1mb file of about 10,000 rows and the connection reset. It worked for much smaller amounts though. What is the size limit for imports through this method?

@davimacedo

This comment has been minimized.

Copy link
Owner Author

davimacedo commented Sep 7, 2016

There is no specific limit, but 10,000 would reach your api 10,000 times. You would have to modify little bit the code to make it upload something like 1000 each minute.

@Mamonaku

This comment has been minimized.

Copy link

Mamonaku commented Jan 6, 2017

@mpc20001 keep getting the same error, while using the master key
promises.push(myClassObject.save({useMasterKey:true}));

I have client class creation set, even trying on small file..... not sure what is blocking

Log files say
Fri Jan 06 2017 12:31:53 GMT+0000 (UTC) Error: Can't set headers after they are sent. at ServerResponse.OutgoingMessage.setHeader (_http_outgoing.js:344:11) at ServerResponse.header (/usr/src/app/node_modules/express/lib/response.js:719:10) at ServerResponse.send (/usr/src/app/node_modules/express/lib/response.js:164:12) at ServerResponse.json (/usr/src/app/node_modules/express/lib/response.js:250:15) at /usr/src/app/src/back/app.js:104:9 at Layer.handle_error (/usr/src/app/node_modules/express/lib/router/layer.js:71:5) at trim_prefix (/usr/src/app/node_modules/express/lib/router/index.js:310:13) at /usr/src/app/node_modules/express/lib/router/index.js:280:7 at Function.process_params (/usr/src/app/node_modules/express/lib/router/index.js:330:12) at next (/usr/src/app/node_modules/express/lib/router/index.js:271:10) at Layer.handle_error (/usr/src/app/node_modules/express/lib/router/layer.js:67:12) at trim_prefix (/usr/src/app/node_modules/express/lib/router/index.js:310:13) at /usr/src/app/node_modules/express/lib/router/index.js:280:7 at Function.process_params (/usr/src/app/node_modules/express/lib/router/index.js:330:12) at next (/usr/src/app/node_modules/express/lib/router/index.js:271:10) at Immediate.<anonymous> (/usr/src/app/node_modules/express/lib/application.js:232:9)

@Mamonaku

This comment has been minimized.

Copy link

Mamonaku commented Jan 6, 2017

Seems there is a problem with the promise fulfilment. I changed the code for
Parse.Object.saveAll(myObjectArray).
Unfortunately, the code can save the entries as new objects, but can not migrate data... in other words, attempts to set the object ID results in
{"code":141,"error":"Import failed: Object not found."}

Any ideas?

@Treverr

This comment has been minimized.

Copy link

Treverr commented Apr 6, 2017

This does not work, I get "unexpected token /" or c or t

@nyxee

This comment has been minimized.

Copy link

nyxee commented Jun 18, 2017

I can't get any of this to work either..
I put main.js in my Parse sever directoy and called http://localhost:1337/parse/main.js "returns :error, unouthorize"

@aacassandra

This comment has been minimized.

Copy link

aacassandra commented Jul 17, 2019

hi Davi, this works great, you save my day. @nyxee im tested on localhost and works. your url change to http://localhost:1337/parse/function/{name}

@davimacedo

This comment has been minimized.

Copy link
Owner Author

davimacedo commented Jul 17, 2019

Glad to know!

@milad-abbasi

This comment has been minimized.

Copy link

milad-abbasi commented Dec 28, 2019

Is there a better way of importing data to parse server?
Imagine thousand of rows to import, I'm worried about those for loops there.

@davimacedo

This comment has been minimized.

Copy link
Owner Author

davimacedo commented Jan 5, 2020

@milad-abbasi I've just updated the example to be compliant with the latest version of Parse and also to use Parse.Object.saveAll(). It conveniently splits the requests in batches so you can make sure it works properly. I also added the link for a living example at the Back4App Database Hub so you can just run and see the imported rows or clone, check the cloud code out and play around.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.