Skip to content

Instantly share code, notes, and snippets.

@bottom-up-ai
Created October 31, 2022 10:05
Show Gist options
  • Save bottom-up-ai/e411f3f7551261d15a93dbe1f459ed80 to your computer and use it in GitHub Desktop.
Save bottom-up-ai/e411f3f7551261d15a93dbe1f459ed80 to your computer and use it in GitHub Desktop.
NodeJS file upload
const formidable = require('formidable');
exports.getMetadata = (req, res, next) => {
const form = formidable({
uploadDir: `${__dirname}/../controllers/uploads`,
keepExtensions: true,
});
const filesInfos = [];
form.onPart = (part) => {
// I can filter here based on the mimetype but I don't have the information about the file size
// The formidable package doesn't include it
if (!part.mimetype) return form._handlePart(part);
filesInfos.push({ name: part.name, mimetype: part.mimetype })
};
form.parse(req, (err, fields, files) => {
// when parse() is called, files & fields are being processed already
});
return next();
};
const express = require('express');
const technologyController = require('../controllers/technologyController');
const multipart = require('../utils/multipart');
const validation = require('../utils/validation');
const router = express.Router();
router.route('/')
.post(
multipart.getMetadata, // get metadata from files & fields on multipart header
validation.technology, // validation with express-validator module
technologyController.createTechnology, // validate fields + files (size, mimetype) && create a record in database if ok
)
;
module.exports = router;
@bottom-up-ai
Copy link
Author

PROBLEM :
Packages (multer, formidable, busboy...) usually store the file directly on the disk or memory with a custom validation in options. My problem is that, when I send fields & files at the same time, I don't want to waste resources on processing the entire file on the disk, or memory or even in a external storage like AWS S3, I want to stop everything and return an error to the client.

I would like the following illustrated in the express route above :

1°) Get the file and fields metadata in a multipart request and attach them to the request object
- file : size + mimetype
- fields : values

2°) Some validation on the properties created from previous step (validation with express-validator)

3°) technologyController.createTechnology which does the following :
- If ok : Process the stream of the file & store it directly into AWS S3 (without it being store in memory or in the local disk)
- If error : Doesn't process the file & don't create a record in the database but return a list of error to client

I hope my requirements are clear enough, if not don't hesitate.

I can't figure out how to do it properly, it's been days.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment