Created
October 31, 2022 10:05
-
-
Save bottom-up-ai/e411f3f7551261d15a93dbe1f459ed80 to your computer and use it in GitHub Desktop.
NodeJS file upload
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
const formidable = require('formidable'); | |
exports.getMetadata = (req, res, next) => { | |
const form = formidable({ | |
uploadDir: `${__dirname}/../controllers/uploads`, | |
keepExtensions: true, | |
}); | |
const filesInfos = []; | |
form.onPart = (part) => { | |
// I can filter here based on the mimetype but I don't have the information about the file size | |
// The formidable package doesn't include it | |
if (!part.mimetype) return form._handlePart(part); | |
filesInfos.push({ name: part.name, mimetype: part.mimetype }) | |
}; | |
form.parse(req, (err, fields, files) => { | |
// when parse() is called, files & fields are being processed already | |
}); | |
return next(); | |
}; |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
const express = require('express'); | |
const technologyController = require('../controllers/technologyController'); | |
const multipart = require('../utils/multipart'); | |
const validation = require('../utils/validation'); | |
const router = express.Router(); | |
router.route('/') | |
.post( | |
multipart.getMetadata, // get metadata from files & fields on multipart header | |
validation.technology, // validation with express-validator module | |
technologyController.createTechnology, // validate fields + files (size, mimetype) && create a record in database if ok | |
) | |
; | |
module.exports = router; |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
PROBLEM :
Packages (multer, formidable, busboy...) usually store the file directly on the disk or memory with a custom validation in options. My problem is that, when I send fields & files at the same time, I don't want to waste resources on processing the entire file on the disk, or memory or even in a external storage like AWS S3, I want to stop everything and return an error to the client.
I would like the following illustrated in the express route above :
1°) Get the file and fields metadata in a multipart request and attach them to the request object
- file : size + mimetype
- fields : values
2°) Some validation on the properties created from previous step (validation with express-validator)
3°) technologyController.createTechnology which does the following :
- If ok : Process the stream of the file & store it directly into AWS S3 (without it being store in memory or in the local disk)
- If error : Doesn't process the file & don't create a record in the database but return a list of error to client
I hope my requirements are clear enough, if not don't hesitate.
I can't figure out how to do it properly, it's been days.