Skip to content

Instantly share code, notes, and snippets.

View L04DB4L4NC3R's full-sized avatar
😏
Noob is just a state of mind

Angad Sharma L04DB4L4NC3R

😏
Noob is just a state of mind
View GitHub Profile
@L04DB4L4NC3R
L04DB4L4NC3R / description.md
Last active September 5, 2019 21:56 — forked from lsm/export_gridfs.sh
Export files from MongoDB GridFS with directory paths

So mongodb stores every document in the form of bson. A bson document has a max limit of 16 MB. Therefore mongodb uses something known as gridFS when saving huge files. It creates fs.chunks for storing base64 files and fs.files to store file metadata.

mongodump creates a dump of the whole DB. Like a bson snapshot. But due to version changes we have to run it in a docker container which has mongoDB 4.0 installed. It shows permission denied while creating a file. So I just exec'ed into the container to run the mongodump command.

docker run --rm -v $(pwd):/workdir/ -w /workdir/ mongo:4.0 mongodump -h server -d $database --out /workdir/dump/
@L04DB4L4NC3R
L04DB4L4NC3R / fs_chunks.json
Last active September 6, 2019 00:09
GridFS collections
{
"_id": {
"$oid": "5d1bb7a420471bc4ba7d502e"
},
"files_id": {
"$oid": "5d1bb79d20471bc4ba7d502d"
},
"n": {
"$numberInt": "0"
},
@L04DB4L4NC3R
L04DB4L4NC3R / get.py
Created September 6, 2019 00:13
get files
from pymongo import MongoClient
import gridfs
import base64
import sys
import bson
import json
import os.path
save_path = "./files"
// npm i axios csv-parser
const axios = require('axios');
const csv = require('csv-parser');
const fs = require('fs');
const authHeader = ''
const instance = axios.create({
baseURL: 'http://<HOST>/api/v1',
@L04DB4L4NC3R
L04DB4L4NC3R / routes.md
Last active October 6, 2019 18:10
REST API route spec

Fetch a resource

GET /api/v1/resource/

Query a resource

@L04DB4L4NC3R
L04DB4L4NC3R / manage.py
Created October 15, 2019 17:47
dotenv setting in django
#!/usr/bin/env python
import os
import sys
import re
def read_env():
"""Pulled from Honcho code with minor updates, reads local default
environment variables from a .env file located in the project root
directory.
@L04DB4L4NC3R
L04DB4L4NC3R / entity.go
Created December 26, 2019 22:09
Clean Arch
package user
import "github.com/jinzhu/gorm"
type User struct {
gorm.Model
FirstName string `json:"first_name,omitempty"`
LastName string `json:"last_name,omitempty"`
Password string `json:"password,omitempty"`
PhoneNumber string `json:"phone_number,omitempty"`
@L04DB4L4NC3R
L04DB4L4NC3R / repository.go
Created December 26, 2019 22:12
Clean Arch
package user
import (
"context"
)
type Repository interface {
FindByID(ctx context.Context, id uint) (*User, error)
BuildProfile(ctx context.Context, user *User) (*User, error)
@L04DB4L4NC3R
L04DB4L4NC3R / service.go
Created December 26, 2019 22:14
Clean Arch
package user
import (
"context"
"crypto/md5"
"encoding/hex"
"errors"
)
type Service interface {
@L04DB4L4NC3R
L04DB4L4NC3R / user.go
Created December 26, 2019 22:16
Clean Arch
package handler
import (
"encoding/json"
"net/http"
"github.com/L04DB4L4NC3R/jobs-mhrd/api/middleware"
"github.com/L04DB4L4NC3R/jobs-mhrd/api/views"
"github.com/L04DB4L4NC3R/jobs-mhrd/pkg/user"
"github.com/dgrijalva/jwt-go"