Skip to content

Instantly share code, notes, and snippets.

@marantz

marantz/MongoDB Settting.md

Last active March 29, 2024 08:03
Show Gist options
  • Save marantz/8a0e7e244e07390b3252 to your computer and use it in GitHub Desktop.
Save marantz/8a0e7e244e07390b3252 to your computer and use it in GitHub Desktop.
Download ZIP
MongoDB Setting Markdown
Raw
MongoDB Settting.md

MongoDB Shard Setting

1.서버 데몬의 구성

본 문서는 기본 샤딩의 개념을 실습하고 학습하는 데에 목표를 두고 3대의 서버를 둔다는 가정하에 한대의 물리적인 서버에 샤딩을 구성하며 Sharding 의 Rule 이라고 볼 수 있는 Collectoin의 Shard Chunk 구성은 하지 않는다.

각 싸이트마다 데이터의 규칙을 살펴보고 Chunk 구성까지 하여야 하며 Autosharding 은 가급적 구성하지 않는 것으로 한다.

  • mongod

Sharding이 되는 기본 데몬, 옵션에 따라 configsvr 또는 shardsvr로 나뉘게 된다.

  • monsos

Sharding 을 관장하는 서버이며 mongod config 서버를 기동시 등록하게 되며 내부적으로 db.command 를 통해 shardsvr를 등록 한다.

2.디렉토리 구성

  • mongo home

mongodb binary 가 설치된 기본 Directory

  • mongo data

mongodb data file 이 만들어 질 Directory

  • mongo log

mongodb log file 이 만들어 질 Directory

  • mongo conf

mongodb configuration file 이 만들어 질 Directory

개발서버 기준

구분 Full Diectory 예제 파일
home /mongodb bin/mongo, bin/mongod
data /dbdata/mongotest data_01/journal 및 Collection 파일들
log /logs mongos.log,mongod_01.log
conf /mongodb/conf shard_mongos.conf,shard_mongod_01.conf

3.Config File Setting

a. mongos.conf

net:
  bindIp: 127.0.0.1,192.168.0.47
  port: 30000
  maxIncomingConnections: 5000
processManagement:
    fork: true
sharding:
    configDB: "localhost:27218,localhost:27228,localhost:27238"
    autoSplit: true
systemLog:
  destination: file
  path: "/logs/mongos.log"
  quiet: true
  logAppend: true
  traceAllExceptions: true

b. mongod.conf

  • Under Bar 뒤에 샤드 번호를 구분해준다.

shard_mongod_01.conf, shard_mongod_02.conf, shard_mongod_03.conf, shard_mongod_04.conf

net:
  bindIp: 127.0.0.1,192.168.0.47
  port: 27118
sharding:
  clusterRole:"shardsvr"
processManagement:
  fork: true
security:
  authorization: disabled
  javascriptEnabled: true
storage:
  dbPath: "/dbdata/mongotest/data_01"
  preallocDataFiles: true
  journal:
    enabled: true
  indexBuildRetry: true
  directoryPerDB: true
/* 2.8
  engine: "wiredTiger"
  wiredTiger:
      engineConfig: "cache_size=8G"
      collectionConfig: "block_compressor=snappy"
*/

systemLog:
  destination: file
  path: "/logs/mongod_01.log"
  quiet: false
  logAppend: true
  traceAllExceptions: true

c. mongod.conf

  • Under Bar 뒤에 configsvr 번호를 구분해준다.

shard_config_01.conf shard_config_02.conf shard_config_03.conf

net:
  bindIp: 127.0.0.1,192.168.0.47
  port: 27218
sharding:
  clusterRole:"configsvr"
processManagement:
  fork: true
security:
  authorization: disabled
  javascriptEnabled: true
storage:
  dbPath: "/dbdata/mongotest/conf_01"
  preallocDataFiles: true
  journal:
    enabled: true
  indexBuildRetry: true
systemLog:
  destination: file
  path: "/logs/mongod_config_01.log"
  quiet: true
  logAppend: true
  traceAllExceptions: true

4.Directory Setting

다음과 같이 디렉토리를 만들어 둔다. server_01 디렉토리는 실제 쓰여지지 않으나 향후 버젼업이 되면서 어떻게 바뀔지 모르니 미리 만들어 두자

  • CONF

/dbdata/mongotest/conf_01 /dbdata/mongotest/conf_02 /dbdata/mongotest/conf_03

  • DATA

/dbdata/mongotest/data_01 /dbdata/mongotest/data_02 /dbdata/mongotest/data_03

  • CLUSTER

/dbdata/mongotest/server_01

5.Server 실행

  • shardsvr
./mongod -f ../conf/shard_mongod_01.conf
./mongod -f ../conf/shard_mongod_02.conf
./mongod -f ../conf/shard_mongod_03.conf
  • configsvr
./mongod -f ../conf/shard_config_01.conf
./mongod -f ../conf/shard_config_02.conf
./mongod -f ../conf/shard_config_03.conf
  • server
./mongos -f ../conf/shard_mongos.conf --fork

6.Server Setting

  • Connect DB
./mongos --port 30000
  • Log Setting
use admin
db.runCommand( { logRotate : 1 } );
  • Add User
use admin
db.createUser(
    {
      user: "root",
      pwd: "rootpasswd",
      roles: [ "root" ]
    }
);
  • Add Shard
show dbs
use admin
show collections

db.runCommand({ addShard: '192.168.0.47:27118' }); // shard0001 | db server db01
db.runCommand({ addShard: '192.168.0.47:27128' }); // shard0002 | db server db02
db.runCommand({ addShard: '192.168.0.47:27138' }); // shard0003 | db server db03
  • Move Data
sh.stopBalancer();
sh.getBalancerState();

db.runCommand({ movePrimary : "DB_00", to : "shard0000" });
db.runCommand({ movePrimary : "DB_01", to : "shard0001" });
db.runCommand({ movePrimary : "DB_02", to : "shard0002" });
sh.setBalancerState(true);

7.Test Monitoring

  • mongos.log로 접속이 제대로 되는 지 확인한다.

  • [DB_00 / DB_01 / DB_02] 등등 각 "/logs/mongod_01~04.log" 로 데이터가 들어오는지 확인한다.

  • 기타 기동 및 중단이 잘 되는 지 확인한다.

START.sh

#!/bin/sh
./mongod -f ../conf/shard_mongod_01.conf --shardsvr
./mongod -f ../conf/shard_mongod_02.conf --shardsvr
./mongod -f ../conf/shard_mongod_03.conf --shardsvr
./mongod -f ../conf/shard_config_01.conf --configsvr
./mongod -f ../conf/shard_config_02.conf --configsvr
./mongod -f ../conf/shard_config_03.conf --configsvr
./mongos -f ../conf/shard_mongos.conf --configdb localhost:27218,localhost:27228,localhost:27238 --fork

STOP.sh

#!/bin/sh
echo "####################################################"
echo "#         Stop MONGODB                             #"
echo "####################################################"
echo `date` MONGODB Stop
SERVER_COUNT=`ps -ef | grep -v grep | grep "./mongo" | awk '{print $2}' | wc -l`
if [[ $SERVER_COUNT -eq 0 ]]; then
    echo MONGODB not found
    exit
fi

for PROCESS in `ps -ef | grep -v grep | grep './mongos -f ../conf/shard_mongos.conf' | awk '{print $2}'`
do
    echo "MONGOS SERVER PROCESS down!!"
    kill -15 $PROCESS
done
sleep 5

for PROCESS in `ps -ef | grep -v grep | grep './mongod -f ../conf/shard_config_' | awk '{print $2}'`
do
    echo "MONGO CONFIG SEREVER PROCESS down!!"
    kill -15 $PROCESS
done
sleep 2

for PROCESS in `ps -ef | grep -v grep | grep './mongod -f ../conf/shard_mongod_' | awk '{print $2}'`
do
    echo "MONGOD DATABASE SERVER PROCESS down!!"
    kill -15 $PROCESS
done
sleep 2

8.Database 와 Collection Shard Setting

  • Database Shard 지정
use DB_00
db.createCollection("test_data");
db.test_data.ensureIndex(  { "SERVICE":1,"TYPE":1 } );

use admin
db.runCommand({enablesharding: "DB_00"});
-- SHARD RESULTS
sh.status()
--- Sharding Status ---
  sharding version: {
    "_id": 1,
    "version": 4,
    "minCompatibleVersion": 4,
    "currentVersion": 5,
    "clusterId": ObjectId("5485236afb1fece5ab640073")
  }
  shards:
    {  "_id": "shard0000",  "host": "localhost:27118" }
    {  "_id": "shard0001",  "host": "localhost:27128" }
    {  "_id": "shard0002",  "host": "localhost:27138" }
  databases:
    {  "_id": "admin",  "partitioned": false,  "primary": "config" }

9.기타 유용한 명령어

  • Seeing Currnet Op
db.currentOp();
  • db lock check
db.locks.find()
db.locks.findOne()
  • Shard status
sh.status()
use config
db.settings.save( { _id:"chunksize", value: <sizeInMB> } )

EOF

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment