Skip to content

Instantly share code, notes, and snippets.

@jhjguxin

jhjguxin/mongodb_replication_master.markdown

Last active March 14, 2022 08:19
Show Gist options
  • Star 1 You must be signed in to star a gist
  • Fork 2 You must be signed in to fork a gist
  • Save jhjguxin/5682985 to your computer and use it in GitHub Desktop.
Save jhjguxin/5682985 to your computer and use it in GitHub Desktop.
Download ZIP
how switch master/slave between replSet model on mongodb some prepare for migrate database from grandcloud to aliyun clound
Raw
mongodb_replication_master.markdown

how switch master/slave between replSet model on mongodb

http://www.cnblogs.com/shihao/archive/2013/01/25/2876727.html

assume

  • Physical machine 192.168.10.168 ubuntu 13.04

    • alway as master db

  • virtual machine 192.168.10.167 ubuntu 13.04

    • first slave from 192.168.10.168
    • switch to replication model(between 192.168.10.180)

  • virtual machine 192.168.10.180 centos 6.4

    • first slave from 192.168.10.168
    • switch to replication model(between 192.168.10.167)

  • can revert 192.168.10.167 and 192.168.10.180 to 92.168.10.168's slave machine

  • db version v2.4.3

some code snippet and resources

sudo apt-get update && sudo apt-get upgrade
sudo apt-get install mongodb-server mongodb mongodb-clients
sudo apt-get install openssh-server openssh-client

check what is wrong ?
sudo -u mongodb mongod -f /etc/mongodb.conf

yum list  installed

centos

enable centos-gnoem-desktop

Note: You can switch from GUI to CLI mode manually by using following method:

GUI to CLI: Ctrl + Alt + F6 CLI to GUI: Ctrl + Alt + F1

If you want to start the desktop from CLI console (not SSH session), use following command:

$ startx

change yum sources

http://www.centos.org/docs/5/html/yum/sn-yum-maintenance.html http://kerry.blog.51cto.com/172631/392737

http://mirrors.163.com/ http://mirrors.sohu.com http://centos.ustc.edu.cn/

http://mirrors.163.com/.help/centos.html

eg

cd /etc/yum.repos.d
mv CentOS-Base.repo  CentOS-Base.repo.bak
wget http://centos.ustc.edu.cn/CentOS-Base.repo.5
mv CentOS-Base.repo.5 CentOS-Base.repo

install ssh with centos

yum install openssh.i686 openssh-clients.i686 openssh-server.i686

enable sudo with centos

yum install sudo.i386

start sshd

/etc/init.d/sshd start

add an user named guanxi

ssh root@xxx.xxx.xxx.xxx
man useradd
useradd -d /home/guanxi -s /bin/bash -G adm -m guanxi
passwd guanxi
exit

check if guanxi user is ok?

ssh guanxi@xxx.xxx.xxx.xxx

enable guanxi use sudo

http://www.linuxhelp.net/guides/sudo/

login as 'guanxi'

sudo -l
#[sudo] password for guanxi:
#Sorry, user guanxi may not run sudo on localhost
su
visudo  # insert 'guanxi  ALL=(ALL) ALL'
sudo uname
#Linux

list user group

# list all group on system
awk -F":" '{ print "groupname: " $1 "\tgroupid: " $3 "\tsupplementarygroup: " $4 }' /etc/group
# list current owner groups
groups # guanxi adm

Import and Export MongoDB Data

# sudo ufw allow 27017
# /etc/mongodb.conf
# commit as
# bind_ip = 127.0.0.1

http://docs.mongodb.org/manual/core/import-export/ http://blog.csdn.net/liuzhoulong/article/details/6849978

# back
mongodump -h 192.168.10.168 -o all_test_db_dump
# restore
mongorestore all_test_db_dump/

install mongodb on centos

http://docs.mongodb.org/manual/tutorial/install-mongodb-on-red-hat-centos-or-fedora-linux/

Configure Package Management System (YUM)¶ Create a /etc/yum.repos.d/10gen.repo file to hold information about your repository. If you are running a 64-bit system (recommended,) place the following configuration in /etc/yum.repos.d/10gen.repo file:

[10gen]
name=10gen Repository
baseurl=http://downloads-distro.mongodb.org/repo/redhat/os/x86_64
gpgcheck=0
enabled=1

If you are running a 32-bit system, which isn’t recommended for production deployments, place the following configuration in /etc/yum.repos.d/10gen.repo file:

[10gen]
name=10gen Repository
baseurl=http://downloads-distro.mongodb.org/repo/redhat/os/i686
gpgcheck=0
enabled=1

Install Packages Issue the following command (as root or with sudo) to install the latest stable version of MongoDB and the associated tools:

yum install mongo-10gen mongo-10gen-server

When this command completes, you have successfully installed MongoDB!

Manage Installed Versions You can use the mongo-10gen and mongo-10gen-server packages to install previous releases of MongoDB. To install a specific release, append the version number, as in the following example:

yum install mongo-10gen-2.2.3 mongo-10gen-server-2.2.3

This installs the mongo-10gen and mongo-10gen-server packages with the 2.2.3 release. You can specify any available version of MongoDB; however yum will upgrade the mongo-10gen and mongo-10gen-server packages when a newer version becomes available. Use the following pinning procedure to prevent unintended upgrades.

To pin a package, add the following line to your /etc/yum.conf file:

exclude=mongo-10gen,mongo-10gen-server

enable auth

use admin
db.addUser('guanxi_db','guanxidb')
db.auth('guanxi_db','guanxidb')

# /etc/mongodb.config
# modify bellow as
# Turn on/off security.  Off is currently the default
#noauth = true
auth = true

sudo restart mongodb

# test
mongo 192.168.10.168:27017/admin -u guanxi_db -p guanxidb

start an master mongodb server

# start an mongo directory
# mkdir /data/mongo
# chown mongod:mongod /data/mongo/

# /etc/mongodb.config
master = true

# mongo shell
use local
db.addUser('guanxi_db','guanxidb')
db.auth('guanxi_db','guanxidb')

start an slave mongodb server

# mongodb shell
use local
db.sources.find()
db.sources.insert( { host: "192.168.10.168:27017"  } );
db.addUser('guanxi_db','guanxidb')
db.auth('guanxi_db','guanxidb')


# /etc/mongodb.config
auth  = true
slave = true

# restart db
sudo restart mongodb

alway show

replauthenticate: no user in local.system.users to use for authentication

try key file auth

openssl rand -base64 753 > guanxidb.pub
sudo cp guanxidb.pub /var/lib/mongodb/
#/var/lib/mongodb/guanxidb.pub
#sudo chmod 600 /var/lib/mongodb/guanxidb.pub
# permissions on /var/lib/mongodb/guanxidb.pub are too open if use blleow
# sudo chown mongodb /var/lib/mongodb/guanxidb.pub
#/etc/mongodb.config
keyFile=/var/lib/mongodb/guanxidb.pub

share with other db server

scp guanxidb.pub test1@192.168.10.167:~/
ssh test1@192.168.10.167
sudo cp guanxidb.pub /var/lib/mongodb/
#/var/lib/mongodb/guanxidb.pub
# permissions on /var/lib/mongodb/guanxidb.pub are too open if only use blleow
sudo chown mongodb /var/lib/mongodb/guanxidb.pub
sudo chmod 600 /var/lib/mongodb/guanxidb.pub
#/etc/mongodb.config
keyFile=/var/lib/mongodb/guanxidb.pub

start slave db server on centos

# mongodb shell
use local
db.sources.find()
db.sources.insert( { host: "192.168.10.168:27017"  } );
db.addUser('guanxi_db','guanxidb')
db.auth('guanxi_db','guanxidb')


# /etc/mongodb.config
auth  = true
slave = true

# use keyFile
scp guanxidb.pub test1@192.168.10.180:~/
ssh test1@192.168.10.180
sudo cp guanxidb.pub /var/lib/mongo/
#/var/lib/mongo/guanxidb.pub
sudo chmod 600 /var/lib/mongo/guanxidb.pub
sudo chown mongod /var/lib/mongo/guanxidb.pub
#/etc/mongodb.config
keyFile=/var/lib/mongo/guanxidb.pub

# restart db
sudo /etc/init.d/mongod restart

can switch two slave to replication model ? (yes it work!)

between 192.168.10.167(slave ubuntu) 192.168.10.180(slave centos)

# /etc/mongodb.config
# comment bellow
# slave = true
# add replSet
replSet = rs0

drop local database

mongo admin -u guanxi_db -p guanxidb

use local
db.dropDatabase()
use local
# start with replication set
rsconf = {
           _id: "rs0",
           members: [
	                      {
		                      "_id" : 1,
		                      "host" : "192.168.10.167:27017"
	                      }
                    ]
         }

rs.initiate( rsconf )

192.168.10.180(slave centos) database

# /etc/mongodb.config
# comment bellow
# slave = true
# add replSet
replSet = rs0

drop local database

mongo admin -u guanxi_db -p guanxidb

use local
db.dropDatabase()
use local
# start with replication set

on 192.168.10.167 add 192.168.10.180 as member http://snoopyxdy.blog.163.com/blog/static/60117440201241694254441/

_id：副本集的名字，必须和命令行的名字匹配，也就是您刚才启动mongodb数据库命令行的那个名字，数字字母，不能包含"/";
members：一个数组用来表示副本集中的每个成员，这个数组必须包含_id和host这2个key。

members 数组：
_id：在副本集中的每一个成员都必须有一个_id表示，这个_id是通常是数字，从0开始增长。需要注意的是当其中一个成员退休了（指从副本集config中移除了），新加入的成员不能重新使用这个退休成员的_id；
host：ip地址和端口号；
arbiterOnly（false）：如果是true，则表示这个成员为仲裁节点，不接收数据；
buildIndexes（true）：如果设置为false，则会阻止在这个节点上创建第二索引，通常这个节点是作为纯粹的数据备份，从不用来被查询。不过也因为此节点没有第二索引，所以他写入的东西很少，也就需要很少的内存和磁盘。_id的索引还是会被创建的。只有当priority属性设置为0时，此项才能设置为false，一般不会用到这个选项；
hidden（false）：如果此项为true，不要告诉客户端的此节点的存在，设置隐藏节点的原因是此节点的数据的使用模式和其他节点大为不同，比如：报表，统计，备份等。设置为ture时，允许你针对这个节点发送非主要查询。
priority（1.0）：权重，更高的权重会被选举为主节点
tags（｛｝）：一个文档代表这台服务器的位置，有利于位置感知的读写。其实就是表示此节点位于哪个数据中心的，mongodb会根据tags找近的数据中心节点同步数据。
slaveDelay（0）：同步数据的延迟，设置为0表示立即更新同步数据。
votes（1）：此节点可以发出的投票数，一般不用修改他

settings 对象：settings对象可以在集群建立起来以后用再进行设置，通常使用默认值

add member

mongo 192.168.10.167/admin -u guanxi_db -p guanxidb
rs.add("192.168.10.180:27017")
//rs.add({"_id" : num, "host" : hostname, "priority" : 0, "hidden" : true})

//if show  `replSet we are ahead of the sync source, will try to rool back`
//use local
//db.oplog.rs.drop()
//sudo /etc/init.d/mongod restart

For the current session, this command permits read operations from non-master (i.e. slave or secondary) instances. Practically, use this method in the following form:

db.getMongo().setSlaveOk()

Indicates that “eventually consistent” read operations are acceptable for the current application. This function provides the same functionality as rs.slaveOk().

http://docs.mongodb.org/manual/tutorial/force-member-to-be-primary/

can replication model server switch two slave ? (yes it work!)

backup local ?? can work will??(recommended)

drop(rename) local database, redeine slaver node ??

# disable replSet mode
# /etc/mongo.config

mongo 192.168.10.167:27017/admin -u guanxi_db -p guanxidb
use local

show collections
db.copyDatabase("local","local_bak","localhost")
use local
db.dropDatabase();

# enable slave mode
# /etc/mongo.config
slave = true

# add sources
db.sources.find()
db.sources.insert( { host: "192.168.10.168:27017"  } );
db.addUser('guanxi_db','guanxidb')
db.auth('guanxi_db','guanxidb')

issue

initial sync need a member to be primary or secondary to do our initial sync

replica sets的节点数最小为2（primary和secondary）,primary节点down了，secondary节点自动变为primary节点。这个观点绝对错误，实验n次得出的结论是：如果一个replica sets只存在两个节点，那么primary down了，从节点不会升为primary节点，这时从节点在日志中会显示:自己不能选自己为主节点，不能够为自己投票。 为什么会这样呢？这和投票机制有关，在replica sets中，一个节点只能将自己手中的票投给其他节点，得到票数最多的节点成为primary节点。所以，replica sets节点数最好是奇数。

You mentioned that you run rs.initiate() on both servers. This should only be done on one.

I suggest you start from scratch, by deleting the dbpath directory for each node (backup data before if the db was not empty). Then, start all mongod processes, log into one of them, then call

rs.initiate()
rs.add(<other node 1>)

The other node gets the replica set configuration through the first one automatically. Repeat `rs.add() for each additional node you want to add.

对于生产环境，复制集最少三个节点,2个节点+一个仲裁节点

在一个复制集中：

  • 如果工作节点的个数是偶数个，那个应该添加一个仲裁服务
  • 仲裁服务是一个轻量级的进程，用来参与PRIMARY节点的投票
  • 仲裁服务没有必要放在一个单独的节点上

http://cn.docs.mongodb.org/manual/reference/replica-configuration/ use local

// add new node and 数据差异很大应该隐藏节点
config = {_id: 'rs0', members: [
                          {_id: 0, host: 'xx.xxx.xx.88:27017'},
                          {_id: 1, host: 'xx.xxx.xx.164:27017', "priority" : 0, "hidden" : true},
                          {_id: 2, host: 'xx.xxx.xx.122:27017', arbiterOnly: true}]
           }
rs.initiate(config);

// 如果数据基本一致则可以 初始化 或者 重新初始化
config = {_id: 'rs0', members: [
                          {_id: 0, host: 'xx.xxx.xx.88:27017'},
                          {_id: 1, host: 'xx.xxx.xx.164:27017'},
                          {_id: 2, host: 'xx.xxx.xx.122:27017', arbiterOnly: true}]
           }
rs.reconfig(config);

如果总是提示权限错误尝试

mongod -f /etc/mongodb.config #--repair

Three Member Sets

The minimum recommended architecture for a replica set consists of:

One primary and Two secondary members, either of which can become the primary at any time. This makes failover possible and ensures there exists two full and independent copies of the data set at all times. If the primary fails, the replica set elects another member as primary and continues replication until the primary recovers.

Arbiters

Deploy an arbiter to ensure that a replica set will have a sufficient number of members to elect a primary. While having replica sets with 2 members is not recommended for production environments, if you have just two members, deploy an arbiter. Also, for any replica set with an even number of members, deploy an arbiter.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment