运维开发网
广告位招商联系QQ:123077622
 
广告位招商联系QQ:123077622

通过docker-compose快速搭建mongodb副本集

运维开发网 https://www.qedev.com 2021-05-08 10:44 出处:51CTO 作者:宏睿时空
通过docker-compose快速搭建mongodb副本集,wx608b59dedd9e5的博客原创的Docker文章。

转  https://monkeywie.github.io/2020/03/10/mongodb-replica-set/?utm_source=tuicool&utm_medium=referral

前言

mongoDB 提供的副本集是将数据同步至多个节点,提供了数据冗余备份和节点故障的情况下可以自动转移的高可用特性,架构图如下:

有时候需要要在本地搭建一套 mongoDB 副本集环境来做测试,如果用虚拟机的话还是比较麻烦的,这里记录下如何用 docker-compose 快速搭建 mongoDB 副本集。

构建镜像

  1. 准备秘钥用于节点之间的认证
1
openssl rand -base64 756 > auth.key
  1. 编写 Dockerfile

调整 mongo 官方提供的镜像,像里面添加秘钥文件

1
2
3
4
5
6
FROM mongo:3.4.10
#将秘钥文件复制到镜像中
COPY auth.key /app/auth.key
RUN chown -R mongodb:mongodb /app/auth.key
#设置秘钥文件权限,这一步非常关键
RUN chmod 600 /app/auth.key
  1. 构建镜像
1
docker build -t mongo-replset .
  1. 编写 docker-compose

开启三个容器,组成一主两从的副本集集群,另外再开启一个容器监听和初始化副本集,这里我的实现方式并不优雅,就是一直重试直到前面三个 mongod 进程启动完成(官方有个解决方案来检测依赖服务的启动,不过有点麻烦这里就略过了),启动完成之后执行副本集初始化命令:mongo mongodb://root:[email protected]:27011/admin --eval 'rs.initiate({ _id: "rs", members: [{_id:1,host:"mongo-1:27011"},{_id:2,host:"mongo-2:27012"},{_id:3,host:"mongo-3:27013"}]})'

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
version: "3.1"
services:
  mongo-1:
    image: mongo-replset
    hostname: mongo-1
    restart: always
    ports:
      - 27011:27011
    environment:
      MONGO_INITDB_ROOT_USERNAME: root
      MONGO_INITDB_ROOT_PASSWORD: 123
    command:
      - --port
      - "27011"
      - --replSet
      - rs
      - --keyFile
      - /app/auth.key
  mongo-2:
    image: mongo-replset
    hostname: mongo-2
    restart: always
    ports:
      - 27012:27012
    command:
      - --port
      - "27012"
      - --replSet
      - rs
      - --keyFile
      - /app/auth.key
  mongo-3:
    image: mongo-replset
    hostname: mongo-3
    restart: always
    ports:
      - 27013:27013
    command:
      - --port
      - "27013"
      - --replSet
      - rs
      - --keyFile
      - /app/auth.key
  mongo-init:
    image: mongo:3.4.10
    depends_on:
      - mongo-1
      - mongo-2
      - mongo-3
    restart: on-failure:5
    command:
      - mongo
      - mongodb://root:[email protected]:27011/admin
      - --eval
      - 'rs.initiate({ _id: "rs", members: [{_id:1,host:"mongo-1:27011"},{_id:2,host:"mongo-2:27012"},{_id:3,host:"mongo-3:27013"}]})'
  1. 运行
1
2
3
4
5
docker-compose up -d
Creating mongo-replica_mongo-1_1 ... done
Creating mongo-replica_mongo-3_1 ... done
Creating mongo-replica_mongo-2_1 ... done
Creating mongo-replica_mongo-init_1 ... done
  1. 查看副本集状态
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
docker-compose exec mongo-1 mongo mongodb://root:[email protected]:27011/admin --eval 'rs.status()'
MongoDB shell version v3.4.10
connecting to: mongodb://root:[email protected]:27011/admin
MongoDB server version: 3.4.10
{
        "set" : "rs",
        "date" : ISODate("2020-03-10T07:01:59.764Z"),
        "myState" : 1,
        "term" : NumberLong(1),
        "heartbeatIntervalMillis" : NumberLong(2000),
        "optimes" : {
                "lastCommittedOpTime" : {
                        "ts" : Timestamp(1583823710, 1),
                        "t" : NumberLong(1)
                },
                "appliedOpTime" : {
                        "ts" : Timestamp(1583823710, 1),
                        "t" : NumberLong(1)
                },
                "durableOpTime" : {
                        "ts" : Timestamp(1583823710, 1),
                        "t" : NumberLong(1)
                }
        },
        "members" : [
                {
                        "_id" : 1,
                        "name" : "mongo-1:27011",
                        "health" : 1,
                        "state" : 1,
                        "stateStr" : "PRIMARY",
                        "uptime" : 263,
                        "optime" : {
                                "ts" : Timestamp(1583823710, 1),
                                "t" : NumberLong(1)
                        },
                        "optimeDate" : ISODate("2020-03-10T07:01:50Z"),
                        "electionTime" : Timestamp(1583823469, 1),
                        "electionDate" : ISODate("2020-03-10T06:57:49Z"),
                        "configVersion" : 1,
                        "self" : true
                },
                {
                        "_id" : 2,
                        "name" : "mongo-2:27012",
                        "health" : 1,
                        "state" : 2,
                        "stateStr" : "SECONDARY",
                        "uptime" : 261,
                        "optime" : {
                                "ts" : Timestamp(1583823710, 1),
                                "t" : NumberLong(1)
                        },
                        "optimeDurable" : {
                                "ts" : Timestamp(1583823710, 1),
                                "t" : NumberLong(1)
                        },
                        "optimeDate" : ISODate("2020-03-10T07:01:50Z"),
                        "optimeDurableDate" : ISODate("2020-03-10T07:01:50Z"),
                        "lastHeartbeat" : ISODate("2020-03-10T07:01:59.120Z"),
                        "lastHeartbeatRecv" : ISODate("2020-03-10T07:01:58.109Z"),
                        "pingMs" : NumberLong(0),
                        "syncingTo" : "mongo-1:27011",
                        "configVersion" : 1
                },
                {
                        "_id" : 3,
                        "name" : "mongo-3:27013",
                        "health" : 1,
                        "state" : 2,
                        "stateStr" : "SECONDARY",
                        "uptime" : 261,
                        "optime" : {
                                "ts" : Timestamp(1583823710, 1),
                                "t" : NumberLong(1)
                        },
                        "optimeDurable" : {
                                "ts" : Timestamp(1583823710, 1),
                                "t" : NumberLong(1)
                        },
                        "optimeDate" : ISODate("2020-03-10T07:01:50Z"),
                        "optimeDurableDate" : ISODate("2020-03-10T07:01:50Z"),
                        "lastHeartbeat" : ISODate("2020-03-10T07:01:59.120Z"),
                        "lastHeartbeatRecv" : ISODate("2020-03-10T07:01:58.103Z"),
                        "pingMs" : NumberLong(0),
                        "syncingTo" : "mongo-1:27011",
                        "configVersion" : 1
                }
        ],
        "ok" : 1
}

 

增加节点

在新的服务器上部署并启动容器,部署的时候需要将mongodb.key和.env文件一起复制过去,端口根据实际情况修改映射

docker-compsoe.yaml文件

version: '3.1'

services:

  mongodb1:

    image: mongo

    restart: always

    container_name: mongo4

    volumes:

      - ./data/db/mongo4:/data/db

      - ./mongodb.key:/data/mongodb.key

    ports:

      - 27020:27017

    environment:

      MONGO_INITDB_ROOT_USERNAME: ${MONGO_INITDB_ROOT_USERNAME}

      MONGO_INITDB_ROOT_PASSWORD: ${MONGO_INITDB_ROOT_PASSWORD}

    command: mongod --replSet mongos --keyFile /data/mongodb.key

    entrypoint:

      - bash

      - -c

      - |

        chmod 400 /data/mongodb.key

        chown 999:999 /data/mongodb.key

        exec docker-entrypoint.sh $$@

在Primary节点数据库中,进入shell添加新创建的容器,服务器IP和端口

mongos:PRIMARY> rs.add("192.168.1.111:27020")

原文链接:https://blog.csdn.net/biao0309/java/article/details/87641272

 

python读取mongo读写分离

https://api.mongodb.com/python/current/examples/high_availability.html

>>> from pymongo import ReadPreference

>>> client.read_preference

SecondaryPreferred(tag_sets=None)

>>> db = client.get_database('test', read_preference=ReadPreference.SECONDARY)

>>> db.read_preference

Secondary(tag_sets=None)

>>> coll = db.get_collection('test', read_preference=ReadPreference.PRIMARY)

>>> coll.read_preference

Primary()

 

利用python测试mongodb副本集数据同步延迟

转  https://blog.51cto.com/hnr520/1874506

本文主要介绍python如何连接mongodb副本集及读写分离配置,mongodb副本集数据同步延迟测试。

一、python连接mongodb副本集

1.连接副本集

在新版的驱动中直接使用MongoClient连接

如下:

from pymongo import MongoClient

conn = MongoClient(['192.168.3.11:27017', '192.168.3.12:27017', '192.168.3.13:27017'])

2.读写分离配置

from pymongo import ReadPreference

db = conn.get_database('hnrtest', read_preference=ReadPreference.SECONDARY_PREFERRED)

副本集ReadPreference有5个选项:

PRIMARY:默认选项,从primary节点读取数据

PRIMARY_PREFERRED:优先从primary节点读取,如果没有primary节点,则从集群中可用的secondary节点读取

SECONDARY:从secondary节点读取数据

SECONDARY_PREFERRED:优先从secondary节点读取,如果没有可用的secondary节点,则从primary节点读取

NEAREST:从集群中可用的节点读取数据

 

二、测试mongodb延迟脚本

1.原理

往集群中插入一批数据,然后从secondary节点读取,直到读到数据总数与插入的总数相等。

2.脚本内容

#!/usr/bin/python

# -*- coding: utf-8 -*-

import pymongo

from pymongo import MongoClient

from pymongo import ReadPreference

import time

import argparse

# 定义执行时长装饰器函数

def warps(*args):

    def deco(func):

        def _deco(*args, **kwargs):

            # 记录开始时间

            start = time.clock()

            # 回调原函数

            func(*args, **kwargs)

            # 记录结束时间

            end = time.clock()

            # 计算执行时长

            delat = end - start

            # 转换成ms输出

            print("delay:%sms" % (int(delat*1000)))

        return _deco

    return deco

# 连接副本集

conn = MongoClient(['192.168.3.11:27017', '192.168.3.12:27017', '192.168.3.13:27017'])

# 读写分离

db = conn.get_database('hnrtest', read_preference=ReadPreference.SECONDARY_PREFERRED)

# 定义连接的集合

collection = db.student

# 创建插入数据函数

def data_insert(num):

    try:

        for i in range(1, num):

            collection.insert({"name": "student"+str(i), "age": (i % 100), "city": "FuZhou"})

    except Exception as e:

        print("insert data:", e)

# 创建查询数据函数,引用装饰器函数

@warps()

def data_select(num):

    try:

        count = collection.find().count()

        while count != num - 1:

            count = collection.find().count()

    except Exception as e:

        print("select data:", e)

# 创建删除数据函数

def data_delete():

    try:

        collection.remove({})

    except Exception as e:

        print("delete data:", e)

# 创建计算延迟时长函数

def data_delay(num):

    data_insert(num)

    data_select(num)

if __name__ == '__main__':

    # 定义脚本需要传入插入的数据量,默认值为1000,通过-n传入参数

    parser = argparse.ArgumentParser(description='insert data to mongodb number')

    parser.add_argument('-n', action='store', dest='num', type=int, required=False, default=1000)

    given_args = parser.parse_args()

    num = given_args.num

    data_delete()

    data_delay(num)

 

后记

使用 docker-compose 搭建的 mongodb 副本集就完成了,以后可以基于此快速搭建一套 mongodb 副本集环境进行测试。

扫码领视频副本.gif

0

精彩评论

暂无评论...
验证码 换一张
取 消

关注公众号