当前位置：Gxlcms > mysql > 绝好的MongoDB学习资料之十.Sharding(2)

绝好的MongoDB学习资料之十.Sharding(2)

时间：2021-07-01 10:21:17 帮助过：33人阅读

MongoDB Auto-Sharding 解决了海量存储和动态扩容的问题，但离实际生产环境所需的高可靠(high reliability)、高可用(high availability)还有些距离。 解决方案: Shard: 使用 Replica Sets，确保每个数据节点都具有备份、自动容错转移、自动恢复能力。 Config:

MongoDB Auto-Sharding 解决了海量存储和动态扩容的问题，但离实际生产环境所需的高可靠(high reliability)、高可用(high availability)还有些距离。

解决方案:
  • Shard: 使用 Replica Sets，确保每个数据节点都具有备份、自动容错转移、自动恢复能力。
  • Config: 使用 3 个配置服务器，确保元数据完整性(two-phase commit)。
  • Route: 配合 LVS，实现负载平衡，提高接入性能(high performance)。
以下我们配置一个 Replica Sets + Sharding 测试环境。
装个配置过程建议都用 IP 地址，以免出错。

(1) 首先建好所有的数据库目录。

  1. $ sudo mkdir -p /var/mongodb/10001
  2. $ sudo mkdir -p /var/mongodb/10002
  3. $ sudo mkdir -p /var/mongodb/10003
  4. $ sudo mkdir -p /var/mongodb/10011
  5. $ sudo mkdir -p /var/mongodb/10012
  6. $ sudo mkdir -p /var/mongodb/10013
  7. $ sudo mkdir -p /var/mongodb/config1
  8. $ sudo mkdir -p /var/mongodb/config2
  9. $ sudo mkdir -p /var/mongodb/config3

(2) 配置 Shard Replica Sets。

  1. $ sudo ./mongod --shardsvr --fork --logpath /dev/null --dbpath /var/mongodb/10001 --port 10001 --nohttpinterface --replSet set1
  2. forked process: 4974
  3. all output going to: /dev/null
  4. $ sudo ./mongod --shardsvr --fork --logpath /dev/null --dbpath /var/mongodb/10002 --port 10002 --nohttpinterface --replSet set1
  5. forked process: 4988
  6. all output going to: /dev/null
  7. $ sudo ./mongod --shardsvr --fork --logpath /dev/null --dbpath /var/mongodb/10003 --port 10003 --nohttpinterface --replSet set1
  8. forked process: 5000
  9. all output going to: /dev/null

  1. $ ./mongo --port 10001
  2. MongoDB shell version: 1.6.2
  3. connecting to: 127.0.0.1:10001/test
  4. > cfg = { _id:'set1', members:[
  5. ... { _id:0, host:'192.168.1.202:10001' },
  6. ... { _id:1, host:'192.168.1.202:10002' },
  7. ... { _id:2, host:'192.168.1.202:10003' }
  8. ... ]};
  9. > rs.initiate(cfg)
  10. {
  11.         "info" : "Config now saved locally.  Should come online in about a minute.",
  12.         "ok" : 1
  13. }
  14. > rs.status()
  15. {
  16.         "set" : "set1",
  17.         "date" : "Tue Sep 07 2010 10:25:28 GMT+0800 (CST)",
  18.         "myState" : 5,
  19.         "members" : [
  20.                 {
  21.                         "_id" : 0,
  22.                         "name" : "yuhen-server64:10001",
  23.                         "health" : 1,
  24.                         "state" : 5,
  25.                         "self" : true
  26.                 },
  27.                 {
  28.                         "_id" : 1,
  29.                         "name" : "192.168.1.202:10002",
  30.                         "health" : -1,
  31.                         "state" : 6,
  32.                         "uptime" : 0,
  33.                         "lastHeartbeat" : "Thu Jan 01 1970 08:00:00 GMT+0800 (CST)"
  34.                 },
  35.                 {
  36.                         "_id" : 2,
  37.                         "name" : "192.168.1.202:10003",
  38.                         "health" : -1,
  39.                         "state" : 6,
  40.                         "uptime" : 0,
  41.                         "lastHeartbeat" : "Thu Jan 01 1970 08:00:00 GMT+0800 (CST)"
  42.                 }
  43.         ],
  44.         "ok" : 1
  45. }

配置第二组 Shard Replica Sets。

  1. $ sudo ./mongod --shardsvr --fork --logpath /dev/null --dbpath /var/mongodb/10011 --port 10011 --nohttpinterface --replSet set2
  2. forked process: 5086
  3. all output going to: /dev/null
  4. $ sudo ./mongod --shardsvr --fork --logpath /dev/null --dbpath /var/mongodb/10012 --port 10012 --nohttpinterface --replSet set2
  5. forked process: 5098
  6. all output going to: /dev/null
  7. $ sudo ./mongod --shardsvr --fork --logpath /dev/null --dbpath /var/mongodb/10013 --port 10013 --nohttpinterface --replSet set2
  8. forked process: 5112
  9. all output going to: /dev/null

  1. $ ./mongo --port 10011
  2. MongoDB shell version: 1.6.2
  3. connecting to: 127.0.0.1:10011/test
  4. > cfg = { _id:'set2', members:[
  5. ... { _id:0, host:'192.168.1.202:10011' },
  6. ... { _id:1, host:'192.168.1.202:10012' },
  7. ... { _id:2, host:'192.168.1.202:10013' }
  8. ... ]}
  9. > rs.initiate(cfg)
  10. {
  11.         "info" : "Config now saved locally.  Should come online in about a minute.",
  12.         "ok" : 1
  13. }
  14. > rs.status()
  15. {
  16.         "set" : "set2",
  17.         "date" : "Tue Sep 07 2010 10:28:37 GMT+0800 (CST)",
  18.         "myState" : 1,
  19.         "members" : [
  20.                 {
  21.                         "_id" : 0,
  22.                         "name" : "yuhen-server64:10011",
  23.                         "health" : 1,
  24.                         "state" : 1,
  25.                         "self" : true
  26.                 },
  27.                 {
  28.                         "_id" : 1,
  29.                         "name" : "192.168.1.202:10012",
  30.                         "health" : 0,
  31.                         "state" : 6,
  32.                         "uptime" : 0,
  33.                         "lastHeartbeat" : "Tue Sep 07 2010 10:28:36 GMT+0800 (CST)",
  34.                         "errmsg" : "still initializing"
  35.                 },
  36.                 {
  37.                         "_id" : 2,
  38.                         "name" : "192.168.1.202:10013",
  39.                         "health" : 1,
  40.                         "state" : 5,
  41.                         "uptime" : 1,
  42.                         "lastHeartbeat" : "Tue Sep 07 2010 10:28:36 GMT+0800 (CST)",
  43.                         "errmsg" : "."
  44.                 }
  45.         ],
  46.         "ok" : 1
  47. }

(3) 启动 Config Server。

我们可以只使用 1 个 Config Server，但 3 个理论上更有保障性。

  1. Chunk information is the main data stored by the config servers.  Each config server has a complete copy of all chunk information.  A two-phase
  2. commit is used to ensure the consistency of the configuration data among the config servers.
  3. If any of the config servers is down, the cluster's meta-data goes read only. However, even in such a failure state, the MongoDB cluster can still
  4. be read from and written to.

注意！这个不是 Replica Sets，不需要 --replSet 参数。

  1. $ sudo ./mongod --configsvr --fork --logpath /dev/null --dbpath /var/mongodb/config1 --port 20000 --nohttpinterface
  2. forked process: 5177
  3. all output going to: /dev/null
  4. $ sudo ./mongod --configsvr --fork --logpath /dev/null --dbpath /var/mongodb/config2 --port 20001 --nohttpinterface
  5. forked process: 5186
  6. all output going to: /dev/null
  7. $ sudo ./mongod --configsvr --fork --logpath /dev/null --dbpath /var/mongodb/config3 --port 20002 --nohttpinterface
  8. forked process: 5195
  9. all output going to: /dev/null

  1. $ ps aux | grep configsvr | grep -v grep
  2. root  ./mongod --configsvr --fork --logpath /dev/null --dbpath /var/mongodb/config1 --port 20000 --nohttpinterface
  3. root  ./mongod --configsvr --fork --logpath /dev/null --dbpath /var/mongodb/config2 --port 20001 --nohttpinterface
  4. root  ./mongod --configsvr --fork --logpath /dev/null --dbpath /var/mongodb/config3 --port 20002 --nohttpinterface


(4) 启动 Route Server。

注意 --configdb 参数。

  1. $ sudo ./mongos --fork --logpath /dev/null --configdb "192.168.1.202:20000,192.168.1.202:20001,192.168.1.202:20002"
  2. forked process: 5209
  3. all output going to: /dev/null

  1. $ ps aux | grep mongos | grep -v grep
  2. root ./mongos --fork --logpath /dev/null --configdb 192.168.1.202:20000,192.168.1.202:20001,192.168.1.202:20002

(5) 开始配置 Sharding。

注意 addshard 添加 Replica Sets 的格式。

  1. $ ./mongo
  2. MongoDB shell version: 1.6.2
  3. connecting to: test
  4. > use admin
  5. switched to db admin
  6. > db.runCommand({ addshard:'set1/192.168.1.202:10001,192.168.1.202:10002,192.168.1.202:10003' })
  7. { "shardAdded" : "set1", "ok" : 1 }
  8. > db.runCommand({ addshard:'set2/192.168.1.202:10011,192.168.1.202:10012,192.168.1.202:10013' })
  9. { "shardAdded" : "set2", "ok" : 1 }
  10. > db.runCommand({ enablesharding:'test' })
  11. { "ok" : 1 }
  12. > db.runCommand({ shardcollection:'test.data', key:{_id:1} })
  13. { "collectionsharded" : "test.data", "ok" : 1 }
  14. > db.runCommand({ listshards:1 })
  15. {
  16.         "shards" : [
  17.                 {
  18.                         "_id" : "set1",
  19.                         "host" : "set1/192.168.1.202:10001,192.168.1.202:10002,192.168.1.202:10003"
  20.                 },
  21.                 {
  22.                         "_id" : "set2",
  23.                         "host" : "set2/192.168.1.202:10011,192.168.1.202:10012,192.168.1.202:10013"
  24.                 }
  25.         ],
  26.         "ok" : 1
  27. }
  28. > printShardingStatus()
  29. --- Sharding Status ---
  30.   sharding version: { "_id" : 1, "version" : 3 }
  31.   shards:
  32.       {
  33.         "_id" : "set1",
  34.         "host" : "set1/192.168.1.202:10001,192.168.1.202:10002,192.168.1.202:10003"
  35.       }
  36.       {
  37.         "_id" : "set2",
  38.         "host" : "set2/192.168.1.202:10011,192.168.1.202:10012,192.168.1.202:10013"
  39.       }
  40.   databases:
  41.         { "_id" : "admin", "partitioned" : false, "primary" : "config" }
  42.         { "_id" : "test", "partitioned" : true, "primary" : "set1" }
  43.                 test.data chunks:
  44.                         { "_id" : { $minKey : 1 } } -->> { "_id" : { $maxKey : 1 } } on : set1 { "t" : 1000, "i" : 0 }


---- 配置结束 ------

OK! 基本搞定，可以测试一下。

  1. > use test
  2. switched to db test
  3. > db.data.insert({name:1})
  4. > db.data.insert({name:2})
  5. > db.data.insert({name:3})
  6. > db.data.find()
  7. { "_id" : ObjectId("4c85a6d9ce93b9b1b302ebe7"), "name" : 1 }
  8. { "_id" : ObjectId("4c85a6dbce93b9b1b302ebe8"), "name" : 2 }
  9. { "_id" : ObjectId("4c85a6ddce93b9b1b302ebe9"), "name" : 3 }

人气教程排行

本站所有资源全部来源于网络，若本站发布的内容侵害到您的隐私或者利益，请联系我们删除！