当前位置:Gxlcms > 数据库问题 > DRBD-MYSQL分布式块设备实现高可用

DRBD-MYSQL分布式块设备实现高可用

时间:2021-07-01 10:21:17 帮助过:18人阅读

分布式块设备实现高可用

 

一、介绍DRBD:

DRBD是一个用软件实现的、无共享的、服务器之间镜像块设备内容的存储复制解决方案。 DRBD Logo数据镜像:实时、透明、同步(所有服务器都成功后返回)、异步(本地服务器成功后返回)。DBRD的核心功能通过Linux的内核实现,最接近系统的IO栈,但它不能神奇地添加上层的功能比如检测到EXT3文件系统的崩溃。DBRD的位置处于文件系统以下,比文件系统更加靠近操作系统内核及IO栈。

二、DRBD安装:(ha高可用集群。:在centos7的版本下)

环境:

192.168.5.101  cml1

192.168.5.102  cml2

1、修改hosts文件保证hosts之间能够互相访问:

[root@cml1 ~]# cat /etc/hosts

127.0.0.1  localhost localhost.localdomain localhost4 localhost4.localdomain4

::1        localhost localhost.localdomain localhost6 localhost6.localdomain6

192.168.5.101 cml1 wwwNaNl1.com

192.168.5.102 cml2 wwwNaNl2.com

192.168.5.104 cml3 wwwNaNl3.com

192.168.5.105 cml4 wwwNaNl4.com

2、修改ssh互信:

[root@cml1 ~]# ssh-keygen

[root@cml1 ~]# ssh-copy-id cml2

3、设置时钟同步:

[root@cml1 ~]# crontab -l

*/5 * * * * ntpdate cn.pool.ntp.org

4.安装DRBD:

[root@cml1 ~]#  rpm --importhttps://www.elrepo.org/RPM-GPG-KEY-elrepo.org

[root@cml1 ~]#  rpm -Uvh http://www.elrepo.org/elrepo-release-7.0-2.el7.elrepo.noarch.rpm

[root@cml1 ~]#  yum install -y kmod-drbd84 drbd84-utils

5、主配置文件:

/etc/drbd.conf #主配置文件

/etc/drbd.d/global_common.conf#全局配置文件

6、查看主配置文件:

[root@cml1 ~]# cat /etc/drbd.conf

# You can find an example in  /usr/share/doc/drbd.../drbd.conf.example

 

 

include"drbd.d/global_common.conf";

include "drbd.d/*.res";

7、配置文件说明:

[root@cml1 ~]# vim/etc/drbd.d/global_common.conf

global {

   usage-count no;  #是否参加DRBD使用统计,默认为yes。官方统计drbd的装机量

    #minor-count dialog-refresh disable-ip-verification

}

common {

   protocol C;      #使用DRBD的同步协议

   handlers {

       pri-on-incon-degr "/usr/lib/drbd/notify-pri-on-incon-degr.sh;/usr/lib/drbd/notify-emergency-reboot.sh; echo b > /proc/sysrq-trigger ;reboot -f";

       pri-lost-after-sb "/usr/lib/drbd/notify-pri-lost-after-sb.sh;/usr/lib/drbd/notify-emergency-reboot.sh; echo b > /proc/sysrq-trigger ;reboot -f";

       local-io-error "/usr/lib/drbd/notify-io-error.sh;/usr/lib/drbd/notify-emergency-shutdown.sh; echo o > /proc/sysrq-trigger ;halt -f";

    }

   startup {

       # wfc-timeout degr-wfc-timeout outdated-wfc-timeout wait-after-sb

    }

   options {

       # cpu-mask on-no-data-accessible

    }

   disk {

       on-io-error detach; #配置I/O错误处理策略为分离

       # size max-bio-bvecs on-io-error fencing disk-barrier disk-flushes

       # disk-drain md-flushes resync-rate resync-after al-extents

       # c-plan-ahead c-delay-target c-fill-target c-max-rate

       # c-min-rate disk-timeout

    }

   net {

 

       # protocol timeout max-epoch-size max-buffers unplug-watermark

       # connect-int ping-int sndbuf-size rcvbuf-size ko-count

       # allow-two-primaries cram-hmac-alg shared-secret after-sb-0pri

       # after-sb-1pri after-sb-2pri always-asbp rr-conflict

       # ping-timeout data-integrity-alg tcp-cork on-congestion

       # congestion-fill congestion-extents csums-alg verify-alg

       # use-rle

    }

   syncer {

       rate 1024M;    #设置主备节点同步时的网络速率

    }

}

注释: on-io-error 策略可能为以下选项之一

detach 分离:这是默认和推荐的选项,如果在节点上发生底层的硬盘I/O错误,它会将设备运行在Diskless无盘模式下

pass_on:DRBD会将I/O错误报告到上层,在主节点上,它会将其报告给挂载的文件系统,但是在此节点上就往往忽略(因此此节点上没有可以报告的上层)

-local-in-error:调用本地磁盘I/O处理程序定义的命令;这需要有相应的local-io-error调用的资源处理程序处理错误的命令;这就给管理员有足够自由的权力命令命令或是脚本调用local-io-error处理I/O错误定义一个资源

8、创建配置文件:

[root@cml1 ~]# cat/etc/drbd.d/mysql.res

resource mysql {   ##资源名称

protocol C;                  ##使用协议

meta-disk internal;

device /dev/drbd1;    ##DRBD设备名称

syncer {

verify-alg sha1;                   ##加密算法

}

net {

allow-two-primaries;

}

on cml1 {

disk /dev/sdb1;            ##drbd使用的磁盘分区为“mysql”(可以不想同的磁盘名字)

address 192.168.5.101:7789;    ##设置DRBD监听地址的端口

}

on cml2 {

disk /dev/sdb1;

address 192.168.5.102:7789;

}

}

 

9、然后把配置文件copy到对面的机器上:

[root@cml1 ~]# scp -rp /etc/drbd.d/*cml2:/etc/drbd.d/

##然后在两台机器上安装mariadb

[root@cml1 ~]# yum install -ymariadb-server mariadb

[root@cml1 ~]# mkdir /data/

[root@cml1 ~]# cat /etc/my.cnf

[mysqld]

datadir=/data

socket=/var/lib/mysql/mysql.sock

# Disabling symbolic-links is recommendedto prevent assorted security risks

symbolic-links=0

# Settings user and group are ignored whensystemd is used.

# If you need to run mysqld under adifferent user or group,

# customize your systemd unit file formariadb according to the

# instructions inhttp://fedoraproject.org/wiki/Systemd

 

[mysqld_safe]

log-error=/var/log/mariadb/mariadb.log

pid-file=/var/run/mariadb/mariadb.pid

 

#

# include all files from the configdirectory

#

!includedir /etc/my.cnf.d

[root@cml1 ~]# chown mysql:mysql /data/

10、在node2上面启动:

[root@cml1 ~]#  drbdadm create-md mysql

initializing activity log

initializing bitmap (160 KB) to all zero

Writing meta data...

New drbd meta data block successfullycreated.

[root@cml1 ~]# modprobe drbd

##查看内核是否已经加载了模块:

[root@cml1 drbd.d]# lsmod | grep drbd

drbd                  396875  1

libcrc32c              12644  4 xfs,drbd,ip_vs,nf_conntrack

###

 

[root@cml1 ~]# drbdadm up mysql

[root@cml1 ~]# drbdadm -- --force primarymysql

##查看状态:

[root@cml1 ~]# cat /proc/drbd

version: 8.4.10-1 (api:1/proto:86-101)

GIT-hash: a4d5de01fffd7e4cde48a080e2c686f9e8cebf4cbuild by mockbuild@, 2017-09-15 14:23:22

 

 1:cs:WFConnection ro:Primary/Unknown ds:UpToDate/DUnknown C r----s

   ns:0 nr:0 dw:0 dr:912 al:8 bm:0 lo:0 pe:0 ua:0 ap:0 ep:1 wo:foos:5240636

11、在对端节点执行:

[root@cml2 ~]# chown mysql:mysql /data/

[root@cml2 ~]# drbdadm create-md mysql

[root@cml2 ~]# modprobe drbd

[root@cml2 ~]# drbdadm up mysql

在从上面可以查看数据同步的状态:

[root@cml2 ~]# cat /proc/drbd

version: 8.4.10-1 (api:1/proto:86-101)

GIT-hash:a4d5de01fffd7e4cde48a080e2c686f9e8cebf4c build by mockbuild@, 2017-09-1514:23:22

 

 1:cs:SyncTarget ro:Secondary/Primary ds:Inconsistent/UpToDate C r-----

   ns:0 nr:1513472 dw:1513472 dr:0 al:8 bm:0 lo:0 pe:0 ua:0 ap:0 ep:1 wo:foos:3727164

       [====>...............] sync‘ed: 29.0% (3636/5116)M

       finish: 0:01:32 speed: 40,192 (38,804) want: 102,400 K/sec

12、格式化并挂载:

[root@cml1 ~]# mkfs.ext4 /dev/drbd1

meta-data=/dev/drbd1             isize=512    agcount=4, agsize=327540 blks

        =                      sectsz=512   attr=2, projid32bit=1

        =                       crc=1        finobt=0, sparse=0

data    =                      bsize=4096   blocks=1310159,imaxpct=25

        =                      sunit=0      swidth=0 blks

naming  =version 2             bsize=4096   ascii-ci=0 ftype=1

log     =internal log          bsize=4096   blocks=2560,version=2

        =                      sectsz=512   sunit=0 blks,lazy-count=1

realtime =none                   extsz=4096   blocks=0, rtextents=0

[root@cml1 ~]# mount /dev/drbd1 /mnt

[root@cml1 ~]# df -h

Filesystem           Size Used Avail Use% Mounted on

/dev/mapper/cl-root   17G 6.9G   11G  41% /

devtmpfs             588M     0 588M   0% /dev

tmpfs                599M     0 599M   0% /dev/shm

tmpfs                599M   34M 565M   6% /run

tmpfs                599M     0 599M   0% /sys/fs/cgroup

/dev/sda1           1014M  168M 847M  17% /boot

/dev/drbd1           5.0G   33M 5.0G   1% /mnt

注####要想使得从可以挂载,我们必须,先把主切换成丛,然后再到从上面挂载:

 

有如下资源:

资源的连接状态;一个资源可能有以下连接状态中的一种

StandAlone 独立的:网络配置不可用;资源还没有被连接或是被管理断开(使用 drbdadm disconnect 命令),或是由于出现认证失败或是脑裂的情况

Disconnecting 断开:断开只是临时状态,下一个状态是StandAlone独立的

Unconnected 悬空:是尝试连接前的临时状态,可能下一个状态为WFconnection和WFReportParams

Timeout 超时:与对等节点连接超时,也是临时状态,下一个状态为Unconected悬空

BrokerPipe:与对等节点连接丢失,也是临时状态,下一个状态为Unconected悬空

NetworkFailure:与对等节点推动连接后的临时状态,下一个状态为Unconected悬空

ProtocolError:与对等节点推动连接后的临时状态,下一个状态为Unconected悬空

TearDown 拆解:临时状态,对等节点关闭,下一个状态为Unconected悬空

WFConnection:等待和对等节点建立网络连接

WFReportParams:已经建立TCP连接,本节点等待从对等节点传来的第一个网络包

Connected 连接:DRBD已经建立连接,数据镜像现在可用,节点处于正常状态

StartingSyncS:完全同步,有管理员发起的刚刚开始同步,未来可能的状态为SyncSource或PausedSyncS

StartingSyncT:完全同步,有管理员发起的刚刚开始同步,下一状态为WFSyncUUID

WFBitMapS:部分同步刚刚开始,下一步可能的状态为SyncSource或PausedSyncS

WFBitMapT:部分同步刚刚开始,下一步可能的状态为WFSyncUUID

WFSyncUUID:同步即将开始,下一步可能的状态为SyncTarget或PausedSyncT

SyncSource:以本节点为同步源的同步正在进行

SyncTarget:以本节点为同步目标的同步正在进行

PausedSyncS:以本地节点是一个持续同步的源,但是目前同步已经暂停,可能是因为另外一个同步正在进行或是使用命令(drbdadm pause-sync)暂停了同步

PausedSyncT:以本地节点为持续同步的目标,但是目前同步已经暂停,这可以是因为另外一个同步正在进行或是使用命令(drbdadm pause-sync)暂停了同步

VerifyS:以本地节点为验证源的线上设备验证正在执行

VerifyT:以本地节点为验证目标的线上设备验证正在执行

13、查看资源角色命令:

[root@cml1 ~]# cat /proc/drbd

version: 8.4.10-1 (api:1/proto:86-101)

GIT-hash:a4d5de01fffd7e4cde48a080e2c686f9e8cebf4c build by mockbuild@, 2017-09-1514:23:22

 

 1:cs:Connected ro:Primary/Secondary ds:UpToDate/UpToDate C r-----

   ns:10526345 nr:340312 dw:392669 dr:10488683 al:24 bm:0 lo:0 pe:0 ua:0ap:0 ep:1 wo:f oos:0

14、在刚才的环境模拟脑裂的情况:

[root@cml1 etc]# drbd-overview

NOTE: drbd-overview will be deprecatedsoon.

Please consider using drbdtop.

 

 1:mysql/0 Connected Secondary/Primary UpToDate/UpToDate

You have new mail in /var/spool/mail/root

 

node3现在是master状态:

[root@cml2 ~]# drbd-overview

NOTE: drbd-overview will be deprecatedsoon.

Please consider using drbdtop.

 

 1:mysql/0 Connected Primary/Secondary UpToDate/UpToDate /test xfs 5.0G 33M 5.0G 1%

You have new mail in /var/spool/mail/root

 

15、我们模拟故障,我在node3上面开启iptables,是的不能互相通信,模拟master故障:

[root@cml2 ~]# iptables -P INPUT DROP.  ##注意这里要先开启22端口:

 

###我们查看一下cml1的状态:

[root@cml1 etc]# drbd-overview

NOTE: drbd-overview will be deprecatedsoon.

Please consider using drbdtop.

 

 1:mysql/0 WFConnection Secondary/Unknown UpToDate/DUnknown

You have new mail in /var/spool/mail/root

 

###现在是Unknown状态,是不可用的:

 

16、我们将cml1升级为master:

[root@cml1 etc]# drbdadm primary mysql

[root@cml1 etc]# drbd-overview

NOTE: drbd-overview will be deprecatedsoon.

Please consider using drbdtop.

 

 1:mysql/0 WFConnection Primary/Unknown UpToDate/DUnknown

[root@cml1 etc]# mount /dev/drbd1 /mnt/

[root@cml1 etc]# ls /mnt/

xiaoluo.txt

17、这个时候加入node3网络恢复了,就会出现脑裂的情况:

iptables -P INPUT ACCEPT

 

##查看两个结点的状态:

[root@cml1 etc]# drbdadm role mysql

Primary/Unknown

[root@cml2 ~]# drbdadm role mysql

Primary/Unknown

 

##查看两个的链接状态:

[root@cml1 etc]# drbd-overview

NOTE: drbd-overview will be deprecatedsoon.

Please consider using drbdtop.

 

 1:mysql/0 StandAlone Primary/Unknown UpToDate/DUnknown /mnt xfs 5.0G 33M 5.0G 1%

 

[root@cml2 ~]# drbd-overview

NOTE: drbd-overview will be deprecatedsoon.

Please consider using drbdtop.

 

 1:mysql/0 StandAlone Primary/Unknown UpToDate/DUnknown /test xfs 5.0G 33M 5.0G 1%

 

18、现在在cml1上面做处理办法,把他将为备用结点:

[root@cml1 ~]# umount /mnt/

[root@cml1 ~]# drbdadm disconnect mysql

??: Failure: (162) Invalid configurationrequest

additional info from kernel:

unknown connection

Command ‘drbdsetup-84 disconnectipv4:192.168.113.143:7789 ipv4:192.168.113.144:7789‘ terminated with exit code10

要把mariadb stop掉才可以切成secondary

 

[root@cml1 ~]# drbdadm secondary mysql

[root@cml1 ~]# drbd-overview

NOTE: drbd-overview will be deprecatedsoon.

Please consider using drbdtop.

 

 1:mysql/0 StandAlone Secondary/Unknown UpToDate/DUnknown

[root@cml1 ~]# drbdadm connect--discard-my-data mysql  ##在从主机上做

 

##现在我们查看一下集群结点还是不能用的:

[root@cml1 ~]# drbd-overview

NOTE: drbd-overview will be deprecatedsoon.

Please consider using drbdtop.

 

 1:mysql/0 WFConnection Secondary/Unknown UpToDate/DUnknown

 

19、最后再到node3结点上重新连接:

[root@cml2 ~]# drbdadm connect mysql   ##在主的主机上面做

 

##然后我们再查看一下状态:

[root@cml2 ~]# drbd-overview

NOTE: drbd-overview will be deprecatedsoon.

Please consider using drbdtop.

 

 1:mysql/0 Connected Primary/Secondary UpToDate/UpToDate /test xfs 5.0G 33M 5.0G 1%

 

[root@cml1 ~]# drbd-overview

NOTE: drbd-overview will be deprecatedsoon.

Please consider using drbdtop.

 

 1:mysql/0 Connected Secondary/Primary UpToDate/UpToDate

 


本文出自 “第一个legehappy51cto博客” 博客,请务必保留此出处http://legehappy.blog.51cto.com/13251607/1975804

DRBD-MYSQL分布式块设备实现高可用

标签:mysql   drbd   分布式   

人气教程排行