电子技术论坛_中国专业的电子工程师学习交流社区-中电网技术论坛

标题: IBM dashDB Local 从入门到精通，第 4 部分 dashDB local MPP Linux 安装指南（11） [打印本页]

作者: look_w 时间: 2017-12-18 10:19 标题: IBM dashDB Local 从入门到精通，第 4 部分 dashDB local MPP Linux 安装指南（11）

编译 GPFS portability layer rpm（每台机器）

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29

[root@i-si5oq07a src]# mv /etc/redhat-release/etc/redhat-release.original 
[root@i-si5oq07a src]# cat/etc/redhat-release.original 
CentOS Linux release 7.3.1611 (Core) 
[root@i-si5oq07a src]# echo "Red Hat Enterprise Linux Server release 7.3" > /etc/redhat-release 
[root@i-si5oq07a src]# cat /etc/redhat-release 
Red Hat Enterprise Linux Server release 7.3 
[root@i-si5oq07a src]# /usr/lpp/mmfs/bin/mmbuildgpl 
-------------------------------------------------------- 
mmbuildgpl: Building GPL module begins at Tue Feb 7 12:54:19 CST 2017. 
-------------------------------------------------------- 
Verifying Kernel Header... 
... ... 
-------------------------------------------------------- 
mmbuildgpl: Building GPL module completed successfully at Tue Feb 7 12:54:32 CST 2017. 
-------------------------------------------------------- 
as results, it creates /usr/lpp/mmfs/src/config/env.mcr file: 
[root@i-8t7n6xp2 config]# more env.mcr 
#define GPFS_ARCH_X86_64 
#define GPFS_LINUX 
LINUX_DISTRIBUTION := REDHAT_AS_LINUX 
#define LINUX_KERNEL_VERSION 31000514 
KERNEL_BUILD_DIR := /usr/src/kernels/3.10.0-514.6.1.el7.x86_64 
[root@i-8t7n6xp2 config]# 
[root@i-si5oq07a src]# cd /lib/modules/`uname -r`/extra 
[root@i-si5oq07a extra]# ls -l 
total 4040 
-rw-r--r-- 1 root root 3508824 Feb 7 12:54 mmfs26.ko 
-rw-r--r-- 1 root root 554336 Feb 7 12:54 mmfslinux.ko 
-rw-r--r-- 1 root root 68648 Feb 7 12:54 tracedev.ko

配置 GPFS（在 head 节点 i-8t7n6xp2 运行）

1.执行下述命令

1	vi /root/.bashrc<br> export PATH=$PATH:/usr/lpp/mmfs/bin/

2.创建 nodes 文件 gpfs.nodefile

1
2
3
4

vi gpfs.nodefile 
i-8t7n6xp2:quorum-manager 
i-oisebqqf:quorum-manager 
i-si5oq07a:quorum-manager

3.创建 GPFS cluster
我们可以使用 mmcrcluster 命令创建 GPFS cluster，如下所示，注意，一定要在防火墙中打开 22,1191 端口：

1
2
3
4
5
6
7

[root@i-8t7n6xp2 ~]# /usr/lpp/mmfs/bin/mmcrcluster -N gpfs.nodefile -p
i-8t7n6xp2 -s i-oisebqqf -C GPFSdashDB -A -r /usr/bin/ssh -R /usr/bin/scp 
mmcrcluster: Performing preliminary node verification ... 
... ... 
Use the mmchlicense command to designate licenses as needed. 
mmcrcluster: Propagating the cluster configuration data to all 
affected nodes. This is an asynchronous process.

我们可以使用 mmlscluster 命令查看创建的 GPFS 集群，如下所示：

1
2
3
4
5
6
7
8

[root@i-8t7n6xp2 ~]#/usr/lpp/mmfs/bin/mmlscluster 
GPFS cluster information 
... ... 
Node Daemon node name IP address Admin node name Designation 
------------------------------------------------------------------- 
1 i-8t7n6xp2 192.168.0.2 i-8t7n6xp2 quorum-manager 
2 i-oisebqqf 192.168.0.3 i-oisebqqf quorum-manager 
3 i-si5oq07a 192.168.0.4 i-si5oq07a quorum-manager

4.为各个节点设置 license mode

1
2
3
4
5
6
7
8
9

[root@i-8t7n6xp2 ~]# /usr/lpp/mmfs/bin/mmchlicense server --accept -N
i-8t7n6xp2 
[root@i-8t7n6xp2 ~]# /usr/lpp/mmfs/bin/mmchlicense server --accept -N i-oisebqqf 
[root@i-8t7n6xp2 ~]# /usr/lpp/mmfs/bin/mmchlicense server --accept -N i-si5oq07a 
The following nodes will be designated as possessing server licenses: 
i-si5oq07a 
mmchlicense: Command successfully completed 
mmchlicense: Propagating the cluster configuration data to all 
affected nodes. This is an asynchronous process.

5.设置 Network Shared Disks (NSDs)
我们可以使用 fdisk -l 或 df -Th 命令查看需要创建 NSD 的磁盘设备，正如我们上边介绍的，本次示例中，我们已经在 Virtual SAN 中创建了/dev/sdc 设备，容量 200GB。
首先，我们需要创建 NSD 配置文件 gpfs.nsd，如下所示：

1 2	vi gpfs.nsd<br> %nsd: device=/dev/sdc nsd=mynsd usage=dataAndMetadata

接下来，我们通过执行 mmcrnsd 命令创建 NSD，如下所示：

1
2
3
4

[root@i-8t7n6xp2 ~]# /usr/lpp/mmfs/bin/mmcrnsd -F gpfs.nsd -v no 
mmcrnsd: Processing disk sdc 
mmcrnsd: Propagating the cluster configuration data to all 
affected nodes. This is an asynchronous process.

我们可以通过 mmlsnsd 命令查看创建的 NSD，如下所示：

1
2
3
4

[root@i-8t7n6xp2 ~]# /usr/lpp/mmfs/bin/mmlsnsd -m 
Disk name NSD volume ID Device Node name Remarks 
------------------------------------------------------------------------- 
mynsd C0A800025899B4B5 /dev/sdc i-8t7n6xp2

6.启动 GPFS cluster

1	[root@i-8t7n6xp2 ~]#/usr/lpp/mmfs/bin/mmstartup -a

7.检查 GPFS cluster 状态，确认所有节点状态为 active

1
2
3
4

echo ======================================================== 
echo Check status of GPFS cluster peer domain 
echo ======================================================== 
[root@i-8t7n6xp2~]#/usr/lpp/mmfs/bin/mmgetstate -a -L

8.创建 GPFS 文件系统

1
2
3
4
5

[root@i-8t7n6xp2 ~]# /usr/lpp/mmfs/bin/mmcrfs clusterfs -F gpfs.nsd -A
yes -B 512k -T /mnt/clusterfs The following disks of clusterfs will be formatted on node i-8t7n6xp2: mynsd: size
204800 MB Formatting file system ... ... ... Completed creation of file system
/dev/clusterfs. mmcrfs: Propagating the cluster configuration data to all affected
nodes. This is an asynchronous process.

将 GPFS 文件系统挂接在所有节点上

1
2
3
4
5
6
7

[root@i-8t7n6xp2 ~]# /usr/lpp/mmfs/bin/mmmount clusterfs /mnt/clusterfs
-N all 
Tue Feb 7 19:55:24 CST 2017: mmmount: Mounting file systems ... 
i-8t7n6xp2: Warning: Permanently added 'i-8t7n6xp2,192.168.0.2' (ECDSA) to the list of known hosts. 
i-oisebqqf: Warning: Permanently added 'i-oisebqqf,192.168.0.3' (ECDSA) to the list of known hosts. 
i-si5oq07a: Warning: Permanently added 'i-si5oq07a,192.168.0.4' (ECDSA) to
the list of known hosts.

我们可以通过 mmlsmount 命令查看文件系统挂接状态

1 2	[root@i-8t7n6xp2 ~]#<strong> /usr/lpp/mmfs/bin/mmlsmount all</strong><br> File system clusterfs is mounted on 3 nodes.

GPFS 相关管理命令

1 2	关闭 GPFS 节点<br><strong>/usr/lpp/mmfs/bin/mmumount all -a</strong><br><strong>/usr/lpp/mmfs/bin/mmshutdown -a</strong><br> 启动 gpfs 节点<br><strong>/usr/lpp/mmfs/bin/mmstartup -a</strong><br><strong>/usr/lpp/mmfs/bin/mmgetstate -a -L</strong><br><strong>/usr/lpp/mmfs/bin/mmmount all -a</strong>

欢迎光临电子技术论坛_中国专业的电子工程师学习交流社区-中电网技术论坛 (http://bbs.eccn.com/)