运维开发网

centos6 – 重启后NFS服务器不提供共享

运维开发网 https://www.qedev.com 2020-05-08 15:42 出处:网络 作者:运维开发网整理
我有两个Linode容器.方框A是我们的通用Web服务器.它偶尔需要访问Box B,它被设置为NFS服务器.
我有两个Linode容器.方框A是我们的通用Web服务器.它偶尔需要访问Box B,它被设置为NFS服务器.

当Box B重新启动时,无论我做什么,Box A都无法访问任何NFS共享.经过几个小时的故障排除后,我终于能够将其缩小到一步修复.

Box B重启后:

$sudo service nfs restart

这些都是CentOS 6.8盒子,是最新的.我相信,与NFS相关的软件包都是通过yum安装的.我确实在设置整个事情时遇到了一些麻烦;这不是一个平稳的过程,但在重新启动nfs服务之后,一切都很顺利.

如果我

$sudo service --status-all

发出重启之前和之后没有区别.也许这是一个时间问题?但我不知道怎么开始麻烦拍这个.我能做什么?

其他注意事项:

>我正在使用autofs自动挂载Box A中的共享,但共享也不会手动挂载

>我在Windows和Mac桌面和服务器上度过了我的日子,但我已经在Linux上运行多年的网站了.我精通我需要做的事情,但这不是我的舒适区域,我花了很多时间在谷歌上搜索如何做新事物.

我甚至不知道在哪里检查.我没有在日志中看到任何明显的东西,但告诉我要寻找什么,我会发布.

更新

在方框B:

[shorowitz@BoxB ~]$sudo chkconfig --list nfs
 nfs                0:off   1:off   2:on    3:on    4:on    5:on    6:off
 [shorowitz@BoxB ~]$sudo chkconfig --list nfslock
 nfslock            0:off   1:off   2:on    3:on    4:on    5:on    6:off

更新2

重新启动BoxB后,运行

$sudo showmount -e BoxB

从BoxA显示预期的挂载点,但我无法挂载它们.只需在BoxB上重启nfs即可

$sudo service nfs restart
 Shutting down NFS daemon:                                  [  OK  ]
 Shutting down NFS mountd:                                  [  OK  ]
 Shutting down NFS services:                                [  OK  ]
 Shutting down RPC idmapd:                                  [  OK  ]
 FATAL: Module nfsd not found.
 FATAL: Error running install command for nfsd
 Starting NFS services:                                     [  OK  ]
 Starting NFS mountd:                                       [  OK  ]
 Starting NFS daemon:                                       [  OK  ]
 Starting RPC idmapd:                                       [  OK  ]

BoxA上的支架立即可用.当NFS已经工作时,这些致命的错误也出现在后续的重启中,所以我不知道它们有多相关(我以为我已经发布了它们).

其他日志信息

我在11月15日9:29发布了reboot命令

grep -i "nfs" /var/log/message*
 messages:Nov 15 09:29:08 BoxB kernel: nfsd: last server has exited, flushing export cache
 messages:Nov 15 09:29:54 BoxB kernel: RPC: Registered tcp NFSv4.1 backchannel transport module.
 messages:Nov 15 09:29:54 BoxB kernel: FS-Cache: Netfs 'nfs' registered for caching
 messages:Nov 15 09:29:54 BoxB kernel: NFS: Registering the id_resolver key type
 messages:Nov 15 09:29:54 BoxB kernel: nfs4filelayout_init: NFSv4 File Layout Driver Registering...
 messages:Nov 15 09:29:54 BoxB kernel: Installing knfsd (copyright (C) 1996 okir@monad.swb.de).
 messages:Nov 15 09:29:54 BoxB kernel: xenfs: not registering filesystem on non-xen platform
 messages:Nov 15 09:29:54 BoxB rpc.mountd[2740]: NFS v4 mounts will be disabled unless fsid=0
 messages:Nov 15 09:29:54 BoxB kernel: NFSD: Using /var/lib/nfs/v4recovery as the NFSv4 state recovery directory
 messages:Nov 15 09:29:54 BoxB kernel: NFSD: starting 90-second grace period (net ****************)
 messages:Nov 15 09:33:39 BoxB kernel: nfsd: last server has exited, flushing export cache
 messages:Nov 15 09:33:40 BoxB kernel: NFSD: Using /var/lib/nfs/v4recovery as the NFSv4 state recovery directory
 messages:Nov 15 09:33:40 BoxB kernel: NFSD: starting 90-second grace period (net ****************)

更新3:

盒B

[shorowitz@BoxB ~]$sudo chkconfig --list | egrep "nfs|rpc"
 nfs                0:off   1:off   2:on    3:on    4:on    5:on    6:off
 nfslock            0:off   1:off   2:on    3:on    4:on    5:on    6:off
 rpcbind            0:off   1:off   2:on    3:on    4:on    5:on    6:off
 rpcgssd            0:off   1:off   2:off   3:on    4:on    5:on    6:off
 rpcsvcgssd         0:off   1:off   2:off   3:off   4:off   5:off   6:off

 [shorowitz@BoxB ~]$sudo iptables --list -n -v
 Chain INPUT (policy ACCEPT 0 packets, 0 bytes)
  pkts bytes target     prot opt in     out     source               destination         
     0     0 ACCEPT     all  --  lo     *       0.0.0.0/0            0.0.0.0/0           
     0     0 REJECT     all  --  !lo    *       127.0.0.0/8          0.0.0.0/0           reject-with icmp-port-unreachable 
    18   710 ACCEPT     icmp --  *      *       0.0.0.0/0            0.0.0.0/0           state NEW icmp type 8 
   471 26200 ACCEPT     tcp  --  *      *       0.0.0.0/0            0.0.0.0/0           tcp dpt:22 state NEW 
  204K  393M ACCEPT     all  --  *      *       {BoxA IP}            0.0.0.0/0           
  6721  754K ACCEPT     all  --  *      *       0.0.0.0/0            0.0.0.0/0           state RELATED,ESTABLISHED 
  2859  168K LOG        all  --  *      *       0.0.0.0/0            0.0.0.0/0           limit: avg 5/min burst 5 LOG flags 0 level 7 prefix `iptables_INPUT_denied: ' 
  9229  628K REJECT     all  --  *      *       0.0.0.0/0            0.0.0.0/0           reject-with icmp-port-unreachable 

 Chain FORWARD (policy ACCEPT 0 packets, 0 bytes)
  pkts bytes target     prot opt in     out     source               destination         
     0     0 LOG        all  --  *      *       0.0.0.0/0            0.0.0.0/0           limit: avg 5/min burst 5 LOG flags 0 level 7 prefix `iptables_FORWARD_denied: ' 
     0     0 REJECT     all  --  *      *       0.0.0.0/0            0.0.0.0/0           reject-with icmp-port-unreachable 

 Chain OUTPUT (policy ACCEPT 278K packets, 8386M bytes)
  pkts bytes target     prot opt in     out     source               destination        

 [shorowitz@BoxB ~]$sudo rpcinfo -p
program vers proto   port  service
 100000    4   tcp    111  portmapper
 100000    3   tcp    111  portmapper
 100000    2   tcp    111  portmapper
 100000    4   udp    111  portmapper
 100000    3   udp    111  portmapper
 100000    2   udp    111  portmapper
 100024    1   udp  38148  status
 100024    1   tcp  45681  status
 100005    1   udp  37846  mountd
 100005    1   tcp  59259  mountd
 100005    2   udp  59934  mountd
 100005    2   tcp  42645  mountd
 100005    3   udp  33867  mountd
 100005    3   tcp  41823  mountd
 100003    2   tcp   2049  nfs
 100003    3   tcp   2049  nfs
 100003    4   tcp   2049  nfs
 100227    2   tcp   2049  nfs_acl
 100227    3   tcp   2049  nfs_acl
 100003    2   udp   2049  nfs
 100003    3   udp   2049  nfs
 100003    4   udp   2049  nfs
 100227    2   udp   2049  nfs_acl
 100227    3   udp   2049  nfs_acl
 100021    1   udp  37287  nlockmgr
 100021    3   udp  37287  nlockmgr
 100021    4   udp  37287  nlockmgr
 100021    1   tcp  37579  nlockmgr
 100021    3   tcp  37579  nlockmgr
 100021    4   tcp  37579  nlockmgr

这没有任何回报:

grep -v "^#" /etc/sysconfig/nfs

BOXA

$chkconfig --list | egrep "nfs|rpc"
 nfs                0:off   1:off   2:on    3:on    4:on    5:on    6:off
 nfslock            0:off   1:off   2:on    3:on    4:on    5:on    6:off
 rpcbind            0:off   1:off   2:on    3:on    4:on    5:on    6:off
 rpcgssd            0:off   1:off   2:off   3:on    4:on    5:on    6:off
 rpcsvcgssd         0:off   1:off   2:off   3:off   4:off   5:off   6:off

 $iptables --list -n -v
 Chain INPUT (policy ACCEPT 0 packets, 0 bytes)
  pkts bytes target     prot opt in     out     source               destination         
  390K   58M ACCEPT     all  --  lo     *       0.0.0.0/0            0.0.0.0/0           
     0     0 REJECT     all  --  *      *       0.0.0.0/0            127.0.0.0/8         reject-with icmp-port-unreachable 
  990K 7850M ACCEPT     all  --  *      *       0.0.0.0/0            0.0.0.0/0           state RELATED,ESTABLISHED 
     0     0 DROP       all  --  *      *       43.255.188.145       0.0.0.0/0           
     8   388 ACCEPT     tcp  --  *      *       0.0.0.0/0            0.0.0.0/0           tcp dpt:587 
 11864  608K ACCEPT     tcp  --  *      *       0.0.0.0/0            0.0.0.0/0           tcp dpt:25 
     1    40 ACCEPT     tcp  --  *      *       0.0.0.0/0            0.0.0.0/0           tcp dpt:636 
  4545  238K ACCEPT     tcp  --  *      *       0.0.0.0/0            0.0.0.0/0           tcp dpt:80 
  9759  553K ACCEPT     tcp  --  *      *       0.0.0.0/0            0.0.0.0/0           tcp dpt:443 
    24   960 ACCEPT     tcp  --  *      *       0.0.0.0/0            0.0.0.0/0           tcp dpt:8080 
   320 19152 ACCEPT     tcp  --  *      *       0.0.0.0/0            0.0.0.0/0           state NEW tcp dpt:22 
    85  5681 ACCEPT     icmp --  *      *       0.0.0.0/0            0.0.0.0/0           
  3254  194K LOG        all  --  *      *       0.0.0.0/0            0.0.0.0/0           limit: avg 5/min burst 5 LOG flags 0 level 7 prefix `iptables denied: ' 
  3634  227K DROP       all  --  *      *       0.0.0.0/0            0.0.0.0/0           

 Chain FORWARD (policy ACCEPT 0 packets, 0 bytes)
  pkts bytes target     prot opt in     out     source               destination         
     0     0 DROP       all  --  *      *       0.0.0.0/0            0.0.0.0/0           

 Chain OUTPUT (policy ACCEPT 0 packets, 0 bytes)
  pkts bytes target     prot opt in     out     source               destination         
 1360K 1907M ACCEPT     all  --  *      *       0.0.0.0/0            0.0.0.0/0           

 $rpcinfo -p
program vers proto   port  service
 100000    4   tcp    111  portmapper
 100000    3   tcp    111  portmapper
 100000    2   tcp    111  portmapper
 100000    4   udp    111  portmapper
 100000    3   udp    111  portmapper
 100000    2   udp    111  portmapper
 100024    1   udp  55882  status
 100024    1   tcp  58283  status
 100011    1   udp    875  rquotad
 100011    2   udp    875  rquotad
 100011    1   tcp    875  rquotad
 100011    2   tcp    875  rquotad
 100005    1   udp  43136  mountd
 100005    1   tcp  55047  mountd
 100005    2   udp  51117  mountd
 100005    2   tcp  42791  mountd
 100005    3   udp  44511  mountd
 100005    3   tcp  46535  mountd
 100003    2   tcp   2049  nfs
 100003    3   tcp   2049  nfs
 100003    4   tcp   2049  nfs
 100227    2   tcp   2049  nfs_acl
 100227    3   tcp   2049  nfs_acl
 100003    2   udp   2049  nfs
 100003    3   udp   2049  nfs
 100003    4   udp   2049  nfs
 100227    2   udp   2049  nfs_acl
 100227    3   udp   2049  nfs_acl
 100021    1   udp  43509  nlockmgr
 100021    3   udp  43509  nlockmgr
 100021    4   udp  43509  nlockmgr
 100021    1   tcp  38725  nlockmgr
 100021    3   tcp  38725  nlockmgr
 100021    4   tcp  38725  nlockmgr

 $mount | grep nfs
 nfsd on /proc/fs/nfsd type nfsd (rw)
 sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw)

11月14日更新

BoxA:

 $cat /etc/auto.master.d/nfs
 xdata  -rw boxb:/srv/nfs/xdata
 xbackup    -rw boxb:/srv/nfs/xbackup
 zbackups   -rw boxb:/srv/nfs/zbackups

 $mount | grep nfs
 mount |grep nfs
 nfsd on /proc/fs/nfsd type nfsd (rw)
 sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw)
 boxb:/srv/nfs/xdata on /mnt/nfs/xdata type nfs (rw,sloppy,vers=4,addr={boxb ip},clientaddr={boxa ip})
你能用更多信息更新你的问题吗?

在NFS服务器上运行

chkconfig --list | egrep "nfs|rpc"
iptables --list -n -v
rpcinfo -p

如果您的nfs服务器设置未自定义,则下一个应该不返回任何内容.

grep -v "^#" /etc/sysconfig/nfs

但是,如果您正在运行iptables – 它应该是自定义的.看这里

https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/6/html/Storage_Administration_Guide/s2-nfs-nfs-firewall-config.html

在您的客户端运行

chkconfig --list | egrep "nfs|rpc"
iptables --list -n -v
rpcinfo -p
mount | grep nfs

如果您正在运行NFSV2或NFSV3,则还需要在客户端上运行nfslock

https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/6/html/Storage_Administration_Guide/s1-nfs-start.html

启动NFS时输出中看起来很奇怪 – 这是下面的行

FATAL: Module nfsd not found.
 FATAL: Error running install command for nfsd

我刚刚意识到一件事 – 因为你在openvz上运行 – 下面的链接适用于你的情况.看看这是否有帮助

https://openvz.org/NFS_server_inside_container

编辑1:

我今天在openvz容器上运行测试.

你看到的消息

FATAL: Module nfsd not found.
FATAL: Error running install command for nfsd

是无害的.它在这里描述https://openvz.org/NFS_server_inside_container

我无法重现你的问题.重新启动nfs服务器后 – nfs客户端仍然能够浏览文件,创建新文件,从nfs共享中删除文件.

你现在可以发布自动挂载配置和输出

mount|grep nfs

来自方框A,你已经做过了.但是在卸载自动挂载的文件系统时你就做到了.所以现在在Box A cd上自动挂载dir,然后运行命令.这将提供有关安装期间使用哪些选项的信息.

此外,下次重新启动Box B并且存在此automount问题时 – 从站点A运行此命令

netstat -anp|grep ipofB

这将提供有关涉及哪些端口的信息.

在这种情况下,在B和A上收集tcpdumps也很好.

我倾向于认为你的配置没有任何问题 – 但是在vzhosts上使用iptables会发生一些奇怪的事情.不在您的容器中,而是在主机上.

您还可以尝试做的是在主机A上安装nmap,并在出现问题时 – 从A的角度扫描主机B以查看哪些端口是打开的(有些可能建议使用netstat或ss,但是这种情况下容器前面还有openvz主机防火墙,我们不知道它们是否阻塞了某些东西)

编辑2(11月25日)

你的挂载选项有点奇怪

比较你的输出

$mount | grep nfs
 mount |grep nfs
 nfsd on /proc/fs/nfsd type nfsd (rw)
 sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw)
 boxb:/srv/nfs/xdata on /mnt/nfs/xdata type nfs (rw,sloppy,vers=4,addr={boxb ip},clientaddr={boxa ip})

在下面开采.这是我的/etc/auto.misc.第6行是默认值.第16行我添加了

[root@vznfsclient /]# egrep -vn "^#|^$" /etc/auto.misc
6:cd            -fstype=iso9660,ro,nosuid,nodev :/dev/cdrom
16:nfs          -fstype=nfs             192.168.0.54:/nfs_export

所以当我cd到/ misc / nfs时 – 我的共享被挂载了.但请查看第12行的默认选项.

[root@vznfsclient ~]# mount|egrep -n "nfs|auto"
4:nfsd on /proc/fs/nfsd type nfsd (rw,relatime)
5:sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
10:/etc/auto.misc on /misc type autofs (rw,relatime,fd=6,pgrp=768,timeout=300,minproto=5,maxproto=5,indirect)
11:-hosts on /net type autofs (rw,relatime,fd=12,pgrp=768,timeout=300,minproto=5,maxproto=5,indirect)
12:192.168.0.54:/nfs_export/ on /misc/nfs type nfs (rw,relatime,vers=3,rsize=131072,wsize=131072,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,mountaddr=192.168.0.54,mountvers=3,mountport=42089,mountproto=udp,local_lock=none,addr=192.168.0.54)

首先它是nfsv3,它使用的是udp.好的udp我们可以通过将/etc/auto.misc更改为此来更改为tcp

[root@vznfsclient /]# egrep -vn "^#|^$" /etc/auto.misc
6:cd            -fstype=iso9660,ro,nosuid,nodev :/dev/cdrom
16:nfs          -fstype=nfs,proto=tcp   192.168.0.54:/nfs_export

并且挂载选项将更改为

192.168.0.54:/nfs_export/ on /misc/nfs type nfs (rw,relatime,vers=3,rsize=131072,wsize=131072,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,mountaddr=192.168.0.54,mountvers=3,mountport=45378,mountproto=tcp,local_lock=none,addr=192.168.0.54)

当我尝试在/etc/auto.misc -fstype = nfs4中使用时 – 我甚至无法cd / / misc / nfs哪种有意义,因为在容器https://openvz.org/NFS_server_inside_container内不支持openvz nfsv4

请注意,您的装载选项中包含了草率和简单的rw.这基本上意味着

>传递给mount.nfs的选项并不完全正确,所以它试图解决它.阅读man mount.nfs并搜索/ sloppy.

>我假设它正在尝试使用nfsv4.如果容器内部不支持,甚至不知道它是如何工作的.

我建议将自动挂载映射更改为正确的语法(请参阅我的示例)或参见此处的示例https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/6/html/Storage_Administration_Guide/s2-nfs-config-autofs.html

并测试.我刚刚在2个不同的openvz主机上运行的相同2个容器中运行autofs测试 – 在nfs服务器重启后 – 客户端仍然很高兴.

编辑3

我甚至无法重现你的场景.我将我的/etc/auto.misc改为下面,这几乎就是你所拥有的

[root@vznfsclient nfs]# egrep -nv "^#|^$" /etc/auto.misc
6:cd            -fstype=iso9660,ro,nosuid,nodev :/dev/cdrom
16:nfs          -rw                     192.168.0.54:/nfs_export

重启后,cd / misc / nfs我仍然有这个

[root@vznfsclient nfs]# mount|grep nfs
nfsd on /proc/fs/nfsd type nfsd (rw,relatime)
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
192.168.0.54:/nfs_export/ on /misc/nfs type nfs (rw,relatime,vers=3,rsize=131072,wsize=131072,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,mountaddr=192.168.0.54,mountvers=3,mountport=46453,mountproto=udp,local_lock=none,addr=192.168.0.54)

所以在我的情况下,它正确地尝试使用nfsv3.

现在唯一要做的就是

在/etc/autofs.conf中激活debug

[root@vznfsclient nfs]# grep -i debug /etc/autofs.conf|grep -v "^#"
    logging = debug

启用调试严重性以将其发送到/etc/rsyslog.conf中的/ var / log / messages.改变这个

[root@vznfsclient nfs]# grep info /etc/rsyslog.conf
*.info;mail.none;authpriv.none;cron.none                -/var/log/messages

对此(.info到.debug)

[root@vznfsclient nfs]# grep debug /etc/rsyslog.conf
*.debug;mail.none;authpriv.none;cron.none                -/var/log/messages

重新启动autofs和rsyslog,然后当你更改为自动挂载位置时 – 你应该在/ var / log / messages中看到调试输出

这是我的测试系统的输出

Nov 25 03:06:00 vznfsclient automount[583]: attempting to mount entry /misc/nfs
Nov 25 03:06:00 vznfsclient automount[583]: lookup_mount: lookup(file): looking up nfs
Nov 25 03:06:00 vznfsclient automount[583]: lookup_mount: lookup(file): nfs -> -rw#011#011#011192.168.0.54:/nfs_export
Nov 25 03:06:00 vznfsclient automount[583]: parse_mount: parse(sun): expanded entry: -rw#011#011#011192.168.0.54:/nfs_export
Nov 25 03:06:00 vznfsclient automount[583]: parse_mount: parse(sun): gathered options: rw
Nov 25 03:06:00 vznfsclient automount[583]: parse_mount: parse(sun): dequote("192.168.0.54:/nfs_export") -> 192.168.0.54:/nfs_export
Nov 25 03:06:00 vznfsclient automount[583]: parse_mount: parse(sun): core of entry: options=rw, loc=192.168.0.54:/nfs_export
Nov 25 03:06:00 vznfsclient automount[583]: sun_mount: parse(sun): mounting root /misc, mountpoint nfs, what 192.168.0.54:/nfs_export, fstype nfs, options rw
Nov 25 03:06:00 vznfsclient automount[583]: mount_mount: mount(nfs): root=/misc name=nfs what=192.168.0.54:/nfs_export, fstype=nfs, options=rw
Nov 25 03:06:00 vznfsclient automount[583]: mount_mount: mount(nfs): nfs options="rw", nobind=0, nosymlink=0, ro=0
Nov 25 03:06:00 vznfsclient automount[583]: get_nfs_info: called with host 192.168.0.54(192.168.0.54) proto 6 version 0x40
Nov 25 03:06:00 vznfsclient automount[583]: get_nfs_info: called with host 192.168.0.54(192.168.0.54) proto 6 version 0x70
Nov 25 03:06:00 vznfsclient automount[583]: get_nfs_info: nfs v3 rpc ping time: 0.000366
Nov 25 03:06:00 vznfsclient automount[583]: get_nfs_info: host 192.168.0.54 cost 365 weight 0
Nov 25 03:06:00 vznfsclient automount[583]: get_nfs_info: called with host 192.168.0.54(192.168.0.54) proto 17 version 0x70
Nov 25 03:06:00 vznfsclient automount[583]: get_nfs_info: nfs v3 rpc ping time: 0.000507
Nov 25 03:06:00 vznfsclient automount[583]: get_nfs_info: nfs v2 rpc ping time: 0.000692
Nov 25 03:06:00 vznfsclient automount[583]: get_nfs_info: host 192.168.0.54 cost 599 weight 0
Nov 25 03:06:00 vznfsclient automount[583]: prune_host_list: selected subset of hosts that support NFS3 over TCP
Nov 25 03:06:00 vznfsclient automount[583]: mount_mount: mount(nfs): calling mkdir_path /misc/nfs
Nov 25 03:06:00 vznfsclient automount[583]: mount_mount: mount(nfs): calling mount -t nfs -s -o rw 192.168.0.54:/nfs_export /misc/nfs
Nov 25 03:06:00 vznfsclient automount[583]: spawn_mount: mtab link detected, passing -n to mount
Nov 25 03:06:00 vznfsclient automount[583]: mount(nfs): mounted 192.168.0.54:/nfs_export on /misc/nfs
Nov 25 03:06:00 vznfsclient automount[583]: ioctl_send_ready: token = 28
Nov 25 03:06:00 vznfsclient automount[583]: mounted /misc/nfs
0

精彩评论

暂无评论...
验证码 换一张
取 消