Solved [SOLUTION] Asuswrt-Merlin NFS v3 File Locking using NSM (statd) & NLM (lockd)

[garycnew]

All:

I've been successfully using NFS v3 without File Locking for quite some time.

I've reached a point where I need to in institute NSM (statd) & NLM (lockd).

I've noticed that statd and lockd are running on the Primary Router (NFS v3 Server).

However, when I attempt to remotely mount the export shares using the lock option, I receive the following errors:

kernel: svc: failed to register lockdv1 RPC service (errno 111).
kernel: lockd_up: makesock failed, error=-111

Has anyone successfully implemented NFS v3 Locking on Asuswrt-Merlin?

Respectfully,


Gary
--
Asus RT-AC66U_B1 (384.19)

 

[ColinTaylor]

Seems to work for me.

# mount -t nfs -o nfsvers=3,lock 192.168.1.1:/tmp/mnt/ClickUSB1/ASUS /cdrom

# mount | grep "ASUS"
192.168.1.1:/tmp/mnt/ClickUSB1/ASUS on /cdrom type nfs (rw,relatime,vers=3,rsize=131072,wsize=131072,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,mountaddr=192.168.1.1,mountvers=3,mountport=59315,mountproto=udp,local_lock=none,addr=192.168.1.1)

 

[garycnew]

Seems to work for me.

# mount -t nfs -o nfsvers=3,lock 192.168.1.1:/tmp/mnt/ClickUSB1/ASUS /cdrom

# mount | grep "ASUS"
192.168.1.1:/tmp/mnt/ClickUSB1/ASUS on /cdrom type nfs (rw,relatime,vers=3,rsize=131072,wsize=131072,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,mountaddr=192.168.1.1,mountvers=3,mountport=59315,mountproto=udp,local_lock=none,addr=192.168.1.1)

@ColinTaylor

Would you mind sharing the contents of your exports or /jffs/configs/exports.add file?

Thanks, again!


Gary

 

[ColinTaylor]

# cat /etc/exports
/mnt/ClickUSB1/ASUS *(no_root_squash,rw,sync)

 

[garycnew]

# cat /etc/exports
/mnt/ClickUSB1/ASUS *(no_root_squash,rw,sync)

Would you mind providing the ps results of the statd and lockd processes on the NFS Client & Server Hosts?

Much Appreciated!

 

[ColinTaylor]

Server:

admin@RT-AX86U:/# ps w | egrep "statd|lockd" | grep -v grep
   34 admin        0 SW<  [kblockd]
23107 admin     1900 S    /usr/sbin/statd
23110 admin        0 SW   [lockd]

Client:

root@nuc:~# rpcinfo -p 192.168.1.1
   program vers proto   port  service
    100000    2   tcp    111  portmapper
    100000    2   udp    111  portmapper
    100024    1   udp  45908  status
    100024    1   tcp  35068  status
    100003    2   tcp   2049  nfs
    100003    3   tcp   2049  nfs
    100003    2   udp   2049  nfs
    100003    3   udp   2049  nfs
    100021    1   udp  46365  nlockmgr
    100021    3   udp  46365  nlockmgr
    100021    4   udp  46365  nlockmgr
    100021    1   tcp  59921  nlockmgr
    100021    3   tcp  59921  nlockmgr
    100021    4   tcp  59921  nlockmgr
    100005    1   udp  60531  mountd
    100005    1   tcp  59775  mountd
    100005    2   udp  60531  mountd
    100005    2   tcp  59775  mountd
    100005    3   udp  60531  mountd
    100005    3   tcp  59775  mountd

 

[garycnew]

@ColinTaylor

Server:

# ps w | egrep "statd|lockd" | grep -v grep
   13 admin        0 SW<  [kblockd]
8589 admin      712 S    /usr/sbin/statd
8593 admin        0 SW   [lockd]

# ps w | egrep "nfs" | grep -v grep
8594 admin        0 SW   [nfsd]
8595 admin        0 SW   [nfsd]
8596 admin        0 SW   [nfsd]
8597 admin        0 SW   [nfsd]
8598 admin        0 SW   [nfsd]
8599 admin        0 SW   [nfsd]
8600 admin        0 SW   [nfsd]
8601 admin        0 SW   [nfsd]

# /usr/sbin/statd -v
statd version 1.3.3

# /usr/sbin/nfsd --help
Usage:
nfsd [-d|--debug] [-H hostname] [-p|-P|--port port]
     [-N|--no-nfs-version version] [-V|--nfs-version version]
     [-s|--syslog] [-T|--no-tcp] [-U|--no-udp] [-r|--rdma=]
     [-G|--grace-time secs] [-L|--leasetime secs] nrservs

# lsmod | grep nfsd
nfsd                   81901 11
exportfs                2959  1 nfsd
lockd                  58157  1 nfsd
sunrpc                161418 11 nfsd,lockd

Client:
[Note: My rpcinfo is missing the service names, but the associated ports appear to be similar. It does appear you're running nfs v2 & v3. Do you have "Enable legacy (NFS V2) support" in the WebUI? I have v2 disabled in the WebUI. It's my understanding that lockd wasn't introduced, until nfs v3.]

# rpcinfo -p 192.168.0.1
   program vers proto   port  service
    100000    2   tcp    111
    100000    2   udp    111
    100024    1   udp  60127
    100024    1   tcp  60459
    100003    3   tcp   2049
    100003    3   udp   2049
    100021    1   udp  46714
    100021    3   udp  46714
    100021    4   udp  46714
    100021    1   tcp  39974
    100021    3   tcp  39974
    100021    4   tcp  39974
    100005    3   udp  38569
    100005    3   tcp  16667

What version of nfsd and statd are you running? What nfsd kernel modules do you have loaded?

Thank you for your assistance.

Respectfully,


Gary

 

[ColinTaylor]

Yes I'm running with legacy (NFS V2) support enabled but I also tested without it and the results were the same.

# /usr/sbin/statd -v
statd version 1.3.3

# /usr/sbin/nfsd -v
/usr/sbin/nfsd: invalid option -- 'v'
Invalid argument: '?'
Usage:
nfsd [-d|--debug] [-H hostname] [-p|-P|--port port]
     [-N|--no-nfs-version version] [-V|--nfs-version version]
     [-s|--syslog] [-T|--no-tcp] [-U|--no-udp] [-r|--rdma=]
     [-G|--grace-time secs] [-L|--leasetime secs] nrservs

# lsmod | grep nfsd
nfsd                   95493 11
lockd                  69480  1 nfsd
sunrpc                202215  9 nfsd,lockd
exportfs                3743  1 nfsd

What command are you using on the client to mount the share? What OS is it running?

 

[garycnew]

What command are you using on the client to mount the share? What OS is it running?

I'm mounting it to an AiMesh Node. It mounts fine with the nolock option.

admin@Office-3C73:/tmp/home/root/# /bin/mount -t nfs -o _netdev,rw,exec,hard,intr,nolock 192.168.0.1:/tmp/mnt/SanDiskSDHC/tmp/tor /tmp/tor

 

[ColinTaylor]

Try it after entering this on the client:

modprobe nfs

 

[garycnew]

Try it after entering this on the client:

modprobe nfs

Same error after modprobe nfs on the Client AiMesh Node:

admin@Office-3C73:/tmp/home/root/# modprobe nfs

admin@Office-3C73:/tmp/home/root/# lsmod | grep nfs
nfs                   139801  1
lockd                  58157  1 nfs
sunrpc                161418  8 nfs,lockd

Mar  6 18:44:49 192.168.0.1 mountd[28070]: authenticated mount request from 192.168.0.31:777 for /tmp/mnt/SanDiskSDHC/tmp/tor (/tmp/mnt/SanDiskSDHC/tmp/tor)
Mar  6 18:44:49 Office-3C73-CA04B43-R kernel: svc: failed to register lockdv1 RPC service (errno 111).

 

[ColinTaylor]

EDIT: You could also try modprobe nfsv3 (that's normally done automatically when you mount the share).


Ah, OK. I see you've not enabled the NFS server on the client (yes, really).

Asus routers don't officially support NFS client mode. As such they're missing some things by default. There was a discussion about getting the client to work in this thread. I don't know how much of that is relevant to your device as it's running such an old version of the firmware.

In that other thread I suggested that the user enable the NFS server to save some extra work and then run modprobe nfs. If you don't want to run the server then you'd probably have to do stuff manually. Another user in that thread provided this script for that. Some of it's only applicable to HND models.

#!/bin/sh
modprobe -a exportfs sunrpc grace lockd nfsd nfs nfsv2 nfsv3

[ -e /var/lib/nfs ]       || mkdir -p /var/lib/nfs
[ -e /var/lib/nfs/etab ]  || mkdir -p /var/lib/nfs/etab
[ -e /var/lib/nfs/xtab ]  || mkdir -p /var/lib/nfs/xtab
[ -e /var/lib/nfs/rmtab ] || mkdir -p /var/lib/nfs/rmtab
chmod 0644 /var/lib/nfs/*tab

killall -0 portmap 2>/dev/null || portmap
killall -0 statd   2>/dev/null || statd
killall -0 nfsd    2>/dev/null || nfsd
killall -0 mountd  2>/dev/null || mountd

touch /etc/exports
exportfs -a

 

[garycnew]

EDIT: You could also try modprobe nfsv3 (that's normally done automatically when you mount the share).

Doesn't seem to be available on my firmware:

admin@Office-3C73:/tmp/home/root/# modprobe nfsv3
modprobe: module nfsv3 not found in modules.dep

Ah, OK. I see you've not enabled the NFS server on the client (yes, really).

What if I enabled nfsd_enable=1 on the Client AiMesh Node?

admin@Office-3C73:/tmp/home/root/# nvram set nfsd_enable=1
admin@Office-3C73:/tmp/home/root/# nvram commit
admin@Office-3C73:/tmp/home/root/# reboot

Asus routers don't officially support NFS client mode. As such they're missing some things by default. There was a discussion about getting the client to work in this thread. I don't know how much of that is relevant to your device as it's running such an old version of the firmware.

In that other thread I suggested that the user enable the NFS server to save some extra work and then run modprobe nfs. If you don't want to run the server then you'd probably have to do stuff manually. Another user in that thread provided this script for that. Some of it's only applicable to HND models.

I think I'd rather try enabling the NFS server to save on extra work, if it will load the necessary NFS resources?

 

[ColinTaylor]

Doesn't seem to be available on my firmware:

admin@Office-3C73:/tmp/home/root/# modprobe nfsv3
modprobe: module nfsv3 not found in modules.dep

That's possibly your problem. You talked about nfsv3 and when I mount a share on my router it loads that module.

# lsmod | grep nfs
nfsd                   95493 11
lockd                  69480  1 nfsd
sunrpc                202215  9 nfsd,lockd
exportfs                3743  1 nfsd
# modprobe nfs
# mount -t nfs -o _netdev,rw,exec,hard,intr,lock 192.168.1.1:/tmp/mnt/ClickUSB1/ASUS /cifs1
# lsmod | grep nfs
nfsv3                  23737  1
nfs                   136361  2 nfsv3
nfsd                   95493 11
lockd                  69480  3 nfsv3,nfs,nfsd
sunrpc                202215 17 nfsv3,nfs,nfsd,lockd
exportfs                3743  1 nfsd

What if I enabled nfsd_enable=1 on the Client AiMesh Node?

Sorry, I've never used AiMesh.

 

[garycnew]

Hmm... Even with nfsd running on the Client AiMesh Node, I still can't load the nfsv3 module and I receive the same NFS mount error:

admin@Office-3C73:/tmp/home/root/# lsmod | grep nfs
nfs                   139801  1
lockd                  58157  1 nfs
sunrpc                161418  8 nfs,lockd

admin@Office-3C73:/tmp/home/root/# modprobe nfsd

admin@Office-3C73:/tmp/home/root/# lsmod | grep nfs
nfsd                   81901  0
exportfs                2959  1 nfsd
nfs                   139801  1
lockd                  58157  2 nfsd,nfs
sunrpc                161418  9 nfsd,nfs,lockd

admin@Office-3C73:/tmp/home/root/# modprobe nfsv3
modprobe: module nfsv3 not found in modules.dep

admin@Office-3C73:/tmp/home/root/# /bin/mount -t nfs -o _netdev,rw,exec,hard,intr,lock 192.168.0.1:/tmp/mnt/SanDiskSDHC/tmp/tor /tmp/tor
mount: mounting 192.168.0.1:/tmp/mnt/SanDiskSDHC/tmp/tor on /tmp/tor failed: Connection refused

I have NFS v2 disabled, so it must be connecting using NFS v3 using the nolock option.

 

[garycnew]

@ColinTaylor

UPDATE: I was able to get NFS v3 locking to work on the AiMesh Nodes by implementing my previously proposed solution (it needed the reboot):

# nvram set nfsd_enable=1
# nvram commit
# reboot

NOTE: On Asuswrt-Merlin 384.19 (kernel-2.6), the nfsv3 module is never installed:

# lsmod | grep nfs
nfs                   139801  1 
nfsd                   81901 11 
exportfs                2959  1 nfsd
lockd                  58157  2 nfs,nfsd
sunrpc                161418 14 nfs,nfsd,lockd

However, the NFS lock option is instantiated within the mount:

# /bin/mount -t nfs -o _netdev,rw,exec,hard,intr,lock 192.168.0.1:/tmp/mnt/SanDiskSDHC/tmp/tor /tmp/tor
# mount | grep nfs
192.168.0.1:/tmp/mnt/SanDiskSDHC/tmp/tor on /tmp/tor type nfs (rw,relatime,vers=3,rsize=32768,wsize=32768,namlen=255,hard,proto=tcp,port=65535,timeo=70,retrans=3,sec=sys,addr=192.168.0.1)

Thanks, once again, for your assistance.

Respectfully,


Gary

 

[garycnew]

@ColinTaylor

Would you have any recommendations for the best /jffs/script to mount the NFS volume on the AiMesh Nodes?

Without NFS v3 locking, I was able to mount the NFS volume using /jffs/scipts/init-start. Now, that the AiMesh Nodes have to wait for the nfsd kernel module to load, init-start is way to early in the boot process.

Thanks, again.

Respectfully,


Gary

 

[ColinTaylor]

init-start runs at the beginning of the boot process. How about using services-start which runs at the end?

User scripts

Third party firmware for Asus routers (newer codebase) - RMerl/asuswrt-merlin.ng

github.com

 

[garycnew]

init-start runs at the beginning of the boot process. How about using services-start which runs at the end?

User scripts

Third party firmware for Asus routers (newer codebase) - RMerl/asuswrt-merlin.ng

github.com

@ColinTaylor

Adding the NFS v3 locking mount to the /jffs/scripts/services-start still doesn't give the nfs kernel modules enough time to load, so I've tried adding a sleep 60 prior to the NFS v3 locking mount command hoping it will provide sufficient time.

I'll monitor the next AiMesh Node reboot and update this post.

Respectfully,


Gary

 

[ColinTaylor]

It's awkward getting the timing right with USB based services because there aren't dedicated user hooks for all the services. I had the same problem as you trying to shutdown mdns. Like you I resorted using a sleep in services-start but it was aesthetically displeasing. In the end I used a service-event-end script that triggers on the restart of nasapps (assuming you have a USB storage device plugged in).

#!/bin/sh

if [ "$1" = "restart" ] && [ "$2" = "nasapps" ]; then
    logger -t "$(basename $0)" "Stopping mdns"
    service stop_mdns
fi

The issue with this approach for you might be that it can run multiple times on startup.