[rt-ac86u] ioctl command wrong in 64 bit system from 32 bit user code

[Fitz Mutch]

I apologize in advance for posting this to Merlin's forum because the issue is not about his firmware. However... some people will probably encounter this issue at some point.

I have a program that talks to a driver using ioctl. It works perfectly when both the driver and the program are 64-bit. However, when the driver is 64-bit, and my program is 32-bit, the ioctl command fails with NOTTY (Operation not supported).

Specifically, I'm trying to figure why NEMA+PPS doesn't work on my RT-AC86U running the NTP server compiled as a 32-bit program. To debug the situation, I have compiled pps-tools to run on my RT-AC86U which has a 64-bit kernel.

Compiled pps-tools as 64-bit program on my RT-AC86U (it works)

# modprobe pps_ktimer
# ppstest /dev/pps0
trying PPS source "/dev/pps0"
sizeof(int) is 0x00000004
sizeof(unsigned long int) is 0x00000008
sizeof(struct pps_kparams *) is 0x00000008
sizeof(struct pps_kparams) is 0x00000028
PPS_GETPARAMS is 0x00000000800870a1
ioctl(0x00000003, PPS_GETPARAMS, 0x00000000e5436b78) returned 0x00000000
found PPS source "/dev/pps0"
ok, found 1 source(s), now start fetching data...
source 0 - assert 1517759031.228048780, sequence: 7177 - clear  0.000000000, sequence: 0
source 0 - assert 1517759032.228048580, sequence: 7178 - clear  0.000000000, sequence: 0
source 0 - assert 1517759033.228049080, sequence: 7179 - clear  0.000000000, sequence: 0
source 0 - assert 1517759034.228050300, sequence: 7180 - clear  0.000000000, sequence: 0
source 0 - assert 1517759035.228050160, sequence: 7181 - clear  0.000000000, sequence: 0

Compiled pps-tools as 32-bit program on my RT-AC86U (does not work)

# modprobe pps_ktimer
# ppstest /dev/pps0
trying PPS source "/dev/pps0"
sizeof(int) is 0x00000004
sizeof(unsigned long int) is 0x00000004
sizeof(struct pps_kparams *) is 0x00000004
sizeof(struct pps_kparams) is 0x00000028
PPS_GETPARAMS is 0x800470a1
ioctl(0x00000003, PPS_GETPARAMS, 0xffe55138) returned 0xffffffff
cannot create a PPS source from device "/dev/pps0" (Operation not supported)

Lastly, I don't wish to compile NTP server as a 64-bit program because then all the libraries need to be 64-bit as well. It's a tough b*tch.