Entware Pi-hole directly on the router? Yes!

[ColinTaylor]

It seems if the taint happens once, then the kernel knows it's tainted. At least that's what is out there in other explanations. But why does this kernel have to announce the taint at every turn spamming the log with all those messages?



It's puzzling that PiHole cannot invoke these kernel functions from the regular kernel code, and needs something that comes from a proprietary module somewhere. I would understand if this was some fancy functionality, but the regular shared memory stuff?

You're not understanding what RMerlin said. The entire kernel is tainted (and always will be) because it contains proprietary and out-of-tree modules. Ignore the word "Tainted", that's irrelevant. It's just one small part of the dumped information.

Aug 15 15:55:57 RT-AC86U-9988 kernel: pihole-FTL[37573]: syscall 403
Aug 15 15:55:57 RT-AC86U-9988 kernel: Code: 4620d02f f2404629 463a1393 df00461f (f1104617)
Aug 15 15:55:57 RT-AC86U-9988 kernel: CPU: 1 PID: 37573 Comm: pihole-FTL Tainted: P           O    4.1.27 #2
Aug 15 15:55:57 RT-AC86U-9988 kernel: Hardware name: Broadcom-v8A (DT)
Aug 15 15:55:57 RT-AC86U-9988 kernel: task: ffffffc0173b4080 ti: ffffffc01e86c000 task.ti: ffffffc01e86c000
Aug 15 15:55:57 RT-AC86U-9988 kernel: PC is at 0x2f3520
Aug 15 15:55:57 RT-AC86U-9988 kernel: LR is at 0x2f4349
Aug 15 15:55:57 RT-AC86U-9988 kernel: pc : [<00000000002f3520>] lr : [<00000000002f4349>] pstate: 60000030
Aug 15 15:55:57 RT-AC86U-9988 kernel: sp : 00000000fff07f60
Aug 15 15:55:57 RT-AC86U-9988 kernel: x12: 0000000000367bac
Aug 15 15:55:57 RT-AC86U-9988 kernel: x11: 0000000000000000 x10: 0000000000000000
Aug 15 15:55:57 RT-AC86U-9988 kernel: x9 : 0000000000302600 x8 : 0000000000000000
Aug 15 15:55:57 RT-AC86U-9988 kernel: x7 : 0000000000000193 x6 : 000000000001015d
Aug 15 15:55:57 RT-AC86U-9988 kernel: x5 : 00000000fff07f78 x4 : 0000000000000000
Aug 15 15:55:57 RT-AC86U-9988 kernel: x3 : 0000000000000193 x2 : 00000000fff07fa8
Aug 15 15:55:57 RT-AC86U-9988 kernel: x1 : 00000000fff07f78 x0 : 0000000000000000

 

[jacklul]

It's puzzling that PiHole cannot invoke these kernel functions from the regular kernel code, and needs something that comes from a proprietary module somewhere. I would understand if this was some fancy functionality, but the regular shared memory stuff?

One of the proprietary modules might be wrapping around those functions, I'm not an expert here.

 

[bibikalka]

You're not understanding what RMerlin said. The entire kernel is tainted (and always will be) because it contains proprietary and out-of-tree modules. Ignore the word "Tainted", that's irrelevant. It's just one small part of the dumped information.
...

OK, fair enough. But what is that all other info it's dumping? It does not appear to claim that there is an error, at least, no explicit messages to that effect.

 

[bibikalka]

One of the proprietary modules might be wrapping around those functions, I'm not an expert here.

It is a bit unfortunate to see so much output in the log due to this proprietary modules issue. So sounds like I shall continue filtering those messages via a custom filter:

# cat /opt/etc/syslog-ng.d/bcm_err_vA

filter z_bcm_err {
    ( program("kernel") and
    ( message("^x[0-9]") or
     message("^PC") or
     message("^LR") or
     message("^pc") or
     message("^sp") or
     message("^task") or
     message("^Code:") or
     message("^CPU:") or
     message("^dig") or
     message("^DROP") or
     message("^'\[00000000\]") or
     message("potentially") or
     message("pgd") or
     message("send_redir_page") or
     message("bcm_mcast_mld_add") or
     message("already exist in UDB") or
     message("not mesh client") or
     message("syscall") or
     message("^Hardware")) );
};

log {
    source(src);
    filter(z_bcm_err);
    flags(final);
};

#eof

 

[ColinTaylor]

OK, fair enough. But what is that all other info it's dumping? It does not appear to claim that there is an error, at least, no explicit messages to that effect.

The error is in the first line, e.g. "syscall 403". The PID and program name is on the third line. The rest of the information is probably of no use to you unless you're running the program in a debugging environment.

 

[bibikalka]

The error is in the first line, e.g. "syscall 403". The PID and program name is on the third line. The rest of the information is probably of no use to you unless you're running the program in a debugging environment.

It does not say that there is an error. Seems just stating the syscal number. Why would every invocation end up as an error?

 

[jacklul]

It does not say that there is an error. Seems just stating the syscal number. Why would every invocation end up as an error?

Those are error messages, they show which syscall caused the error and in which process.
Process did something that kernel couldn't execute.
Rest of the info contains machine code that caused the stop and CPU registers state, only useful for debugging.

If that's a bug in Pi-hole's FTL code then it's up to developers to fix it, provided you can reproduce this issue on another machine.

 

[Martinski]

It does not say that there is an error. Seems just stating the syscal number. Why would every invocation end up as an error?

What you're getting are crash reports, which are an indication that something *is* going wrong repeatedly during the module execution. A crash dump will not necessarily or explicitly say "error," but the crash report itself shows that there's clearly a problem (e.g. syscall 403).

Essentially, a crash dump shows some crucial information WRT the state of the system when the crash occurred. This includes the values of CPU registers at the time of the crash. For example, the Program Counter (PC) has the address of the next instruction to be executed, or sometimes it may contain the address of the instruction causing the crash. The Link Register (LR) has the return address where the running process should resume after the current function completes execution; this is important as it identifies the function that called the module or routine where the crash occurred. The Stack Pointer (SP) points to the top of the stack in memory. Other general-purpose registers (e.g. x0-xN) store relevant information. There are also important kernel data structure addresses, like task_struct (task) and thread_info (ti), which provide the kernel's internal representation of processes and threads when the crash happened.

Of course, none of the above information is really useful or meaningful unless you're running a debugger (like gdb) along with the raw source code at the time that the crash occurs.

In any case, it's clear that a runtime execution error is occurring.

 

[jacklul]

@bibikalka If you're feeling adventurous - I patched the source code to log a critical error everytime gettimeofday() fails - download "packages" artifact from here (you might need an account) then unpack and install the appropriate ipk (opkg install asdf.ipk), you might have to uninstall the current package first.
Note that this package is based on development branch, I suggest you backup the configuration file and gravity.db before trying this out.
Most of the messages should be logged to /opt/var/log/pihole/FTL.log and/or pihole.log, some might end up in the syslog.
It will log a line like this: "<source function>: gettimeofday() failed: <error message>" so grep'ing it by "gettimeofday() failed:" should work.

 

[bibikalka]

@bibikalka If you're feeling adventurous - I patched the source code to log a critical error everytime gettimeofday() fails - download "packages" artifact from here (you might need an account) then unpack and install the appropriate ipk (opkg install asdf.ipk), you might have to uninstall the current package first.
Note that this package is based on development branch, I suggest you backup the configuration file and gravity.db before trying this out.
Most of the messages should be logged to /opt/var/log/pihole/FTL.log and/or pihole.log, some might end up in the syslog.
It will log a line like this: "<source function>: gettimeofday() failed: <error message>" so grep'ing it by "gettimeofday() failed:" should work.

OK - but what would this accomplish? Is there any diagnostic value in this?

If this were to be addressed, is not the key question why Merlin's kernel does not like PiHole's requests even though something like a Debian kernel handles those just fine?

Debian kernel is quite a bit newer, but these seem to be rather old functions, so why are they not working?

My proxmox kernel:
Linux pihole 6.8.12-13-pve 1 SMP PREEMPT_DYNAMIC PMX 6.8.12-13 (2025-07-22T10:00Z) x86_64 GNU/Linux

 

[jacklul]

OK - but what would this accomplish? Is there any diagnostic value in this?

If this were to be addressed, is not the key question why Merlin's kernel does not like PiHole's requests even though something like a Debian kernel handles those just fine?

Debian kernel is quite a bit newer, but these seem to be rather old functions, so why are they not working?

My proxmox kernel:
Linux pihole 6.8.12-13-pve 1 SMP PREEMPT_DYNAMIC PMX 6.8.12-13 (2025-07-22T10:00Z) x86_64 GNU/Linux

It might be because Pi-hole is targeted at newer kernels, though I have not observed such issues on my router which is running 4.19.183.
It's up to you if you want to keep looking at it, I personally would not be able to just ignore those errors and sleep well.

 

[SomeWhereOverTheRainBow]

It might be because Pi-hole is targeted at newer kernels, thought I have not observed such issues on my router which is running 4.19.183.

Could it also be because of how Pihole itself operates? it typically operates as its own user "pihole", but sometimes tries performs root level tasks. it may require a fakeroot (or fakeroot-ng) in that sense-to grant it the ability to perform these system calls. Idk, This is just conjecture. I am only seeking to better understand like yourself.

 

[jacklul]

Could it also be because of how Pihole itself operates? it typically operates as its own user "pihole", but sometimes tries performs root level tasks. it may require a fakeroot (or fakeroot-ng) in that sense-to grant it the ability to perform these system calls. Idk, This is just conjecture. I am only seeking to better understand like yourself.

They switched to using setcap long time ago (they used privileged user before), so you can now basically run it as any user.
You can of course run it as pihole user as instructed on the wiki, you just need to add the "pihole" user in USB->Samba (or passwd.add), init script will pick it up and use it instead of the nobody user.

 

[bibikalka]

They switched to using setcap long time ago (they used privileged user before), so you can now basically run it as any user.
You can of course run it as pihole user as instructed on the wiki, you just need to add the "pihole" user in USB->Samba (or passwd.add), init script will pick it up and use it instead of the nobody user.

This was an easy test, running as "pihole" user. No impact though, still the same errors.

#ps wT |grep pihole
1907297 pihole   10900 S <  pihole-FTL -- -u pihole -g pihole
1907357 pihole   10900 S <  {database} pihole-FTL -- -u pihole -g pihole
1907358 pihole   10900 S <  {housekeeper} pihole-FTL -- -u pihole -g pihole
1907359 pihole   10900 S <  {dns-client} pihole-FTL -- -u pihole -g pihole
1907360 pihole   10900 S <  {timer} pihole-FTL -- -u pihole -g pihole
1907362 pihole   10900 S N  {civetweb-timer} pihole-FTL -- -u pihole -g pihole
1907363 pihole   10900 S N  {civetweb-master} pihole-FTL -- -u pihole -g pihole
1910242 admin     5980 S    grep pihole

I don't want to ignore the errors - just not clear what to do ...

I'd test this as the sole DNS provider, but I am testing this on a production router, with limited time windows I can be relatively undisturbed. So not the best setup for this ...

 

[bibikalka]

@jacklul

Just to double check, is this PiHole compiled as 32 or 64 bits?

The kernel is this:
Linux RT-AC86U-9988 4.1.27 SMP PREEMPT Sun Nov 17 14:59:02 EST 2024 aarch64 ASUSWRT-Merlin

Could there be some sort of bitness mismatch somewhere when calling kernel functions?

 

[SomeWhereOverTheRainBow]

They switched to using setcap long time ago (they used privileged user before), so you can now basically run it as any user.
You can of course run it as pihole user as instructed on the wiki, you just need to add the "pihole" user in USB->Samba (or passwd.add), init script will pick it up and use it instead of the nobody user.

Yea, I am not too concerned about the ownership, more so how the system observes the calls.

 

[jacklul]

@jacklul

Just to double check, is this PiHole compiled as 32 or 64 bits?

The kernel is this:
Linux RT-AC86U-9988 4.1.27 SMP PREEMPT Sun Nov 17 14:59:02 EST 2024 aarch64 ASUSWRT-Merlin

Could there be some sort of bitness mismatch somewhere when calling kernel functions?

If you have aarch64 Entware setup then it would be 64 bit arm.

Yea, I am not too concerned about the ownership, more so how the system observes the calls.

I suppose a way to test this would be running it as root/admin user.

 

[bibikalka]

If you have aarch64 Entware setup then it would be 64 bit arm.

I suppose a way to test this would be running it as root/admin user.

Yes, I could try this as a root - probably tomorrow. Just to see if that fixes the issues. What do I need to change?

 

[bibikalka]

If you have aarch64 Entware setup then it would be 64 bit arm.

I installed file command, and :

#opkg install file
/opt/bin# file pihole-FTL
pihole-FTL: ELF 32-bit LSB executable, ARM, EABI5 version 1 (SYSV), statically linked, BuildID[sha1]=949b28711603e2a5165cf8a54026c625fe00f100, with debug_info, not stripped

So it does not appear to be a 64 bit binary? And could static linking be an issue for something that invokes kernel functions?

Package locations appear to be correct:

# cat /opt/etc/opkg.conf
src/gz entware https://entware.diversion.ch/aarch64-k3.10
src/gz pi-hole https://jacklul.github.io/entware-pi-hole/aarch64-k3.10
dest root /
dest ram /opt/tmp
lists_dir ext /opt/var/opkg-lists
option tmp_dir /opt/tmp
arch all 100
arch aarch64-3.10 160

Other things are 64 bit indeed:

admin@RT-AC86U-9988:/tmp/mnt/ac86u/entware/bin# file bash
bash: ELF 64-bit LSB executable, ARM aarch64, version 1 (SYSV), dynamically linked, interpreter /opt/lib/ld-linux-aarch64.so.1, for GNU/Linux 3.7.0, stripped
admin@RT-AC86U-9988:/tmp/mnt/ac86u/entware/bin# file curl
curl: ELF 64-bit LSB executable, ARM aarch64, version 1 (SYSV), dynamically linked, interpreter /opt/lib/ld-linux-aarch64.so.1, for GNU/Linux 3.7.0, stripped

 

[jacklul]

Turns out... I had a bug in my build script and it never properly deleted the binary from the previously built package, making it use the same binary for all packages...
Update the package and let me know if the issue is gone now.