uiScribe uiScribe - Custom System Log page for "scribed" logs

[QuikSilver]

0.4.0 might pleasantly surprise you

Wonder what's in it @L&LD?!? hehe

 

[QuikSilver]

IT's ALIVE!
and working on my RT-AC5300
View attachment 18163

I can imagine how the page would look if I had 30 or more filters......

@Jack Yaz

You are slowly taking over the GUI..

What is going to be the next adventure... ScriMerlin with a database with all the possible examlpes writable to /opt for ease of use ?

Mine doesn't show as many as yours does.....Hmmm...

 

[elorimer]

Updated to 0.4.0. Messages show fine. All the other ones only show "Log goes here". Some of the titles are logs I don't have. The bottom two have garbled titles: One is "osqueryd.results.log") prefix(".osquery.") ...) { (click to show/hide)"; the other is "pacct") follow-freq(1) ...) { (click to show/hide)"

This on an 87U with 12b1. Not trying to be discouraging, cuz this is nifty. I particularly like the way this builds on scribe, doesn't mess with it and allows me to customize with syslog-ng.

 

[Swistheater]

A small request for v1.0.

Mine doesn't show as many as yours does.....Hmmm...
View attachment 18166

I made extra filters.

 

[shark]

Updated from version 0.2.0 to 0.4.0 and i was getting "Log goes here" with these:

But then i tried uninstalling uiScribe and installing it again and it solved the problem.

 

[cmkelley]

I hid these as I wasn't sure what impact, if any, the settings had when running through syslog-ng. @cmkelley do you know if the settings still apply?

Okay, a little investigation by way of uninstalling uiScribe and changing the settings to see what gets passed to syslogd.

The "Log only messages more urgent than" number has no impact on scribe and can be hidden. That number is passed to syslogd which syslogd uses to toss out messages that priority or lower (the lowest is 7, so priority 8 logs everything). IIRC (i.e. I think I remember @RMerlin stating it somewhere) the "Default message log level" sets the priority level of messages generated by the router firmware. You can leave that exposed in so people can set that level in case they want to filter by priority level. None of the filters I provide with scribe filter by priority, but syslog-ng does support that. There are examples on the syslog-ng github.

 

[Swistheater]

Mine doesn't show as many as yours does.....Hmmm...
View attachment 18166

here are some extra base filters
use nano in ssh and
nano /jffs/scripts/syslog-ng.d-filters
copy and paste into terminal and ctrl-x to save

#!/bin/sh
cat > /opt/etc/syslog-ng.d/diversion <<'EOF'
# log all diversion and uiDivStats logs to diversion.log

destination d_diversion {
    file("/opt/var/log/diversion.log");
};

filter f_diversion {
    program("Diversion") or
    program("uiDivStats");
};

log {
    source(src);
    filter(f_diversion);
    destination(d_diversion);
    flags(final);
};

#eof
EOF
chmod 600 /opt/etc/syslog-ng.d/diversion

cat > /opt/etc/syslog-ng.d/ethernet <<'EOF'
# log ethernet change to /opt/var/log/ethernet.log only

destination d_ethernet {
    file("/opt/var/log/ethernet.log");
};

filter f_kernel {
    program("kernel");
};

filter f_ethernet {
    message("eth1") or
    message("eth2") or
    message("eth3") or
    message("eth4") or
    message("eth5") or
    message("eth6") or
    message("eth7") or
    message("eth8") or
    message("br0:");
};

log {
    source(src);
    filter(f_kernel);
    filter(f_ethernet);
    destination(d_ethernet);
    flags(final);
};

#eof
EOF
chmod 600 /opt/etc/syslog-ng.d/ethernet

cat > /opt/etc/syslog-ng.d/openvpn <<'EOF'
# log all openvpn server and client logs into one file - /opt/var/log/openvpn.log and stop processing openvpn logs

destination d_openvpn {
    file("/opt/var/log/openvpn.log");
};

filter f_openvpn {
    program("ovpn-server1") or
    program("ovpn-server2") or
    program("ovpn-client1") or
    program("ovpn-client2") or
    program("ovpn-client3") or
    program("ovpn-client4") or
    program("ovpn-client5") or
    program("openvpn-routing");
};

log {
    source(src);
    filter(f_openvpn);
    destination(d_openvpn);
    flags(final);
};

#eof
EOF
chmod 600 /opt/etc/syslog-ng.d/openvpn

cat > /opt/etc/syslog-ng.d/pixelserv <<'EOF'
# log all pixelserv-tls logs to /opt/var/log/pixelserv.log and stop processing pixelserv-tls logs

destination d_pixelserv {
    file("/opt/var/log/pixelserv.log");
};

filter f_pixelserv {
    program("pixelserv-tls");
};

log {
    source(src);
    filter(f_pixelserv);
    destination(d_pixelserv);
    flags(final);
};

#eof
EOF
chmod 600 /opt/etc/syslog-ng.d/pixelserv

then copy and paste sh /jffs/scripts/syslog-ng.d-filters

next the logrotate
same
nano /jffs/scripts/logrotate.d-filters

#!/bin/sh
cat > /opt/etc/logrotate.d/diversion <<'EOF'
/opt/var/log/diversion.log {
    postrotate
        /usr/bin/killall -HUP syslog-ng
    endscript
}
EOF
chmod 600 /opt/etc/logrotate.d/diversion

cat > /opt/etc/logrotate.d/ethernet <<'EOF'
/opt/var/log/ethernet.log {
    postrotate
        /usr/bin/killall -HUP syslog-ng
    endscript
}
EOF
chmod 600 /opt/etc/logrotate.d/ethernet

cat > /opt/etc/logrotate.d/openvpn <<'EOF'
/opt/var/log/openvpn.log {
    monthly
    rotate 9
    postrotate
        /usr/bin/killall -HUP syslog-ng
    endscript
}
EOF
chmod 600 /opt/etc/logrotate.d/openvpn

cat > /opt/etc/logrotate.d/pixelserv <<'EOF'
/opt/var/log/pixelserv.log {
    rotate 9
    postrotate
        /usr/bin/killall -HUP syslog-ng
    endscript
}
EOF
chmod 600 /opt/etc/logrotate.d/pixelserv

then sh /jffs/scripts/logrotate.d-filters

to top it off you can use options LR and RS inside scribe to get things started

force up date uiscribe as i am not sure if you will see new logs present yet

 

[cmkelley]

here are some extra base filters
use nano in ssh and
nano /jffs/scripts/syslog-ng.d-filters
copy and paste into terminal and ctrl-x to save

#!/bin/sh
cat > /opt/etc/syslog-ng.d/diversion <<'EOF'
# log all diversion and uiDivStats logs to diversion.log

destination d_diversion {
    file("/opt/var/log/diversion.log");
};

filter f_diversion {
    program("Diversion") or
    program("uiDivStats");
};

log {
    source(src);
    filter(f_diversion);
    destination(d_diversion);
    flags(final);
};

#eof
EOF
chmod 600 /opt/etc/syslog-ng.d/diversion

cat > /opt/etc/syslog-ng.d/ethernet <<'EOF'
# log ethernet change to /opt/var/log/ethernet.log only

destination d_ethernet {
    file("/opt/var/log/ethernet.log");
};

filter f_kernel {
    program("kernel");
};

filter f_ethernet {
    message("eth1") or
    message("eth2") or
    message("eth3") or
    message("eth4") or
    message("eth5") or
    message("eth6") or
    message("eth7") or
    message("eth8") or
    message("br0:");
};

log {
    source(src);
    filter(f_kernel);
    filter(f_ethernet);
    destination(d_ethernet);
    flags(final);
};

#eof
EOF
chmod 600 /opt/etc/syslog-ng.d/ethernet

cat > /opt/etc/syslog-ng.d/openvpn <<'EOF'
# log all openvpn server and client logs into one file - /opt/var/log/openvpn.log and stop processing openvpn logs

destination d_openvpn {
    file("/opt/var/log/openvpn.log");
};

filter f_openvpn {
    program("ovpn-server1") or
    program("ovpn-server2") or
    program("ovpn-client1") or
    program("ovpn-client2") or
    program("ovpn-client3") or
    program("ovpn-client4") or
    program("ovpn-client5") or
    program("openvpn-routing");
};

log {
    source(src);
    filter(f_openvpn);
    destination(d_openvpn);
    flags(final);
};

#eof
EOF
chmod 600 /opt/etc/syslog-ng.d/openvpn

cat > /opt/etc/syslog-ng.d/pixelserv <<'EOF'
# log all pixelserv-tls logs to /opt/var/log/pixelserv.log and stop processing pixelserv-tls logs

destination d_pixelserv {
    file("/opt/var/log/pixelserv.log");
};

filter f_pixelserv {
    program("pixelserv-tls");
};

log {
    source(src);
    filter(f_pixelserv);
    destination(d_pixelserv);
    flags(final);
};

#eof
EOF
chmod 600 /opt/etc/syslog-ng.d/pixelserv

then copy and paste sh /jffs/scripts/syslog-ng.d-filters

next the logrotate
same
nano /jffs/scripts/logrotate.d-filters

#!/bin/sh
cat > /opt/etc/logrotate.d/diversion <<'EOF'
/opt/var/log/diversion.log {
    postrotate
        /usr/bin/killall -HUP syslog-ng
    endscript
}
EOF
chmod 600 /opt/etc/logrotate.d/diversion

cat > /opt/etc/logrotate.d/ethernet <<'EOF'
/opt/var/log/ethernet.log {
    postrotate
        /usr/bin/killall -HUP syslog-ng
    endscript
}
EOF
chmod 600 /opt/etc/logrotate.d/ethernet

cat > /opt/etc/logrotate.d/openvpn <<'EOF'
/opt/var/log/openvpn.log {
    monthly
    rotate 9
    postrotate
        /usr/bin/killall -HUP syslog-ng
    endscript
}
EOF
chmod 600 /opt/etc/logrotate.d/openvpn

cat > /opt/etc/logrotate.d/pixelserv <<'EOF'
/opt/var/log/pixelserv.log {
    rotate 9
    postrotate
        /usr/bin/killall -HUP syslog-ng
    endscript
}
EOF
chmod 600 /opt/etc/logrotate.d/pixelserv

then sh /jffs/scripts/logrotate.d-filters

to top it off you can use options LR and RS inside scribe to get things started

force up date uiscribe as i am not sure if you will see new logs present yet

Er, all of those are at /opt/share/syslog-ng/examples and /opt/share/logrotate/examples You just need to copy them from the examples directory to the appropriate directory in /opt/etc - syslog.d or logrotate.d as required. No need to re-create them.

 

[Swistheater]

Er, all of those are at /opt/share/syslog-ng/examples and /opt/share/logrotate/examples You just need to copy them from the examples directory to the appropriate directory in /opt/etc - syslog.d or logrotate.d as required. No need to re-create them.

correct you can use cp.
i just like to make sure there is no other nuisances.
don't forget chmod 600 all the way all day.

 

[cmkelley]

correct you can use cp.
i just like to make sure there is no other nuisances.
don't forget chmod 600 all the way all day.

cp -p copies permissions and preserves date/time stamps. All of the (usable) files in those directories have their permissions set properly.

I really need to stop playing with it and write some documentation. It isn't as obvious as it seems to me.

 

[Swistheater]

cp -p copies permissions and preserves date/time stamps. All of the (usable) files in those directories have their permissions set properly.

I really need to stop playing with it and write some documentation. It isn't as obvious as it seems to me.

you did an awesome job, what are you talking about "playing with it".

 

[cmkelley]

@Jack Yaz, couple things (v0.4.0):

The "apply" button is broken. Pushing it has the effect of resetting the page entirely. If either checkbox has been cleared it's re-checked and the windows collapsed. Same behaviour on my AC86U and AC3200.

Just a data point, but updating from v0.1.0 didn't work right for some reason. I only got messages and crash logs showing, all the rest has the "log goes here" message. Uninstalling and reinstalling fixed it.

 

[cmkelley]

you did an awesome job, what are you talking about "playing with it".

Adding features. Working on the ability to stop syslog-ng (and restart syslogd/klogd) without uninstalling. Turning out to have a few unintended consequences and logic traps. I thought I was ready to roll out v2.1_0 last night, but wanted to bang on it some more and I found another issue tonight. :-(

Also, I have a day job that isn't coding, so my "free" time is limited and coding is more interesting than documenting, and being summer and all I need to ride my bike more as well.

 

[joe scian]

Hi Jack
Im getting "Log goes here" on everything except system messages - on v 0.2.0 and 0.4.0

 

[cmkelley]

Hi Jack
Im getting "Log goes here" on everything except system messages - on v 0.2.0 and 0.4.0

try uninstalling uiScribe and reinstalling.

 

[joe scian]

try uninstalling uiScribe and reinstalling.

Done that - no change

 

[Jack Yaz]

@Jack Yaz, couple things (v0.4.0):

The "apply" button is broken. Pushing it has the effect of resetting the page entirely. If either checkbox has been cleared it's re-checked and the windows collapsed. Same behaviour on my AC86U and AC3200.

Just a data point, but updating from v0.1.0 didn't work right for some reason. I only got messages and crash logs showing, all the rest has the "log goes here" message. Uninstalling and reinstalling fixed it.

The button will be disappearing again as you've confirmed the log level stuff doesn't apply.

I messed with some filenames since 0.1 so that's probably why - sorry!

 

[Jack Yaz]

Done that - no change

What does

 ls -la /www/ext/uiScribe
cat /jffs/scripts/uiScribe.d/.logs

produce?

Is syslog-ng definitely running? Check status in scribe

 

[Jack Yaz]

Okay, a little investigation by way of uninstalling uiScribe and changing the settings to see what gets passed to syslogd.

The "Log only messages more urgent than" number has no impact on scribe and can be hidden. That number is passed to syslogd which syslogd uses to toss out messages that priority or lower (the lowest is 7, so priority 8 logs everything). IIRC (i.e. I think I remember @RMerlin stating it somewhere) the "Default message log level" sets the priority level of messages generated by the router firmware. You can leave that exposed in so people can set that level in case they want to filter by priority level. None of the filters I provide with scribe filter by priority, but syslog-ng does support that. There are examples on the syslog-ng github.

I've removed the filtering options in 0.4.1 - those wanting to filter by priority should set this up in syslog-ng

 

[joe scian]

What does

 ls -la /www/ext/uiScribe
cat /jffs/scripts/uiScribe.d/.logs

produce?

Is syslog-ng definitely running? Check status in scribe

Yes definitely running

wx    1 joescian root            26 May  5  2018 logrotate.log.htm -> /opt/var/log/logrotate.log
lrwxrwxrwx    1 joescian root            30 May  5  2018 logs.htm -> /jffs/scripts/uiScribe.d/.logs
lrwxrwxrwx    1 joescian root            21 May  5  2018 messages.htm -> /opt/var/log/messages
lrwxrwxrwx    1 joescian root            24 May  5  2018 openvpn.log.htm -> /opt/var/log/openvpn.log
lrwxrwxrwx    1 joescian root            26 May  5  2018 pixelserv.log.htm -> /opt/var/log/pixelserv.log
lrwxrwxrwx    1 joescian root            25 May  5  2018 skynet-0.log.htm -> /opt/var/log/skynet-0.log
lrwxrwxrwx    1 joescian root            26 May  5  2018 syslog-ng.log.htm -> /opt/var/log/syslog-ng.log
lrwxrwxrwx    1 joescian root            26 May  5  2018 wlceventd.log.htm -> /opt/var/log/wlceventd.log
joescian@RT-AC5300:/tmp/home/root#
joescian@RT-AC5300:/tmp/home/root# cat /jffs/scripts/uiScribe.d/.logs
/opt/var/log/crash.log
/opt/var/log/diversion.log
/opt/var/log/ethernet.log
file("/opt/var/log/expanded-syslog.log" template(t_expandlog));
/opt/var/log/logrotate.log
/opt/var/log/openvpn.log
/opt/var/log/pixelserv.log
/opt/var/log/skynet-0.log
/opt/var/log/syslog-ng.log
/opt/var/log/wlceventd.log

 syslog-ng and logrotate installation
     v2.0_1 (master)  Coded by cynicastic


 checking system for necessary scribe hooks ...

          checking S01syslog-ng ... present.

         checking service-event ... present.

            checking post-mount ... present.

    checking logrotate cron job ... present.

 checking syslog-ng configuration ...

    syslog-ng.conf syntax check ... okay!

      checking syslog-ng daemon ... alive.

          scribe installed version: v2.0_1 (master)
             scribe GitHub version: v2.0_1 (master)
                    scribe is up to date!