What's new

Scribe scribe - syslog-ng and logrotate installer

  • SNBForums Code of Conduct

    SNBForums is a community for everyone, no matter what their level of experience.

    Please be tolerant and patient of others, especially newcomers. We are all here to share and learn!

    The rules are simple: Be patient, be nice, be helpful or be gone!

Me too, and still getting Skynet hourly [save] lines in Loggy, just nowhere on my router that I can find. o_O

I know they have to be *somewhere* or they would never get sent to Loggly. I just have no idea where.
If you set it up as I suggested, it is the first to process any log message, and because there is no final flag, the message falls to the rest of the configs, in alphabetic order, and if it isn't pulled out by one of them, to messages. If it is pulled out to the skynet-0 log file (likely), then skynet will operate on it. When skynet purges the log, the documentation says it deletes the hourly message too. Then it writes a new hourly summary that ends up in the skynet-0 log. That doesn't happen to me, but I'm skipping some hourly messages.

The clue has to be here:
Code:
Purge_Logs () {
        sed '\~BLOCKED -~!d' "$syslog1loc" "$syslogloc" 2>/dev/null >> "$skynetlog"
        sed -i '\~BLOCKED -~d' "$syslog1loc" "$syslogloc" 2>/dev/null
        if [ "$(du "$skynetlog" | awk '{print $1}')" -ge "10240" ]; then
            sed -i '\~BLOCKED -~d' "$skynetlog"
            sed -i '\~Skynet: \[#\] ~d' "$skynetevents"
            if [ "$(du "$skynetlog" | awk '{print $1}')" -ge "3000" ]; then
                true > "$skynetlog"
            fi
        fi
        if [ "$1" = "all" ] || [ "$(grep -cE "Skynet: [#] " "$syslogloc" 2>/dev/null)" -gt "24" ] 2>/dev/null; then
            sed '\~Skynet: \[#\] ~!d' "$syslog1loc" "$syslogloc" 2>/dev/null >> "$skynetevents"
            sed -i '\~Skynet: \[#\] ~d;\~Skynet: \[i\] ~d;\~Skynet: \[\*\] Lock ~d' "$syslog1loc" "$syslogloc" 2>/dev/null
        fi
}
But darned if I can figure this out. I think the first line uses the tilde as the delimiter of the pattern to match, so it looks for every line in skynet-0.log (and the backup if there is one) that has "BLOCKED -" in it, doesn't delete it, but appends it to skynet's own log. Then it deletes all those lines.

Then it looks at the size of the skynet log, and if it is more than 10240 kilobytes, it deletes all the BLOCKED lines from the skynet log. Then it deletes all the Skynet: messages from the events log, which would include the hourlies. If the size of the log is more than 3000 kilobytes, it just exits.

@Adamm, care to take pity on @Butterfly Bones and me?
 
Last edited:
One more update, finally have confirmation that Skynet banmalware is updating. I'm not sure if it has to do with my last method to install Scribe with Skynet temporarily disabled or learning how to search effectively in Loggly. :)

I did search back a few days and only saw today's banmalware event after learning better search techniques, but not finding logs does not mean it did not occur via cron, as I manually ran it a few times to insure Skynet was current.

Here is a screen capture, those using Loggly, check the filter.
Code:
 syslog.appName : Skynet

screenshot-timsaw-loggly-com-2019-04-20-07-15-42.png
 
(moved here to the proper thread)
HI cmkelley

Just wanna ask , how far your installer for syslog-ng project is growing?
May i have a try ?

Greetings ivi
Please do. I believe it to be feature complete and it seems to be stable for everyone who has tried, I am happy to have more people try it out.

Thanks,
 
Had something odd this morning while I was gone for a couple hours, logging stopped, that is syslog-ng stopped, Found this in the syslog-ng,log.
Code:
Apr 22 08:56:42 RT-AC86U-4608 syslog-ng[2505]: syslog-ng shutting down; version='3.19.1'
Apr 22 10:20:14 RT-AC86U-4608 syslog-ng[8095]: syslog-ng starting up; version='3.19.1'
Apr 22 10:20:14 RT-AC86U-4608 syslog-ng[8095]: Syslog connection established; fd='17', server='AF_INET(52.24.1.210:514)', local='AF_INET(0.0.0.0:0)'
(the server='AF-INET___' is the connection to Loggly from my AC86U and is normal)

No idea why, the restart at 10:20 was me running "scribe restart" when I got home and found nothing showing in Loggly. Check all the other logs, nada, and of course nothing in the "messages" either. I'm stumped. Any idea why it would stop after days or running with not even a hiccup?
 
@cmkelley , not sure if you are aware but once ntpd service is invoked by the system Scribe stops working and the default logger takes over. Restarting scribe resumes normal operations:)
firmware version :384.11_alpha3-g417036f50b


Code:
Apr 24 01:33:01 RT-AC86U dropbear[11711]: Exit (jose): Terminated by signal
Apr 24 01:33:01 RT-AC86U ntpd: Started ntpd
Apr 24 01:33:01 RT-AC86U miniupnpd[471]: shutting down MiniUPnPd
Apr 24 01:33:01 kernel: klogd started: BusyBox v1.25.1 (2019-04-16 21:17:50 EDT)
Apr 24 01:33:01 kernel: PC is at 0xf71f9f44
 
@cmkelley , not sure if you are aware but once ntpd service is invoked by the system Scribe stops working and the default logger takes over. Restarting scribe resumes normal operations:)
firmware version :384.11_alpha3-g417036f50b


Code:
Apr 24 01:33:01 RT-AC86U dropbear[11711]: Exit (jose): Terminated by signal
Apr 24 01:33:01 RT-AC86U ntpd: Started ntpd
Apr 24 01:33:01 RT-AC86U miniupnpd[471]: shutting down MiniUPnPd
Apr 24 01:33:01 kernel: klogd started: BusyBox v1.25.1 (2019-04-16 21:17:50 EDT)
Apr 24 01:33:01 kernel: PC is at 0xf71f9f44
What version of scribe are you using? I don't remember what version I added it, but the latest (v0.9_2) add a line to service-event that calls scribe when the service "logger" is started/restarted. scribe then kills klogd and syslogd. But as I say, that relies on the service "logger" being called, not klogd or syslogd independently. This seems to work without problem on 384.10_2.

Notably, I haven't tested 384.11 alpha yet, I haven't had time. Maybe 384.11 runs them separately? If you "grep restart messages" from /opt/var/log do you see any messages that say "restart klogd" or do they say "restart logger"?
 
What version of scribe are you using? I don't remember what version I added it, but the latest (v0.9_2) add a line to service-event that calls scribe when the service "logger" is started/restarted. scribe then kills klogd and syslogd. But as I say, that relies on the service "logger" being called, not klogd or syslogd independently. This seems to work without problem on 384.10_2.

Notably, I haven't tested 384.11 alpha yet, I haven't had time. Maybe 384.11 runs them separately? If you "grep restart messages" from /opt/var/log do you see any messages that say "restart klogd" or do they say "restart logger"?
my bad, version v0.9_1.
updating now and will report back.
 
my bad, version v0.9_1.
updating now and will report back.
unable to update:
Code:
 Do you want to update syslog-ng and logrotate example files? [y|n] y
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
curl: (60) SSL certificate problem: certificate is not yet valid
More details here: https://curl.haxx.se/docs/sslcerts.html

curl failed to verify the legitimacy of the server and therefore could not
establish a secure connection to it. To learn more about this situation and
how to fix it, please visit the web page mentioned above.

 scribe GitHub repository is unavailable!  -- Aborting.
 
unable to update:
Code:
 Do you want to update syslog-ng and logrotate example files? [y|n] y
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
curl: (60) SSL certificate problem: certificate is not yet valid
More details here: https://curl.haxx.se/docs/sslcerts.html

curl failed to verify the legitimacy of the server and therefore could not
establish a secure connection to it. To learn more about this situation and
how to fix it, please visit the web page mentioned above.

 scribe GitHub repository is unavailable!  -- Aborting.
I'm clueless. I have nothing to do with SSL certs for github. Not sure what happened there.
 
I'm clueless. I have nothing to do with SSL certs for github. Not sure what happened there.
Np Man . found the problem at this end ,
The rt-ac86u is taking the piss again, something seems broken , possibly Entware related.
Tried on the rt-ac87u and updated without issues.
Disclosure:
(2x routers with 2 fibre connections)
 
Certificate not valid yet = check your clock.

Sent from my ELE-L04 using Tapatalk
 
Code:
Apr 24 22:49:27 RT-AC86U ntpd: Started ntpd
Apr 24 22:49:27 RT-AC86U miniupnpd[8909]: shutting down MiniUPnPd
Apr 24 22:49:27 kernel: klogd started: BusyBox v1.25.1 (2019-04-16 21:17:50 EDT)
Apr 24 22:49:27 kernel: ubi1: attaching mtd9
@cmkelley
tested in both routers (same firmware)

Now, it may not be the ntpd service itself, as this happens when i press the apply button on the administration page, i assumed it was ntpd as it was the last entry in the syslog_ng before the default logger kicks in.
 
Code:
Apr 24 22:49:27 RT-AC86U ntpd: Started ntpd
Apr 24 22:49:27 RT-AC86U miniupnpd[8909]: shutting down MiniUPnPd
Apr 24 22:49:27 kernel: klogd started: BusyBox v1.25.1 (2019-04-16 21:17:50 EDT)
Apr 24 22:49:27 kernel: ubi1: attaching mtd9
@cmkelley
tested in both routers (same firmware)

Now, it may not be the ntpd service itself, as this happens when i press the apply button on the administration page, i assumed it was ntpd as it was the last entry in the syslog_ng before the default logger kicks in.
Right, please run "grep restart /opt/var/log/messages", and look for klogd, syslogd, or logger. That should answer the question.
 
Right, please run "grep restart /opt/var/log/messages", and look for klogd, syslogd, or logger. That should answer the question.
None of those words show up in the log.
Log here
 
None of those words show up in the log.
Log here
Well, that sucks. Not sure how to kill klogd if it's not throwing a message somewhere that it's starting. :(

I managed to bork the computer connected to my test router, so I'm trying to get that back up and running. I'll upgrade my test router to 384.11 alpha when I get that computer back up. I really don't want to risk alpha firmware on my main router. I know RMerlin is good, but I don't want an unhappy spouse if something unexpected happens.
 
Dirty flash on a 87U from 10 to 11A4 seems to be working fine with scribe.
 
Dirty flash on a 87U from 10 to 11A4 seems to be working fine with scribe.
Please let me know if you start seeing klogd or syslogd starting and not being killed. It's going to be over the weekend before I can update my test router.
 
Had something odd this morning while I was gone for a couple hours, logging stopped, that is syslog-ng stopped, Found this in the syslog-ng,log.
Code:
Apr 22 08:56:42 RT-AC86U-4608 syslog-ng[2505]: syslog-ng shutting down; version='3.19.1'
Apr 22 10:20:14 RT-AC86U-4608 syslog-ng[8095]: syslog-ng starting up; version='3.19.1'
Apr 22 10:20:14 RT-AC86U-4608 syslog-ng[8095]: Syslog connection established; fd='17', server='AF_INET(52.24.1.210:514)', local='AF_INET(0.0.0.0:0)'
(the server='AF-INET___' is the connection to Loggly from my AC86U and is normal)

No idea why, the restart at 10:20 was me running "scribe restart" when I got home and found nothing showing in Loggly. Check all the other logs, nada, and of course nothing in the "messages" either. I'm stumped. Any idea why it would stop after days or running with not even a hiccup?
Got it again, random stop and restart, logging stops, no clues in any logs, snippet of /opt/var/log/syslog-ng.log.
Code:
Apr 25 07:25:37 RT-AC86U-4608 syslog-ng[7107]: syslog-ng shutting down; version='3.19.1'
Apr 25 07:26:25 RT-AC86U-4608 syslog-ng[8425]: syslog-ng starting up; version='3.19.1'
Apr 25 07:26:26 RT-AC86U-4608 syslog-ng[8425]: Syslog connection established; fd='17', server='AF_INET(52.21.83.61:514)', local='AF_INET(0.0.0.0:0)'
Apr 25 10:28:37 RT-AC86U-4608 syslog-ng[8425]: syslog-ng shutting down; version='3.19.1'
Apr 25 10:28:39 RT-AC86U-4608 syslog-ng[25299]: syslog-ng starting up; version='3.19.1'
No idea why it stopped at 07:25:37 then started at 07:26:25 - the 10:28:37 shut down and starting up was me running "scribe restart".

This is how I realized something was wrong "cat /opt/var/log/chkwan.log" that should run via cron every 5 minutes. I randomly check that as I'm still fine tuning that script, I'm really good at shooting myself in the foot with these things. :oops:
Code:
Apr 25 07:25:00 RT-AC86U-4608 (ChkWAN.sh)[7518]: 7258 v1.11 Monitoring WAN connection using 4 target PING hosts ([redacted] 192.168.1.1 8.8.8.8 1.1.1.1) (Tries=3)
Apr 25 07:25:00 RT-AC86U-4608 (ChkWAN.sh)[7523]: 7258 Monitoring pass 1 out of 3
Apr 25 07:25:00 RT-AC86U-4608 (ChkWAN.sh)[7556]: 7258 Monitoring WAN connection OK.....(Successful ping to '[redacted]'); Terminating due to ACTIVE cron schedule
Apr 25 10:30:01 RT-AC86U-4608 (ChkWAN.sh)[25694]: 25389 v1.11 Monitoring WAN connection using 4 target PING hosts ([redacted] 192.168.1.1 8.8.8.8 1.1.1.1) (Tries=3)
Apr 25 10:30:01 RT-AC86U-4608 (ChkWAN.sh)[25705]: 25389 Monitoring pass 1 out of 3
Apr 25 10:30:01 RT-AC86U-4608 (ChkWAN.sh)[25787]: 25389 Monitoring WAN connection OK.....(Successful ping to '[redacted]'); Terminating due to ACTIVE cron schedule
 
Got it again, random stop and restart, logging stops, no clues in any logs, snippet of /opt/var/log/syslog-ng.log.
Code:
Apr 25 07:25:37 RT-AC86U-4608 syslog-ng[7107]: syslog-ng shutting down; version='3.19.1'
Apr 25 07:26:25 RT-AC86U-4608 syslog-ng[8425]: syslog-ng starting up; version='3.19.1'
Apr 25 07:26:26 RT-AC86U-4608 syslog-ng[8425]: Syslog connection established; fd='17', server='AF_INET(52.21.83.61:514)', local='AF_INET(0.0.0.0:0)'
Apr 25 10:28:37 RT-AC86U-4608 syslog-ng[8425]: syslog-ng shutting down; version='3.19.1'
Apr 25 10:28:39 RT-AC86U-4608 syslog-ng[25299]: syslog-ng starting up; version='3.19.1'
No idea why it stopped at 07:25:37 then started at 07:26:25 - the 10:28:37 shut down and starting up was me running "scribe restart".

This is how I realized something was wrong "cat /opt/var/log/chkwan.log" that should run via cron every 5 minutes. I randomly check that as I'm still fine tuning that script, I'm really good at shooting myself in the foot with these things. :oops:
Code:
Apr 25 07:25:00 RT-AC86U-4608 (ChkWAN.sh)[7518]: 7258 v1.11 Monitoring WAN connection using 4 target PING hosts ([redacted] 192.168.1.1 8.8.8.8 1.1.1.1) (Tries=3)
Apr 25 07:25:00 RT-AC86U-4608 (ChkWAN.sh)[7523]: 7258 Monitoring pass 1 out of 3
Apr 25 07:25:00 RT-AC86U-4608 (ChkWAN.sh)[7556]: 7258 Monitoring WAN connection OK.....(Successful ping to '[redacted]'); Terminating due to ACTIVE cron schedule
Apr 25 10:30:01 RT-AC86U-4608 (ChkWAN.sh)[25694]: 25389 v1.11 Monitoring WAN connection using 4 target PING hosts ([redacted] 192.168.1.1 8.8.8.8 1.1.1.1) (Tries=3)
Apr 25 10:30:01 RT-AC86U-4608 (ChkWAN.sh)[25705]: 25389 Monitoring pass 1 out of 3
Apr 25 10:30:01 RT-AC86U-4608 (ChkWAN.sh)[25787]: 25389 Monitoring WAN connection OK.....(Successful ping to '[redacted]'); Terminating due to ACTIVE cron schedule
I'm stumped. That's not from a SIG-HUP, syslog-ng is shutting down and restarting, or rather, something is shutting it down and restarting it a minute later somehow (weird!).

Mine seems to run without issue, other than I get tons of blank kernel: lines. I'm not sure how to find a blank line and filter it out with a filter rule.
 

Latest threads

Sign Up For SNBForums Daily Digest

Get an update of what's new every day delivered to your mailbox. Sign up here!
Top