What's new

Scribe Scribe / syslog-ng re-load question

  • SNBForums Code of Conduct

    SNBForums is a community for everyone, no matter what their level of experience.

    Please be tolerant and patient of others, especially newcomers. We are all here to share and learn!

    The rules are simple: Be patient, be nice, be helpful or be gone!

archiel

Very Senior Member
scribe and uiscribe seem to be working as intended, but I am curious about
1. why the configuration is reloaded twice per hour at xx:57 and then 3 minutes later at xy:00
2. why the default config is looking for the missing file /var/lib/logrotate.status

Code:
Dec  6 12:57:00 RT-AX88U-5050 syslog-ng[20915]: Follow-mode file source not found, deferring open; filename='/var/lib/logrotate.status'
Dec  6 12:57:00 RT-AX88U-5050 syslog-ng[20915]: Configuration reload request received, reloading configuration;
Dec  6 12:57:00 RT-AX88U-5050 syslog-ng[20915]: Configuration reload finished;
Dec  6 13:00:00 RT-AX88U-5050 syslog-ng[20915]: Follow-mode file source not found, deferring open; filename='/var/lib/logrotate.status'
Dec  6 13:00:00 RT-AX88U-5050 syslog-ng[20915]: Configuration reload request received, reloading configuration;
Dec  6 13:00:00 RT-AX88U-5050 syslog-ng[20915]: Configuration reload finished;

If it is relevant, the consolidated syslog-ng-complete.conf is
Code:
#############################################################################
# syslog-ng.conf customized for scribe on Asuswrt-Merlin firmware
# compare to /opt/share/syslog-ng/examples/syslog-ng.conf-opkg for differences from Entware distribution
#
# syslog-ng documentation: https://www.syslog-ng.com/technical-documents/doc/syslog-ng-open-source-edition/3.29/administration-guide
#
# Release notes: https://github.com/syslog-ng/syslog-ng/releases

@version: 3.29
#@include "scl.conf" # uncomment this line to for additional functionality, see syslog-ng documentation
@include "/opt/etc/syslog-ng.d/" # Put any customization files in this directory
# don't log empty messages from kernel

filter f_blank {
    program("kernel") and
    message("^ *$");
};

log {
    source(src);
    filter(f_blank);
    flags(final);
};

#eof
# log cake-qos change to /opt/var/log/cake.log only

destination d_cake {
    file("/opt/var/log/cake.log");
};

filter f_cake {
    program("CakeQOS-Merlin") or
    message ("Cpuidle Host Clock");
};

log {
    source(src);
    filter(f_cake);
    destination(d_cake);
    flags(final);
};

#eof
# log all diversion and uiDivStats logs to diversion.log

destination d_diversion {
    file("/opt/var/log/diversion.log");
};

filter f_diversion {
    program("uiDivStats") or
    program("Diversion");
};

log {
    source(src);
    filter(f_diversion);
    destination(d_diversion);
    flags(final);
};

#eof
# log ethernet change to /opt/var/log/ethernet.log only

destination d_ethernet {
    file("/opt/var/log/ethernet.log");
};

filter f_kernel {
    program("kernel");
};

filter f_ethernet {
    message("eth1") or
    message("eth2") or
    message("eth3") or
    message("eth4") or
    message("eth5") or
    message("eth6") or
    message("eth7") or
    message("eth8") or
    message("br0:");
};

log {
    source(src);
    filter(f_kernel);
    filter(f_ethernet);
    destination(d_ethernet);
    flags(final);
};

#eof
# log hostapd change to /opt/var/log/hostapd.log only

destination d_hostapd {
    file("/opt/var/log/hostapd.log");
};

filter f_hostapd {
    program("hostapd") or
    program("roamast") or
    message ("CFG80211-ERROR");
};

log {
    source(src);
    filter(f_hostapd);
    destination(d_hostapd);
    flags(final);
};

#eof
# gather logrotate logs together for unified log

destination d_logrotate {
    file("/opt/var/log/logrotate.log");
};

source s_lr_status {
    file("/var/lib/logrotate.status" program-override("logrotate") flags(no-parse));
};

source s_lr_daily {
    file("/opt/tmp/logrotate.daily" program-override("logrotate") flags(no-parse));
};

log {
    source(s_lr_status);
    source(s_lr_daily);
    destination(d_logrotate);
    flags(final);
};

#eof
# log all openvpn server and client logs into one file - /opt/var/log/openvpn.log and stop processing openvpn logs

destination d_openvpn {
    file("/opt/var/log/openvpn.log");
};

filter f_openvpn {
    program("ovpn-server1") or
    program("ovpn-server2") or
    program("ovpn-client1") or
    program("ovpn-client2") or
    program("ovpn-client3") or
    program("ovpn-client4") or
    program("ovpn-client5") or
    program("openvpn-routing");
};

log {
    source(src);
    filter(f_openvpn);
    destination(d_openvpn);
    flags(final);
};

#eof
# log all pixelserv-tls logs to /opt/var/log/pixelserv.log and stop processing pixelserv-tls logs

destination d_pixelserv {
    file("/opt/var/log/pixelserv.log");
};

filter f_pixelserv {
    program("pixelserv-tls");
};

log {
    source(src);
    filter(f_pixelserv);
    destination(d_pixelserv);
    flags(final);
};

#eof
# this MUST BE the file configured in Skynet as the syslog.log location
# DO NOT use /tmp/syslog.log or /opt/var/log/messages here!
destination d_skynet {
    file("/opt/var/log/skynet-0.log");
};

# logs everything from Skynet to /opt/var/log/skynet-0.log
filter f_skynet {
    program("Skynet") or
    message("BLOCKED -") or
    message("DROP IN=");
};

# final flag stops processing of messages matching the f_skynet filter
log {
    source(src);
    filter(f_skynet);
    destination(d_skynet);
    flags(final);
};

#eof
# put spdMerlin's speed test results into /opt/var/log/spdMerlin.log

destination d_spdmerlin {
    file("/opt/var/log/spdmerlin.log");
};

filter f_spdmerlin {
    program("spdMerlin");
};

log {
    source(src);
    filter(f_spdmerlin);
    destination(d_spdmerlin);
    flags(final);
};

#eof
# put syslog-ng's logging stats into /opt/var/log/syslog-ng.log

destination d_syslogng {
    file("/opt/var/log/syslog-ng.log");
};

filter f_syslogng {
    program("syslog-ng")
};

log {
    source(src);
    filter(f_syslogng);
    destination(d_syslogng);
    flags(final);
};

#eof
# log all unbound logs to /opt/var/log/unbound.log and stop processing unbound logs

destination d_unbound {
    file("/opt/var/log/unbound.log");
};

filter f_unbound {
    program("unbound") or
    program("gen_adblock.sh");
};

log {
    source(src);
    filter(f_unbound);
    destination(d_unbound);
    flags(final);
};
#eof
# put wlceventd Assoc/ReAssoc/Disassoc messages into /opt/var/log/wlceventd.log

destination d_wlceventd {
    file("/opt/var/log/wlceventd.log");
};

filter f_wlceventd {
    ( ( program("WLCEVENTD") or
    program("wlceventd") ) and
    ( message("ssoc") or
    message("uth") ) ) or
    ( program("syslog") and
    message("wlceventd") );
};

log {
    source(src);
    filter(f_wlceventd);
    destination(d_wlceventd);
    flags(final);
};

#eof

options {
    chain_hostnames(no); # Enable or disable the chained hostname format.
    create_dirs(yes);
    keep_hostname(yes); # Enable or disable hostname rewriting.
    log_fifo_size(256); # The number of messages that the output queue can store.
    log_msg_size(16384); # Maximum length of a message in bytes.
    stats_freq(21600); # The period between two STATS messages sent by syslog-ng, containing statistics about dropped logs in seconds; 0 disables. (21,600 seconds = 6 hours)
    flush_lines(0); # How many lines are flushed to a destination at a time.
    use_fqdn(no); # Add Fully Qualified Domain Name instead of short hostname.
};

# syslog-ng gets messages from the system, kernel, and syslog-ng (internal)
# DO NOT use system() source; causes issues on HND routers
# so_rcvbuf = maximum number of messages per second * 1024
source src {
    unix-dgram("/dev/log" so_rcvbuf(65536) flags(syslog-protocol));
    file("/proc/kmsg" program_override("kernel") flags(kernel));
    internal();
#    udp(ip(192.168.x.y) port(514)); # uncomment this line to pass all network messages through syslog-ng filters
};

# if you only want to pass network messages through some syslog-ng filters, uncomment the source line below
# then add "soource(net);" to the log statement in any filter you want to pass network messages through
#source net { udp(ip(192.168.x.y) port(514)); };

# set the filename for the default log file - anything not filtered out will end up here
destination messages { file("/opt/var/log/messages"); };

# to send log messages to the local network, uncomment the destination line below
# then add "destination(log_server);" to the log statement in any filter you want to pass network messages through
#destination log_server { udp("192.168.x.y" port(514)); };

log {
    source(src);
#    source(net); # uncomment this and "source net" function above to get udp log messages from local network
    destination(messages);
#    destination(log_server); # uncomment this and "destination log_server" function above to send udp log messages to local network
};
 
The hourly reload is triggered by skynet's rewriting of its log file. I have in my log oddly timed reloads, but they are not a regular occurrence. I've just updated my 86U to the new 386 beta, and syslog-ng to 3.29, so my non-standard scribe install is a little borked, but I am still seeing only the regular hangups.

The logrotate.status thing is I think buried in the old thread. I don't recall exactly what was going on.
 
In my case the two reloads, at :57 and 3 minutes later at :00 are absolutely regular - I don't think there is a problem, but would really like to understand why.

The answer to logrotate.status turns out to be to simply create a null file
Code:
touch /var/lib/logrotate.status
 
Last edited:
If memory serves me correctly, skynet clears out the blocked syslog entries on a regular interval. If this is something you want to keep you need to send it off to a syslog server. Only the most recent logs will remain after every restart.

Suggest you look at the Purge_Logs portion of the script if you really want to look at what its doing.
 
Last edited:
In my case the two reloads, at :57 and 3 minutes later at :00 are absolutely regular - I don't think there is a problem, but would really like to understand why.
You might look at what chron job is running at :57. The reload is triggered by a hangup signal, so it might be another script that is doing that.
The answer to logrotate.status turns out to be to simply create a null file
Code:
I think after the first logrotate of the day, this file is created and the message goes away.
If memory serves me correctly, skynet clears out the blocked syslog entries on a regular interval. If this is something you want to keep you need to send it off to a syslog server.
Not sure that is what the OP was asking, but you could also just add a second destination definition and send a duplicate of the message there, right before the flags-final. I can't think why one would want ever look at them though.
 
logrotate.status: this was not being create automatically (or at all) - as noted the null file has cleared this from the logs.

in respect of skynet, the main log is created in /mnt/router/skynet and as agliani has noted is replaced from time to time. From settings option 11 a copy can be setup to use with scribe/uiscribe and this version is consolidated on the hour
Code:
Dec  7 09:00:02 RT-AX88U-5050 Skynet: [#] 78218 IPs (+0) -- 1733 Ranges Banned (+0) || 1935 Inbound -- 237 Outbound Connections Blocked! [save] [2s]
Dec  7 10:00:02 RT-AX88U-5050 Skynet: [#] 78218 IPs (+0) -- 1733 Ranges Banned (+0) || 2118 Inbound -- 237 Outbound Connections Blocked! [save] [2s]
Dec  7 11:00:02 RT-AX88U-5050 Skynet: [#] 78218 IPs (+0) -- 1733 Ranges Banned (+0) || 2298 Inbound -- 237 Outbound Connections Blocked! [save] [1s]
For me its usefulness is as a quick check via the system log on whether Skynet is behaving normally.
This also explains the syslog rotation on the hour.

Similarly I can see /jffs/addons/unbound/unbound_log.sh which explains the rotation at :57

So - solved
 

Sign Up For SNBForums Daily Digest

Get an update of what's new every day delivered to your mailbox. Sign up here!
Top