Scribe scribe - syslog-ng and logrotate installer

  • ATTENTION! You'll notice a Prefix dropdown when you create a thread. If your post applies to one of the topics listed, please use that Prefix for your post. When browsing the thread list you can use the Prefix to filter the view.
  • ATTENTION! As of November 1, 2020, you are not able to reply to threads 6 months after the thread is opened if there are more than 500 posts in the thread.
    Threads will not be locked, so posts may still be edited by their authors.
    Just start a new thread on the topic to post if you get an error message when trying to reply to a thread.

Butterfly Bones

Very Senior Member
I update Entware five days ago and all good here.
Code:
[email protected]:/tmp/home/root# ls -alh /opt/var/log/
drwxr-xr-x    2 mtn_danc root        4.0K Sep 19 17:00 .
drwxr-xr-x    8 mtn_danc root        4.0K Sep 19 05:40 ..
-rw-------    1 mtn_danc root      131.0K Sep 19 04:26 afpd.log
-rw-------    1 mtn_danc root         188 Aug 27 18:27 afpd.log-20200830.gz
-rw-------    1 mtn_danc root         612 Sep 11 16:30 afpd.log-20200913
-rw-------    1 mtn_danc root        1.1K Apr 15 02:00 amas_lib.log
-rw-------    1 mtn_danc root         369 Sep 15 12:24 crash.log
-rw-r-----    1 nobody   root        9.3M Sep 19 17:09 dnsmasq.log
-rw-r-----    1 nobody   root       17.9M Sep 19 05:20 dnsmasq.log1
-rw-r-----    1 nobody   root       34.3M Sep 19 05:20 dnsmasq.log2
-rw-------    1 mtn_danc root       12.4K Sep 19 16:17 ethernet.log
-rw-------    1 mtn_danc root         705 Aug 21 20:44 ethernet.log-20200823.gz
-rw-------    1 mtn_danc root         871 Aug 28 15:01 ethernet.log-20200830.gz
-rw-------    1 mtn_danc root         513 Sep  2 19:17 ethernet.log-20200906.gz
-rw-------    1 mtn_danc root        9.1K Sep 12 16:52 ethernet.log-20200913
-rw-------    1 mtn_danc root      194.9K Sep 19 00:05 logrotate.log
-rw-------    1 mtn_danc root      407.7K Sep 19 16:25 messages
-rw-------    1 mtn_danc root      166.3K May 17 00:03 messages-20200517.gz
-rw-------    1 mtn_danc root      206.2K Aug 18 00:03 messages-20200818.gz
-rw-------    1 mtn_danc root         491 Aug 19 00:03 messages-20200819.gz
-rw-------    1 mtn_danc root       16.2K Aug 21 00:03 messages-20200821.gz
-rw-------    1 mtn_danc root        6.1K Aug 23 00:03 messages-20200823
-rw-------    1 mtn_danc root      734.4K Sep 19 17:04 openvpn.log
-rw-------    1 mtn_danc root       57.7K Apr 14 18:46 openvpn.log-20200415.gz
-rw-------    1 mtn_danc root       60.6K Aug 11 12:42 openvpn.log-20200812.gz
-rw-------    1 mtn_danc root        1.0M Sep 15 12:26 openvpn.log-20200916
-rw-------    1 mtn_danc root      102.0K Sep 19 04:26 pixelserv.log
-rw-------    1 mtn_danc root       21.9K Sep 19 17:09 skynet-0.log
-rw-------    1 mtn_danc root       37.9K Sep 19 17:00 syslog-ng.log
-rw-------    1 mtn_danc root        2.5K Aug 23 00:05 syslog-ng.log-20200823.gz
-rw-------    1 mtn_danc root        1.8K Aug 30 00:05 syslog-ng.log-20200830.gz
-rw-------    1 mtn_danc root        1.9K Sep  6 00:00 syslog-ng.log-20200906.gz
-rw-------    1 mtn_danc root       43.9K Sep 13 00:05 syslog-ng.log-20200913
-rw-------    1 mtn_danc root       84.9K Sep 19 17:04 wlceventd.log
-rw-------    1 mtn_danc root        3.0K Aug 22 20:21 wlceventd.log-20200823.gz
-rw-------    1 mtn_danc root        5.9K Aug 29 20:53 wlceventd.log-20200830.gz
-rw-------    1 mtn_danc root        4.4K Sep  6 00:04 wlceventd.log-20200906.gz
-rw-------    1 mtn_danc root      106.5K Sep 13 00:00 wlceventd.log-20200913
 

elorimer

Very Senior Member

Butterfly Bones

Very Senior Member
Have you rebooted? I did and syslog-ng isnt working. Haven't had time to puzzle at it.
Yes, at least twice in that time. Moving furniture around. :rolleyes:
 

elorimer

Very Senior Member
Have you rebooted? I did and syslog-ng isnt working. Haven't had time to puzzle at it.
NVM. I had plugged a hard drive into the USB3 port to test the smb sharing, and that was preventing syslog-ng from starting on reboot correctly. When I removed it, everything worked again. So it was something entirely different, which I am not going to bother with.
 

cmkelley

Very Senior Member
I just attached a text file containing the log. The name of the file is Log.txt. I didn't see any line item that it is choking on, but mine are the eyes of inexperience. Are you able to read my attachment?

Ken
Did you modify the /opt/etc/syslog-ng.conf file?

I am sadly quite slammed with other obligations for the foreseeable future, so I'm only going to be of limited help. Sorry.
 

elorimer

Very Senior Member
Yes @giant46man46, you might post your syslog-ng.conf file here.

I can see it is including your custom files in /opt/etc/syslog-ng.d and seems to be processing them ok. I don't think you need the wlcevent file for your 68U. I think those are HND messages.

When syslog-ng wants to write to a destination, if the file doesn't exist, it creates it. So the fact that you don't see, for example, a logrotate file means, for some reason, syslog-ng hasn't yet seen a message that needs to go there. But that should happen nightly; that suggests to me that there is something wrong with the src source destination, or syslog-ng isn't running. That is in the syslog-ng.conf file.
 

Butterfly Bones

Very Senior Member
I made some large changes to my OpenVPN configurations, both client and server, and wanted to break those two out to separate logs to make troubleshooting easier. Here they are if anyone is interested.

Code:
# log all openvpn client logs into a file - /opt/var/log/ovpnclient.log and stop processing openvpn logs

destination d_ovpnclient { 
    file("/opt/var/log/ovpnclient.log");
};

filter f_ovpnclient {
    program("ovpn-client1") or
    program("ovpn-client2") or
    program("ovpn-client3") or
    program("ovpn-client4") or
    program("ovpn-client5") or
    program("openvpn-routing");
};

log {
    source(src);
    filter(f_ovpnclient);
    destination(d_ovpnclient);
    flags(final);
};

#eof
Code:
# log all openvpn server into a file - /opt/var/log/ovpnserver.log and stop processing openvpn logs

destination d_ovpnserver { 
    file("/opt/var/log/ovpnserver.log");
};

filter f_ovpnserver {
    program("ovpn-server1") or
    program("ovpn-server2");
};

log {
    source(src);
    filter(f_ovpnserver);
    destination(d_ovpnserver);
    flags(final);
};

#eof
 

elorimer

Very Senior Member
@Butterfly Bones, I sometimes wonder about optimizing our filters. Our routers don't spit out log messages that fast, so I suppose it doesn't matter, but AND and OR filters take longer because the program has to read each message multiple times for that filter

The program function accepts a regexp, so your f_ovpnclient function could also read like this:
Code:
filter f_ovpnclient {
    program("ovpn-client?", type(glob));
};
I think that would operate much faster (glob being much faster than regexp processing).

Another possibility is to screen messages like this:
Code:
filter f_ovpnclient {
    program("ovpn-client?", type(glob));
};
filter f_ovpnserver{
    program("ovpn-server?", type(glob));
};
filter f_ovpn {
    program("ovpn*", type(glob));
};
log {
    source(src);
    filter(f_ovpn);
    filter(f_ovpnserver);
    destination(d_ovpnserver);
    flags(final);
};
log {
    source(src);
    filter(f_ovpn);
    filter(f_ovpnclient);
    destination(d_ovpnclient);
    flags(final);
};
I honestly don't know if that would be faster, but anything that doesn't come a program starting with ovpn won't get dealt with further, and only those starting with ovpn will get processed by the more complicated filters.


The other thing is to give more complex configuration files, or those for less frequent use, an alphabetic name lower in priority. Because syslog-ng applies configuration files to message alphabetically, and we are using the final function to stop processing a message, simple, frequent log messages can be stripped out in the beginning with first-in-order filters and don't have to pass on to others. Slower filters never get reached, and can be the last of the filters tried before a message drops to messages.
 
Last edited:

Butterfly Bones

Very Senior Member
@elorimer Thank you for the ideas, I will try that optimization soon. My separate ovpn filters are not working as I want, so I have been tweaking them. As usual, Real Life (tm) is getting the way of my router adjustments, so it will be another week or so before I get back to this project.
 

Similar threads

Latest threads

Sign Up For SNBForums Daily Digest

Get an update of what's new every day delivered to your mailbox. Sign up here!
Top