Scribe scribe - syslog-ng and logrotate installer

[cmkelley]

I just attached a text file containing the log. The name of the file is Log.txt. I didn't see any line item that it is choking on, but mine are the eyes of inexperience. Are you able to read my attachment?

Ken

Did you modify the /opt/etc/syslog-ng.conf file?

I am sadly quite slammed with other obligations for the foreseeable future, so I'm only going to be of limited help. Sorry.

 

[elorimer]

Yes @giant46man46, you might post your syslog-ng.conf file here.

I can see it is including your custom files in /opt/etc/syslog-ng.d and seems to be processing them ok. I don't think you need the wlcevent file for your 68U. I think those are HND messages.

When syslog-ng wants to write to a destination, if the file doesn't exist, it creates it. So the fact that you don't see, for example, a logrotate file means, for some reason, syslog-ng hasn't yet seen a message that needs to go there. But that should happen nightly; that suggests to me that there is something wrong with the src source destination, or syslog-ng isn't running. That is in the syslog-ng.conf file.

 

[Butterfly Bones]

I made some large changes to my OpenVPN configurations, both client and server, and wanted to break those two out to separate logs to make troubleshooting easier. Here they are if anyone is interested.

# log all openvpn client logs into a file - /opt/var/log/ovpnclient.log and stop processing openvpn logs

destination d_ovpnclient { 
    file("/opt/var/log/ovpnclient.log");
};

filter f_ovpnclient {
    program("ovpn-client1") or
    program("ovpn-client2") or
    program("ovpn-client3") or
    program("ovpn-client4") or
    program("ovpn-client5") or
    program("openvpn-routing");
};

log {
    source(src);
    filter(f_ovpnclient);
    destination(d_ovpnclient);
    flags(final);
};

#eof

# log all openvpn server into a file - /opt/var/log/ovpnserver.log and stop processing openvpn logs

destination d_ovpnserver { 
    file("/opt/var/log/ovpnserver.log");
};

filter f_ovpnserver {
    program("ovpn-server1") or
    program("ovpn-server2");
};

log {
    source(src);
    filter(f_ovpnserver);
    destination(d_ovpnserver);
    flags(final);
};

#eof

 

[elorimer]

@Butterfly Bones, I sometimes wonder about optimizing our filters. Our routers don't spit out log messages that fast, so I suppose it doesn't matter, but AND and OR filters take longer because the program has to read each message multiple times for that filter

The program function accepts a regexp, so your f_ovpnclient function could also read like this:

filter f_ovpnclient {
    program("ovpn-client?", type(glob));
};

I think that would operate much faster (glob being much faster than regexp processing).

Another possibility is to screen messages like this:

filter f_ovpnclient {
    program("ovpn-client?", type(glob));
};
filter f_ovpnserver{
    program("ovpn-server?", type(glob));
};
filter f_ovpn {
    program("ovpn*", type(glob));
};
log {
    source(src);
    filter(f_ovpn);
    filter(f_ovpnserver);
    destination(d_ovpnserver);
    flags(final);
};
log {
    source(src);
    filter(f_ovpn);
    filter(f_ovpnclient);
    destination(d_ovpnclient);
    flags(final);
};

I honestly don't know if that would be faster, but anything that doesn't come a program starting with ovpn won't get dealt with further, and only those starting with ovpn will get processed by the more complicated filters.


The other thing is to give more complex configuration files, or those for less frequent use, an alphabetic name lower in priority. Because syslog-ng applies configuration files to message alphabetically, and we are using the final function to stop processing a message, simple, frequent log messages can be stripped out in the beginning with first-in-order filters and don't have to pass on to others. Slower filters never get reached, and can be the last of the filters tried before a message drops to messages.

 

[Butterfly Bones]

@elorimer Thank you for the ideas, I will try that optimization soon. My separate ovpn filters are not working as I want, so I have been tweaking them. As usual, Real Life (tm) is getting the way of my router adjustments, so it will be another week or so before I get back to this project.