Scribe scribe - syslog-ng and logrotate installer

[cmkelley]

Just checking in to say scribe has really improved my log viewing of openvpn client events during the development of x3mRouting.

Just noticed that my system log file has no entries. ### Top of Log File ### is the only entry. I have 0 bytes in the log files after the midnight rollover on June 7:

View attachment 18116

This is the first time this has happened. I just upgraded from scribe v2.0_0 (master) to scribe GitHub version: v2.0_1 (master). Then, I bounced openvpn client. Still no entries. This is on 384.12_alpha2-g6a46c21cce.

Edit: A reboot fixed the issue. Seeing log entries again.

Would you mind using the scribe 'lr' function to run logrotate and see if it happens again. And if it does, would you see if just running 's' status or 'rs' restart syslog-ng brings it back? scribe itself shouldn't run when logrotate happens at midnight, only logrotate and syslog-ng (via the HUP signal) should run. The scribe update to v2.0_1 only added a check for firewall-start existing before grepping it for Skynet, and checks service-event to see if it's already been fixed before running a sed replacement against it, so that should have nothing to do with the overnight logrotate run.

Did you upgrade your firmware between scribe v2.0_0 and v2.0_1? @joe scian is having essentially the same problem with his AC5300 on 384.12 alpha2. As far as I know the only for sure "known to work" on 384.12 alpha is the AC86U and AC3100, although the latter required a power removal (not just reboot) to work completely. Not that other models don't work, I've just not seen anyone say it's working.

 

[elorimer]

Working OK on my 87U on 12A1(second build). I rolled back from A2 because other things are borked for the 87U. I have a slightly different startup since I don't cat the syslog.log to messages, but I don't see that affecting anything.

 

[elorimer]

Would you mind using the scribe 'lr' function to run logrotate and see if it happens again.

Just for you, I did this on my 87U with 2.0.0_1, without issue.

 

[cmkelley]

Just for you, I did this on my 87U with 2.0.0_1, without issue.

Noted.

 

[elorimer]

No scribe problems with 12b1 so far.

 

[cmkelley]

No scribe problems with 12b1 so far.

Is that just on the 87U or also your 56U?

 

[L&LD]

RT-AC86U 384.12 Beta 1, no issues, scribe working well.

 

[Jumpstarter]

Not 100% sure what you mean by this ... there is no filter included to send ascd messages to their own log file, so they would go in main "messages" file. Or are none appearing when ascd is not running at ~100% CPU?

So i am noticing i am getting flooded with these in my system logs
acsd: selected channel spec: 0xe09b (149/80) -- my wireless performance is behaving well.
My channel is set to auto--- I know these happen because of that.
I want to know how would i add a filter to send this traffic to its own file for closer monitoring.

 

[Jumpstarter]

Just checking in to say scribe has really improved my log viewing of openvpn client events during the development of x3mRouting.

Just noticed that my system log file has no entries. ### Top of Log File ### is the only entry. I have 0 bytes in the log files after the midnight rollover on June 7:

View attachment 18116

This is the first time this has happened. I just upgraded from scribe v2.0_0 (master) to scribe GitHub version: v2.0_1 (master). Then, I bounced openvpn client. Still no entries. This is on 384.12_alpha2-g6a46c21cce.

Edit: A reboot fixed the issue. Seeing log entries again.

I had this issue when i first installed it for the first time.. a reboot fixed it for me....

 

[elorimer]

Is that just on the 87U or also your 56U?

Just on the 87U. The 56U is offline and I don't have access to it.

 

[elorimer]

So i am noticing i am getting flooded with these in my system logs
acsd: selected channel spec: 0xe09b (149/80) -- my wireless performance is behaving well.
My channel is set to auto--- I know these happen because of that.
I want to know how would i add a filter to send this traffic to its own file for closer monitoring.

Create a config file in syslog-ng.d that is modeled after the pixelserv one in /opt/share/syslog-ng/examples.

 

[cmkelley]

So i am noticing i am getting flooded with these in my system logs
acsd: selected channel spec: 0xe09b (149/80) -- my wireless performance is behaving well.
My channel is set to auto--- I know these happen because of that.
I want to know how would i add a filter to send this traffic to its own file for closer monitoring.

create a file named /opt/etc/syslog-ng.d/ascd

# log all ascd logs to /opt/var/log/ascd.log and stop processing ascd logs

destination d_ascd {
   file("/opt/var/log/ascd.log");
};

filter f_ascd {
   program("ascd");
};

log {
   source(src);
   filter(f_ascd);
   destination(d_ascd);
   flags(final);
};

#eof

create another file named /opt/etc/logrotate.d/ascd

/opt/var/log/ascd.log {
   rotate 2
   postrotate
       /usr/bin/killall -HUP syslog-ng
   endscript
}

You can change the number after rotate to however many you want to keep.

 

[Xentrk]

Would you mind using the scribe 'lr' function to run logrotate and see if it happens again. And if it does, would you see if just running 's' status or 'rs' restart syslog-ng brings it back? scribe itself shouldn't run when logrotate happens at midnight, only logrotate and syslog-ng (via the HUP signal) should run. The scribe update to v2.0_1 only added a check for firewall-start existing before grepping it for Skynet, and checks service-event to see if it's already been fixed before running a sed replacement against it, so that should have nothing to do with the overnight logrotate run.

Did you upgrade your firmware between scribe v2.0_0 and v2.0_1? @joe scian is having essentially the same problem with his AC5300 on 384.12 alpha2. As far as I know the only for sure "known to work" on 384.12 alpha is the AC86U and AC3100, although the latter required a power removal (not just reboot) to work completely. Not that other models don't work, I've just not seen anyone say it's working.

No problem with midnight rollover last night. I had been using scribe v2.0 with 384.12 alpha 2 since June 3. If it happens again, I will run the steps you list in your post.

 

[Jumpstarter]

create a file named /opt/etc/syslog-ng.d/ascd

# log all ascd logs to /opt/var/log/ascd.log and stop processing ascd logs

destination d_ascd {
   file("/opt/var/log/ascd.log");
};

filter f_ascd {
   program("ascd");
};

log {
   source(src);
   filter(f_ascd);
   destination(d_ascd);
   flags(final);
};

#eof

create another file named /opt/etc/logrotate.d/ascd

/opt/var/log/ascd.log {
   rotate 2
   postrotate
       /usr/bin/killall -HUP syslog-ng
   endscript
}

You can change the number after rotate to however many you want to keep.

Unrelated to the post above, but


what does this mean?

syslog: Error locking /var/lock/cfg_mnt.lock: 35 Resource deadlock avoided
 syslog: Error unlocking -1: 9 Bad file descriptor
J

 

[cmkelley]

Unrelated to the post above, but

what does this mean?

syslog: Error locking /var/lock/cfg_mnt.lock: 35 Resource deadlock avoided
 syslog: Error unlocking -1: 9 Bad file descriptor
J

Firstly, it means syslogd is running. Either you disabled syslog-ng and rebooted or something restarted syslogd, and service-event didn't catch it and kill syslogd.

As to what the error itself means, I don't know. I don't know what syslogd uses that file for.

 

[Jumpstarter]

Firstly, it means syslogd is running. Either you disabled syslog-ng and rebooted or something restarted syslogd, and service-event didn't catch it and kill syslogd.

As to what the error itself means, I don't know. I don't know what syslogd uses that file for.

so apparently something is causing syslogd to run concurrently with scribe idk what to do to really troubleshoot this though as I am not aware of when it is actually happening.

 

[Jumpstarter]

Firstly, it means syslogd is running. Either you disabled syslog-ng and rebooted or something restarted syslogd, and service-event didn't catch it and kill syslogd.

As to what the error itself means, I don't know. I don't know what syslogd uses that file for.

it might help me to understand what is causing it if i understood what types of files you are logging maybe something like a script is triggering it.

 

[Butterfly Bones]

it might help me to understand what is causing it if i understood what types of files you are logging maybe something like a script is triggering it.

Use the "d" in the scribe menu. and attach it to a PM for cmkelley.

d.    Generate debug info

 

[cmkelley]

it might help me to understand what is causing it if i understood what types of files you are logging maybe something like a script is triggering it.

I'm not sure I understand your meaning. All of the logs created by syslog-ng are put in the /opt/var/log directory. All of the information comes from the system logging utility, except for 2 files generated by logrotate that syslog-ng reads and puts together into one log file in /opt/var/log. None of the syslog-ng files should be triggering syslogd.

I've tried to get syslogd running and I couldn't. Everything I've tried to start syslogd outside of restarting the logger or time functions has failed to start it. When I use the logger or time functions they are properly captured by service-event and killed off.

Did you edit your service-event file with a windows editor? That would stop service-event from working.

 

[cmkelley]

so apparently something is causing syslogd to run concurrently with scribe idk what to do to really troubleshoot this though as I am not aware of when it is actually happening.

Well, let's see if my assumption is even correct. If you see the error again, run "ps | grep log" and see if syslogd or klogd shows up. Maybe we're chasing a ghost and it's not even syslogd that's causing it.

A point of clarification also. scribe does not run continuously. syslog-ng is the part that runs continuously, all scribe does is set up the system to run syslog-ng and trigger logrotate to run every night. If you're at the command prompt, scribe isn't running. If you're not ssh'd into your router, scribe isn't running. scribe only runs "by itself" when it is called from service-event, in which case it looks for syslogd and klogd, kills them if present, and exits. The next version, 2.1, will also run when the usb is unmounted to stop syslog-ng and restart syslogd & klogd.