Scribe scribe - syslog-ng and logrotate installer

[cmkelley]

It's not midnight yet on the left coast of USizicstania, so, last scribe update of the 2nd decade of the 3rd millennia since the approximate birth year of a Galilean Jewish carpenter who is recognized by Christians as the son of the deity of Abrahamic religions. No, I haven't had anything to drink yet.

In Other Words, scribe v2.4_1 is up.

• Fixed some typos, a bit of code cleanup
• Added syslog-ng and logrotate files for spdMerlin logs to the respective examples directories
• Added an option on scribe utilities menu (su) to run syslog-ng debugging mode, which:
  • Stops syslog-ng if running (warns that this option is intended for use when syslog-ng will not start)
  • Runs syslog-ng in the foreground, showing debugging messages by executing syslog-ng -Fevd.
  • Restarts syslog-ng if it had been running before

Cheers!

 

[SomeWhereOverTheRainBow]

scribe v2.4_0 pushed

• Adds check for Skynet version; currently requires 6.9.2 or later
• Removed time_reap(2) from Skynet filter; as a result syslog-ng 3.19 is lowest working version, was 3.23 before Skynet update
• Added an ntpd log handler & logrotate, Entware's ntpd logs to /opt/var/spool/ntp/ntp.log; read that file and ntpd logs in system log and put into /opt/var/log/ntp.log
• Clarified the show config shows the on-disk configuration, not the loaded configuration; added the ability to view the loaded configuration
• Both on-disk and loaded configuration are now output in debugging file
• Removed "update-filters" as an accepted command line option; "filters" still works, but is "undocumented", command line use is intended for use by the update process only
• Added "reload" as a command line option to reload the configuration; previously was available only in the menu
  
• Added log handler & logrotate file for "bcm63xxx" messages
• Added redaction of usb drive names in debugging log; note drive names with a comma in them won't be redacted (I had to pick SOMETHING for a sed delimiter), will print a message informing that drive name wasn't redacted if it has a comma in it
• Updated the wlceventd filter; I used a slightly different method than proposed in this thread, but I haven't updated to Merlin 384.14 yet, so I haven't tested it

Have fun, I hope I didn't break anything.

Why the regression with skynet filter was it causing an issue?

 

[thelonelycoder]

It's not midnight yet on the left coast of USizicstania, so, last scribe update of the 2nd decade of the 3rd millennia since the approximate birth year of a Galilean Jewish carpenter who is recognized by Christians as the son of the deity of Abrahamic religions. No, I haven't had anything to drink yet.

In Other Words, scribe v2.4_1 is up.

• Fixed some typos, a bit of code cleanup
• Added syslog-ng and logrotate files for spdMerlin logs to the respective examples directories
• Added an option on scribe utilities menu (su) to run syslog-ng debugging mode, which:
  • Stops syslog-ng if running (warns that this option is intended for use when syslog-ng will not start)
  • Runs syslog-ng in the foreground, showing debugging messages by executing syslog-ng -Fevd.
  • Restarts syslog-ng if it had been running before

Cheers!

I had to check again to make sure your 'stubborn' version naming scheme still works in amtm. It does, read the comment in the file if you're curious...

 

[SomeWhereOverTheRainBow]

It's not midnight yet on the left coast of USizicstania, so, last scribe update of the 2nd decade of the 3rd millennia since the approximate birth year of a Galilean Jewish carpenter who is recognized by Christians as the son of the deity of Abrahamic religions. No, I haven't had anything to drink yet.

In Other Words, scribe v2.4_1 is up.

• Fixed some typos, a bit of code cleanup
• Added syslog-ng and logrotate files for spdMerlin logs to the respective examples directories
• Added an option on scribe utilities menu (su) to run syslog-ng debugging mode, which:
  • Stops syslog-ng if running (warns that this option is intended for use when syslog-ng will not start)
  • Runs syslog-ng in the foreground, showing debugging messages by executing syslog-ng -Fevd.
  • Restarts syslog-ng if it had been running before

Cheers!

Keep up the great work and happy New Years to you.

 

[cmkelley]

I had to check again to make sure your 'stubborn' version naming scheme still works in amtm. It does, read the comment in the file if you're curious...

It is finally 2020 here, and I'm 2 rum and cokes in now ... which comment? Mine in scribe or is there one in amtm I missed?

I haven't changed how scribe's versioning works internally since 2.0_0. I follow your rules to play nice with amtm, but I like my way of displaying the version number. We can both be happy.

 

[cmkelley]

Why the regression with skynet filter was it causing an issue?

No, but Adamm changed the Skynet code to obviate the need for time_reap(2).

 

[cmkelley]

Keep up the great work and happy New Years to you.

Thanks, Happy New Year to you as well!

 

[thelonelycoder]

It is finally 2020 here, and I'm 2 rum and cokes in now ... which comment? Mine in scribe or is there one in amtm I missed?

I haven't changed how scribe's versioning works internally since 2.0_0. I follow your rules to play nice with amtm, but I like my way of displaying the version number. We can both be happy.

https://github.com/cynicastic/scribe/blob/98eea3a94baf96abbf121a1bd097a0654b228028/scribe#L40

 

[SomeWhereOverTheRainBow]

No, but Adamm changed the Skynet code to obviate the need for time_reap(2).

@Adamm will always find the obvious solutions that every one else will miss. Brilliance in the face of adversity. (Putting out fires for a living).

 

[Butterfly Bones]

Looks like Github is nursing a New Years hangover this morning.

scribe GitHub repository is unavailable! -- Aborting

edit -
Figured it out, I was still running the cmkelley "guinea pig" version. Changing the lines commented about "branch" resolved it.

 

[nomore4u]

It's not midnight yet on the left coast of USizicstania, so, last scribe update of the 2nd decade of the 3rd millennia since the approximate birth year of a Galilean Jewish carpenter who is recognized by Christians as the son of the deity of Abrahamic religions. No, I haven't had anything to drink yet.

In Other Words, scribe v2.4_1 is up.

• Fixed some typos, a bit of code cleanup
• Added syslog-ng and logrotate files for spdMerlin logs to the respective examples directories
• Added an option on scribe utilities menu (su) to run syslog-ng debugging mode, which:
  • Stops syslog-ng if running (warns that this option is intended for use when syslog-ng will not start)
  • Runs syslog-ng in the foreground, showing debugging messages by executing syslog-ng -Fevd.
  • Restarts syslog-ng if it had been running before

Cheers!

I did a fresh (never installed before) install of scribe v2.4_1 with Asuswrt-Merlin 384.14_2.

Why do I get this message: "reloading syslog-ng.conf ... EOF occurred while reading control socket"

The WLCEVENTD messages appear in System Messages and the filter for wlceventd does not appear to update.


Update available for /opt/etc/syslog-ng.d/wlceventd.
(a)ccept, (r)eject, or (v)iew diff for this file? a

/opt/etc/syslog-ng.d/wlceventd updated!

syslog-ng and logrotate example files updated!
reloading syslog-ng.conf ... EOF occurred while reading control socket

 

[Jack Yaz]

I did a fresh (never installed before) install of scribe v2.4_1 with Asuswrt-Merlin 384.14_2.

Why do I get this message: "reloading syslog-ng.conf ... EOF occurred while reading control socket"

The WLCEVENTD messages appear in System Messages and the filter for wlceventd does not appear to update.


Update available for /opt/etc/syslog-ng.d/wlceventd.
(a)ccept, (r)eject, or (v)iew diff for this file? a

/opt/etc/syslog-ng.d/wlceventd updated!

syslog-ng and logrotate example files updated!
reloading syslog-ng.conf ... EOF occurred while reading control socket

https://www.snbforums.com/threads/scribe-syslog-ng-and-logrotate-installer.55853/page-45#post-525440

NOTE: for certain routers (I think any armv71 router) you will see error messages similar to:
Code:
reloading syslog-ng.conf ... EOF occurred while reading control socket

As far as I can tell, this is a bug in the armv71 version of syslog-ng in Entware, but more importantly, appears to have no effect on operation of syslog-ng.

How are you checking if the filter isn't updating? use option v to see what's happening

 

[nomore4u]

https://www.snbforums.com/threads/scribe-syslog-ng-and-logrotate-installer.55853/page-45#post-525440

How are you checking if the filter isn't updating? use option v to see what's happening

Sorry, I am newbie. Where do I see and use option v?

 

[Jack Yaz]

Sorry, I am newbie. Where do I see and use option v?

When prompted for the filter update (see your post where you used option a at the prompt)

 

[nomore4u]

When prompted for the filter update (see your post where you used option a at the prompt)

Thank you for the help and the quick responses. I don't get an option v now. I guess I ran update enough times that it finally updated.

I still see this message in System Messages: ...syslog: WLCEVENTD wlceventd_proc_event(449): eth2...

Maybe that is normal to see in System Messages?

 

[elorimer]

There are a few things on my list to run down.

1. The unexpected EOF, which as noted appeared when loading syslog-ng on my 87U, but not on my 86U.
2. On the 86U, a new message on loading "Error connecting control socket, socket='/opt/var/syslog-ng.ctl', error='Connection refused'
3. Maybe related to the above, the 86U is not finding /var/lib/logrotate/logrotate.status. I don't have that file, and I'm not sure what is trying to find it in /var as opposed to /opt/var.
4. The loggly destination keeps getting an EOF message too.

But I think I may wait until 3.25 rolls around--the visual configuration viewer looks like it might be interesting.

 

[cmkelley]

Thank you for the help and the quick responses. I don't get an option v now. I guess I ran update enough times that it finally updated.

I still see this message in System Messages: ...syslog: WLCEVENTD wlceventd_proc_event(449): eth2...

Maybe that is normal to see in System Messages?

Huh ... I have never got the message you're getting, so I updated the wlceventd filter based on what people were posting. I thought people had reported that my approach fixed it? Are you seeing new 'syslog: WLCEVENTD' entries or are those the ones that were there before the filter update? It doesn't go back and retroactively scrape the log.

What's the output of

cat /opt/etc/syslog-ng.d/wlceventd

 

[nomore4u]

Huh ... I have never got the message you're getting, so I updated the wlceventd filter based on what people were posting. I thought people had reported that my approach fixed it? Are you seeing new 'syslog: WLCEVENTD' entries or are those the ones that were there before the filter update? It doesn't go back and retroactively scrape the log.

I don't believe I am seeing new 'syslog: WLCEVENTD' messages. They might have been there before the filter update. What is the best way to clear the log to ensure they are not new?

What's the output of

cat /opt/etc/syslog-ng.d/wlceventd

# put wlceventd Assoc/ReAssoc/Disassoc messages into /opt/var/log/wlceventd.log
destination d_wlceventd {
file("/opt/var/log/wlceventd.log");
};
filter f_wlceventd {
( program("WLCEVENTD") or
program("wlceventd") ) and
( message("ssoc") or
message("uth") ) or
( program("syslog") and
message("wlceventd") );
};
log {
source(src);
filter(f_wlceventd);
destination(d_wlceventd);
flags(final);
};
#eof

 

[cmkelley]

There are a few things on my list to run down.

1. The unexpected EOF, which as noted appeared when loading syslog-ng on my 87U, but not on my 86U.
2. On the 86U, a new message on loading "Error connecting control socket, socket='/opt/var/syslog-ng.ctl', error='Connection refused'
3. Maybe related to the above, the 86U is not finding /var/lib/logrotate/logrotate.status. I don't have that file, and I'm not sure what is trying to find it in /var as opposed to /opt/var.
4. The loggly destination keeps getting an EOF message too.

But I think I may wait until 3.25 rolls around--the visual configuration viewer looks like it might be interesting.

1. I'm pretty sure is a bug in some versions of syslog-ng. I get it on my AC3200 but like you, not on my 86U.
2. Hrm, not getting that one on either router. Maybe it's unique to 384.15? I've got 384.14_1-g35d1fd1184 on my 86U.
3. I'm guessing you've rebooted your router since midnight last night. That file gets created by logrotate when logrotate is run. Since it's in /var, it gets wiped on router reboot, and doesn't get re-created until logrotate runs the first time (at 00:05). Use scribe option lr and you'll see it's created then.
4. Can't help you with loggly, sorry.

I looked at the description of the configuration viewer ... IIRC, it creates a graphics file. Dunno if I'll implement a hook to that since it would require getting the file off the router to view it.

It doesn't look like anything in 3.24 or 3.25 will break the current configuration, but with these guys you never know.

 

[cmkelley]

I don't believe I am seeing new 'syslog: WLCEVENTD' messages. They might have been there before the filter update. What is the best way to clear the log to ensure they are not new?

# put wlceventd Assoc/ReAssoc/Disassoc messages into /opt/var/log/wlceventd.log
destination d_wlceventd {
file("/opt/var/log/wlceventd.log");
};
filter f_wlceventd {
( program("WLCEVENTD") or
program("wlceventd") ) and
( message("ssoc") or
message("uth") ) or
( program("syslog") and
message("wlceventd") );
};
log {
source(src);
filter(f_wlceventd);
destination(d_wlceventd);
flags(final);
};
#eof

Okay, so your wlceventd filter got updated, that's good.

To wipe the old ones,

sed -i "/WLCEVENTD/d" /opt/var/log/messages

Or, look in the wlceventd log to see if there are some instances of that message in that log file. If they are there, they're not going to messages.

cat /opt/var/log/wlceventd.log

Or, you should be able to tell by the timestamps in the messages log.