Scribe scribe - syslog-ng and logrotate installer

[skeal]

I didn't follow what you mean here by "the other". You create a separate log destination, and then you create a filter that will trap the messages you want to send there, like filtering on "group key" or "group key handshake" in the MSG part, or hostapd in the PROGRAM part.

I was referring to how to create the two ".d" files needed to strip out these log entries.

 

[skeal]

I didn't follow what you mean here by "the other". You create a separate log destination, and then you create a filter that will trap the messages you want to send there, like filtering on "group key" or "group key handshake" in the MSG part, or hostapd in the PROGRAM part.

Thanks you lit a fire under my butt and I figured it out.

 

[marsan71]

Can't install the script. I get error.

                            _
                         _ ( )           
       ___    ___  _ __ (_)| |_      __   
     /',__) /'___)( '__)| || '_`\  /'__`\
     \__, \( (___ | |   | || |_) )(  ___/
     (____/`\____)(_)   (_)(_,__/'`\____)
     syslog-ng and logrotate installation
     v2.4_2 (master)  Coded by cynicastic



 Diversion detected, checking version ... version 4.1 detected ... okay!


 Skynet detected, checking version ... version v7.1.1 detected ... okay!

 fetching scribe from GitHub master branch ...
curl: error while loading shared libraries: libssl.so.1.1: cannot open shared objecty

 scribe GitHub repository is unavailable!  -- Aborting.

 

[skeal]

Can't install the script. I get error.

                            _
                         _ ( )          
       ___    ___  _ __ (_)| |_      __  
     /',__) /'___)( '__)| || '_`\  /'__`\
     \__, \( (___ | |   | || |_) )(  ___/
     (____/`\____)(_)   (_)(_,__/'`\____)
     syslog-ng and logrotate installation
     v2.4_2 (master)  Coded by cynicastic



 Diversion detected, checking version ... version 4.1 detected ... okay!


 Skynet detected, checking version ... version v7.1.1 detected ... okay!

 fetching scribe from GitHub master branch ...
curl: error while loading shared libraries: libssl.so.1.1: cannot open shared objecty

 scribe GitHub repository is unavailable!  -- Aborting.

I think you need to update entware first in your case. What version of router do you have?

 

[cmkelley]

Can't install the script. I get error.

                            _
                         _ ( )          
       ___    ___  _ __ (_)| |_      __  
     /',__) /'___)( '__)| || '_`\  /'__`\
     \__, \( (___ | |   | || |_) )(  ___/
     (____/`\____)(_)   (_)(_,__/'`\____)
     syslog-ng and logrotate installation
     v2.4_2 (master)  Coded by cynicastic



 Diversion detected, checking version ... version 4.1 detected ... okay!


 Skynet detected, checking version ... version v7.1.1 detected ... okay!

 fetching scribe from GitHub master branch ...
curl: error while loading shared libraries: libssl.so.1.1: cannot open shared objecty

 scribe GitHub repository is unavailable!  -- Aborting.

Most likely you need to update entware (please use amtm or diversion to do so), but also make sure your router's clock is set correctly.

 

[skeal]

This is such a handy script @cmkelley Great work on it!! I'm a late adopter but impressed nonetheless!

 

[cmkelley]

This is such a handy script @cmkelley Great work on it!! I'm a late adopter but impressed nonetheless!

Sometimes it pays to be late to the party, you missed out on some really boneheaded errors I made along the way ...

 

[marsan71]

I think you need to update entware first in your case. What version of router do you have?

Thanks @skeal and @cmkelley . It was the entware. After update from amtm I was able to install the script. Working great

 

[gmt]

Hi, after the upgrade to 384.15 on my AC86U the memory usage seems to get pretty high over 90% https://paste.pics/41372da061f4624af89db0e6190b4c9a anyone also experiencing that? On my other router AC68U which is still on 384.12 the memory usage with the same set of scripts is just around 50% https://paste.pics/93f66df0ce3434973a1c319680cc5050. Top command sorted by %MEM shows syslog-ng as the biggest consumer

Mem: 407228K used, 33192K free, 2632K shrd, 11152K buff, 70652K cached
CPU:  0.1% usr  0.2% sys  0.0% nic 99.6% idle  0.0% io  0.0% irq  0.0% sirq
Load average: 2.32 2.24 2.20 2/167 9063
  PID  PPID USER     STAT   VSZ %VSZ CPU %CPU COMMAND
 2104  2099 user      S     155m 36.0   0  0.0 syslog-ng
 2212     1 nobody   S    47676 10.8   0  0.0 pixelserv-tls 192.168.1.2
25196     1 nobody   S    47144 10.6   1  0.0 dnsmasq --log-async
 1135     1 user      S    19980  4.5   1  0.0 amas_lib
  285     1 user      S    18504  4.1   1  0.0 /bin/swmdk
 1089     1 user      S    13832  3.1   1  0.0 roamast
 3297     1 user      S    12368  2.8   0  0.0 minidlna -f /etc/minidlna.conf -r
 3278     1 user      S    11608  2.6   1  0.0 /usr/sbin/smbd -D -s /etc/smb.conf
 3263     1 user      S    11472  2.6   0  0.0 /usr/sbin/nmbd -D -s /etc/smb.conf
 3264  3263 user      S    11172  2.5   0  0.0 /usr/sbin/nmbd -D -s /etc/smb.conf
  917     1 user      S    11036  2.5   1  0.0 nt_monitor
 1119     1 user      S    10820  2.4   0  0.0 cfg_server
  919     1 user      S    10760  2.4   1  0.0 /sbin/netool
  979     1 user      S    10688  2.4   1  0.0 httpd -i br0
 1008   983 user      S    10436  2.3   0  0.0 vis-dcon

 

[Butterfly Bones]

Hi, after the upgrade to 384.15 on my AC86U the memory usage seems to get pretty high over 90% https://paste.pics/41372da061f4624af89db0e6190b4c9a anyone also experiencing that? On my other router AC68U which is still on 384.12 the memory usage with the same set of scripts is just around 50% https://paste.pics/93f66df0ce3434973a1c319680cc5050. Top command sorted by %MEM shows syslog-ng as the biggest consumer

Code:
Mem: 407228K used, 33192K free, 2632K shrd, 11152K buff, 70652K cached
CPU: 0.1% usr 0.2% sys 0.0% nic 99.6% idle 0.0% io 0.0% irq 0.0% sirq
Load average: 2.32 2.24 2.20 2/167 9063
PID PPID USER STAT VSZ %VSZ CPU %CPU COMMAND
2104 2099 user S 155m 36.0 0 0.0 syslog-ng
2212 1 nobody S 47676 10.8 0 0.0 pixelserv-tls 192.168.1.2
25196 1 nobody S 47144 10.6 1 0.0 dnsmasq --log-async
1135 1 user S 19980 4.5 1 0.0 amas_lib
285 1 user S 18504 4.1 1 0.0 /bin/swmdk
1089 1 user S 13832 3.1 1 0.0 roamast
3297 1 user S 12368 2.8 0 0.0 minidlna -f /etc/minidlna.conf -r
3278 1 user S 11608 2.6 1 0.0 /usr/sbin/smbd -D -s /etc/smb.conf
3263 1 user S 11472 2.6 0 0.0 /usr/sbin/nmbd -D -s /etc/smb.conf
3264 3263 user S 11172 2.5 0 0.0 /usr/sbin/nmbd -D -s /etc/smb.conf
917 1 user S 11036 2.5 1 0.0 nt_monitor
1119 1 user S 10820 2.4 0 0.0 cfg_server
919 1 user S 10760 2.4 1 0.0 /sbin/netool
979 1 user S 10688 2.4 1 0.0 httpd -i br0
1008 983 user S 10436 2.3 0 0.0 vis-dcon

One cannot compare memory usage between the AC86U / AC68U, since the newer Broadcom HND SDK uses memory differently that older routers. Here is one post from RMerlin on that.
https://www.snbforums.com/threads/ax88-ram-usage-how-much-is-normal.60349/page-2#post-529459

Search for "broadcom memory" posts by "RMerlin" and you will find dozens more that will help you understand the differences.

 

[Adamm]

One cannot compare memory usage between the AC86U / AX88U, since the newer Broadcom HND SDK uses memory differently that older routers. Here is one post from RMerlin on that.
https://www.snbforums.com/threads/ax88-ram-usage-how-much-is-normal.60349/page-2#post-529459

Search for "broadcom memory" posts by "RMerlin" and you will find dozens more that will help you understand the differences.

Fair statement but I think the key point from his post is that syslog-ng seems to be using quite a high amount of ram regardless of the platform.

2104 2099 user S 155m 36.0 0 0.0 syslog-ng

 

[cmkelley]

Fair statement but I think the key point from his post is that syslog-ng seems to be using quite a high amount of ram regardless of the platform.

2104 2099 user S 155m 36.0 0 0.0 syslog-ng

To be honest, I never looked at syslog-ng memory usage. I always had 96-ish% memory used, and I think ~110 megs of swap used, so I never really looked at it. I'll have a look at my AC86U and my AC3200 when I get home tonight (if I remember ...)

 

[elorimer]

Fair statement but I think the key point from his post is that syslog-ng seems to be using quite a high amount of ram regardless of the platform.

2104 2099 user S 155m 36.0 0 0.0 syslog-ng

This is what I see on my 86U as well. It is six times larger than on my 87U.

 

[cmkelley]

This is what I see on my 86U as well. It is six times larger than on my 87U.

I wonder if it's related to syslog-ng or the system logging service. I would guess it's in syslog-ng, but I wonder if there's a big difference in syslogd between the 86U and a non-HND router?

 

[elorimer]

On both I have 10 destinations defined, so the memory usage should be around 10*256*16384, or 41 meg, meaning 10 destinations times maximum number of messages in the queue times the maximum length of the message.

 

[skeal]

Where do I find Skynet-0.log? I want to edit two stale entries at the beginning referring to ntp not syncing.

 

[visortgw]

Where do I find Skynet-0.log? I want to edit two stale entries at the beginning referring to ntp not syncing.

/opt/var/log

 

[skeal]

/opt/var/log

Thanks!

 

[elorimer]

I think it might be wise to turn off skynet and scribe before you manually edit the file.

 

[cmkelley]

I think it might be wise to turn off skynet and scribe before you manually edit the file.

You don't have to if you're quick. vi will tell you if the file has changed since you opened it, so if you open it, delete the line(s) and save/close it fairly quickly, there's a reasonable chance it won't have changed in that time. If it has, you can quit without saving and try again. I have used this method a few times.