What's new

Scribe scribe - syslog-ng and logrotate installer

  • SNBForums Code of Conduct

    SNBForums is a community for everyone, no matter what their level of experience.

    Please be tolerant and patient of others, especially newcomers. We are all here to share and learn!

    The rules are simple: Be patient, be nice, be helpful or be gone!

v2.1_2 is up

- asks to install uiScribe when you install (not update) scribe
- removes uiScribe if present when you remove scribe
Found it. Love it. Good job! Keep up the good work!
 
Installed scribe and syslog-ng however nothing from the Skynet.log (BLOCKED - INBOUND) is being sent to my syslog server (Splunk). any idea how to change that? :)
 
We might need a little more information, but I'm guessing you have the splunk server configured in the system log|general log|remote server box? Scribe stops the syslogd and klogd daemons and I'm guessing that stops whatever is going on there. I've not experimented with that so I might be totally wrong. Not the first time.

I'm not aware that we've discussed splunk in this thread, but I suspect that you will need to do two things. The first is to define a destination within syslog-ng that is your splunk server, and then a log statement that selects the log statements you want to send, and then send them to that destination.

The second is to work that logging statement into the processing stream, because scribe is built around a single processing sequence using the flags(final) statement: a message goes through the first filter, and if it matches, sends it to that destination and then discards the message. If it doesn't, it tests the next one the same way. Anything that doesn't match the filters is sent to messages. So if you want something to go to the splunk server, and then go through that process, you need to have your splunk processing messages first. You do that by adding your splunk filter/destination/log as a file in /syslog-ng.d with a name that is alphabetically first, like 0splunk, and not including a flags(final) statement. I am doing this with loggly.
 
I am getting data (system messages) from the router into Splunk via syslog just fine however data from Skynet appears to be getting forked out into it’s own log which is not being sent. So I don’t think Splunk is a factor it seems to be more a scribe/syslog-ng issue. This did start after I installed syslog-ng and scribe. My original intention was to filter out all of the "protocol 0800 is buggy...” messages.
 

Attachments

  • 2019-06-17_19-27-10.jpg
    2019-06-17_19-27-10.jpg
    147.3 KB · Views: 403
I see inbound blocks by Skynet just fine with Loggly. I know nothing about splunk. Here is a partial redacted log. copied from Loggy. I'm filtering by "kernel" and "inbound".
Code:
[BLOCKED - INBOUND] IN=eth0 OUT= MAC=88:d7:f6:1d:46:08:00:01:5c:6d:22:46:08:00 SRC=5.39.221.54 DST=[RO.UT.ER.IP] LEN=40 TOS=0x00 PREC=0x00 TTL=237 ID=11160 PROTO=TCP SPT=57790 DPT=9386 SEQ=1890466525 ACK=0 WINDOW=1024 RES=0x00 SYN URGP=0 MARK=0x8000000
2019-06-17 17:37:34.000
[BLOCKED - INBOUND] IN=eth0 OUT= MAC=88:d7:f6:1d:46:08:00:01:5c:6d:22:46:08:00 SRC=152.231.29.79 DST=[RO.UT.ER.IP] LEN=44 TOS=0x00 PREC=0x00 TTL=46 ID=40799 PROTO=TCP SPT=65521 DPT=23 SEQ=1197290991 ACK=0 WINDOW=53977 RES=0x00 SYN URGP=0 OPT (02040218) MARK=0x8000000
2019-06-17 17:36:48.000
[BLOCKED - INBOUND] IN=eth0 OUT= MAC=88:d7:f6:1d:46:08:00:01:5c:6d:22:46:08:00 SRC=185.176.26.100 DST=[RO.UT.ER.IP] LEN=40 TOS=0x00 PREC=0x00 TTL=238 ID=45482 PROTO=TCP SPT=52693 DPT=4301 SEQ=1972933768 ACK=0 WINDOW=1024 RES=0x00 SYN URGP=0 MARK=0x8000000
2019-06-17 17:36:44.000
[BLOCKED - INBOUND] IN=eth0 OUT= MAC=88:d7:f6:1d:46:08:00:01:5c:6d:22:46:08:00 SRC=120.52.152.16 DST=[RO.UT.ER.IP] LEN=41 TOS=0x00 PREC=0x00 TTL=235 ID=15463 PROTO=UDP SPT=49019 DPT=9600 LEN=21 MARK=0x8000000
2019-06-17 17:36:20.000
[BLOCKED - INBOUND] IN=eth0 OUT= MAC=88:d7:f6:1d:46:08:00:01:5c:6d:22:46:08:00 SRC=92.53.65.52 DST=[RO.UT.ER.IP] LEN=40 TOS=0x00 PREC=0x00 TTL=242 ID=52190 PROTO=TCP SPT=41928 DPT=3463 SEQ=1613951926 ACK=0 WINDOW=1024 RES=0x00 SYN URGP=0 MARK=0x8000000
2019-06-17 17:36:06.000
[BLOCKED - INBOUND] IN=eth0 OUT= MAC=88:d7:f6:1d:46:08:00:01:5c:6d:22:46:08:00 SRC=185.176.27.246 DST=[RO.UT.ER.IP] LEN=40 TOS=0x00 PREC=0x00 TTL=240 ID=25579 PROTO=TCP SPT=56994 DPT=3401 SEQ=3627637581 ACK=0 WINDOW=1024 RES=0x00 SYN URGP=0 MARK=0x8000000
2019-06-17 17:34:33.000
[BLOCKED - INBOUND] IN=eth0 OUT= MAC=88:d7:f6:1d:46:08:00:01:5c:6d:22:46:08:00 SRC=45.14.151.10 DST=[RO.UT.ER.IP] LEN=40 TOS=0x00 PREC=0x00 TTL=237 ID=54321 PROTO=TCP SPT=36810 DPT=60001 SEQ=86921169 ACK=0 WINDOW=65535 RES=0x00 SYN URGP=0 MARK=0x8000000
2019-06-17 17:34:23.000
[BLOCKED - INBOUND] IN=eth0 OUT= MAC=88:d7:f6:1d:46:08:00:01:5c:6d:22:46:08:00 SRC=92.118.37.86 DST=[RO.UT.ER.IP] LEN=40 TOS=0x00 PREC=0x00 TTL=236 ID=9151 PROTO=TCP SPT=54659 DPT=40110 SEQ=3830024958 ACK=0 WINDOW=1024 RES=0x00 SYN URGP=0 MARK=0x8000000
2019-06-17 17:32:45.000
[BLOCKED - INBOUND] IN=eth0 OUT= MAC=88:d7:f6:1d:46:08:00:01:5c:6d:22:46:08:00 SRC=222.186.174.95 DST=[RO.UT.ER.IP] LEN=40 TOS=0x00 PREC=0x00 TTL=106 ID=256 PROTO=TCP SPT=42687 DPT=2433 SEQ=2780758016 ACK=0 WINDOW=16384 RES=0x00 SYN URGP=0 MARK=0x8000000
2019-06-17 17:28:42.000
[BLOCKED - INBOUND] IN=eth0 OUT= MAC=88:d7:f6:1d:46:08:00:01:5c:6d:22:46:08:00 SRC=81.22.45.22 DST=[RO.UT.ER.IP] LEN=40 TOS=0x00 PREC=0x00 TTL=242 ID=27240 PROTO=TCP SPT=54020 DPT=3344 SEQ=258822807 ACK=0 WINDOW=1024 RES=0x00 SYN URGP=0 MARK=0x8000000
2019-06-17 17:27:17.000
[BLOCKED - INBOUND] IN=eth0 OUT= MAC=88:d7:f6:1d:46:08:00:01:5c:6d:22:46:08:00 SRC=146.88.240.4 DST=[RO.UT.ER.IP] LEN=63 TOS=0x00 PREC=0x00 TTL=240 ID=54321 PROTO=UDP SPT=37892 DPT=53 LEN=43 MARK=0x8000000
2019-06-17 17:26:52.000
[BLOCKED - INBOUND] IN=eth0 OUT= MAC=88:d7:f6:1d:46:08:00:01:5c:6d:22:46:08:00 SRC=178.73.215.171 DST=[RO.UT.ER.IP] LEN=44 TOS=0x00 PREC=0x00 TTL=237 ID=54321 PROTO=TCP SPT=35426 DPT=102 SEQ=3031229858 ACK=0 WINDOW=65535 RES=0x00 SYN URGP=0 OPT (020405AC) MARK=0x8000000
2019-06-17 17:26:13.000
[BLOCKED - INBOUND] IN=eth0 OUT= MAC=88:d7:f6:1d:46:08:00:01:5c:6d:22:46:08:00 SRC=83.209.98.36 DST=[RO.UT.ER.IP] LEN=40 TOS=0x00 PREC=0x00 TTL=40 ID=53933 PROTO=TCP SPT=22730 DPT=23 SEQ=1197290991 ACK=0 WINDOW=18648 RES=0x00 SYN URGP=0 MARK=0x8000000
2019-06-17 17:25:58.000
[BLOCKED - INBOUND] IN=eth0 OUT= MAC=88:d7:f6:1d:46:08:00:01:5c:6d:22:46:08:00 SRC=5.39.221.54 DST=[RO.UT.ER.IP] LEN=40 TOS=0x00 PREC=0x00 TTL=236 ID=35501 PROTO=TCP SPT=56345 DPT=9385 SEQ=2862521323 ACK=0 WINDOW=1024 RES=0x00 SYN URGP=0 MARK=0x8000000
2019-06-17 17:25:55.000
[BLOCKED - INBOUND] IN=eth0 OUT= MAC=88:d7:f6:1d:46:08:00:01:5c:6d:22:46:08:00 SRC=49.67.132.168 DST=[RO.UT.ER.IP] LEN=40 TOS=0x00 PREC=0x00 TTL=46 ID=27152 PROTO=TCP SPT=45013 DPT=23 SEQ=1197290991 ACK=0 WINDOW=39397 RES=0x00 SYN URGP=0 MARK=0x8000000
2019-06-17 17:25:25.000
[BLOCKED - INBOUND] IN=eth0 OUT= MAC=88:d7:f6:1d:46:08:00:01:5c:6d:22:46:08:00 SRC=81.22.45.251 DST=[RO.UT.ER.IP] LEN=40 TOS=0x00 PREC=0x00 TTL=242 ID=58837 PROTO=TCP SPT=51251 DPT=5927 SEQ=2745485085 ACK=0 WINDOW=1024 RES=0x00 SYN URGP=0 MARK=0x8000000
2019-06-17 17:25:05.000
[BLOCKED - INBOUND] IN=eth0 OUT= MAC=88:d7:f6:1d:46:08:00:01:5c:6d:22:46:08:00 SRC=60.190.56.9 DST=[RO.UT.ER.IP] LEN=40 TOS=0x00 PREC=0x00 TTL=99 ID=256 PROTO=TCP SPT=39091 DPT=1433 SEQ=1921318912 ACK=0 WINDOW=16384 RES=0x00 SYN URGP=0 MARK=0x8000000
2019-06-17 17:24:58.000
[BLOCKED - INBOUND] IN=eth0 OUT= MAC=88:d7:f6:1d:46:08:00:01:5c:6d:22:46:08:00 SRC=37.49.231.105 DST=[RO.UT.ER.IP] LEN=40 TOS=0x00 PREC=0x00 TTL=235 ID=35840 PROTO=TCP SPT=55744 DPT=50802 SEQ=1808498589 ACK=0 WINDOW=1024 RES=0x00 SYN URGP=0 MARK=0x8000000
2019-06-17 17:24:51.000
[BLOCKED - INBOUND] IN=eth0 OUT= MAC=88:d7:f6:1d:46:08:00:01:5c:6d:22:46:08:00 SRC=5.188.210.158 DST=[RO.UT.ER.IP] LEN=40 TOS=0x00 PREC=0x00 TTL=237 ID=2483 PROTO=TCP SPT=58913 DPT=8181 SEQ=3839239081 ACK=0 WINDOW=1024 RES=0x00 SYN URGP=0 MARK=0x8000000
2019-06-17 17:23:37.000
[BLOCKED - INBOUND] IN=eth0 OUT= MAC=88:d7:f6:1d:46:08:00:01:5c:6d:22:46:08:00 SRC=34.87.16.239 DST=[RO.UT.ER.IP] LEN=40 TOS=0x00 PREC=0x00 TTL=229 ID=19479 PROTO=TCP SPT=32767 DPT=8545 SEQ=1404065945 ACK=0 WINDOW=1024 RES=0x00 SYN URGP=0 MARK=0x8000000
2019-06-17 17:23:21.000
[BLOCKED - INBOUND] IN=eth0 OUT= MAC=88:d7:f6:1d:46:08:00:01:5c:6d:22:46:08:00 SRC=81.22.45.22 DST=[RO.UT.ER.IP] LEN=40 TOS=0x00 PREC=0x00 TTL=242 ID=64198 PROTO=TCP SPT=54020 DPT=3314 SEQ=290112475 ACK=0 WINDOW=1024 RES=0x00 SYN URGP=0 MARK=0x8000000
2019-06-17 17:22:49.000
[BLOCKED - INBOUND] IN=eth0 OUT= MAC=88:d7:f6:1d:46:08:00:01:5c:6d:22:46:08:00 SRC=162.243.145.44 DST=[RO.UT.ER.IP] LEN=40 TOS=0x00 PREC=0x00 TTL=244 ID=54321 PROTO=TCP SPT=37336 DPT=2078 SEQ=3614270277 ACK=0 WINDOW=65535 RES=0x00 SYN URGP=0 MARK=0x8000000
2019-06-17 17:21:48.000
[BLOCKED - INBOUND] IN=eth0 OUT= MAC=88:d7:f6:1d:46:08:00:01:5c:6d:22:46:08:00 SRC=5.188.86.114 DST=[RO.UT.ER.IP] LEN=40 TOS=0x00 PREC=0x00 TTL=239 ID=9588 PROTO=TCP SPT=56949 DPT=2016 SEQ=1547179899 ACK=0 WINDOW=1024 RES=0x00 SYN URGP=0 MARK=0x8000000
2019-06-17 17:21:25.000
[BLOCKED - INBOUND] IN=eth0 OUT= MAC=88:d7:f6:1d:46:08:00:01:5c:6d:22:46:08:00 SRC=81.22.45.254 DST=[RO.UT.ER.IP] LEN=40 TOS=0x00 PREC=0x00 TTL=242 ID=13275 PROTO=TCP SPT=48815 DPT=7389 SEQ=174791401 ACK=0 WINDOW=1024 RES=0x00 SYN URGP=0 MARK=0x8000000
2019-06-17 17:20:56.000
[BLOCKED - INBOUND] IN=eth0 OUT= MAC=88:d7:f6:1d:46:08:00:01:5c:6d:22:46:08:00 SRC=81.22.45.252 DST=[RO.UT.ER.IP] LEN=40 TOS=0x00 PREC=0x00 TTL=242 ID=43551 PROTO=TCP SPT=44517 DPT=455 SEQ=1305124407 ACK=0 WINDOW=1024 RES=0x00 SYN URGP=0 MARK=0x8000000
2019-06-17 17:19:36.000
[BLOCKED - INBOUND] IN=eth0 OUT= MAC=88:d7:f6:1d:46:08:00:01:5c:6d:22:46:08:00 SRC=178.128.214.153 DST=[RO.UT.ER.IP] LEN=40 TOS=0x00 PREC=0x00 TTL=241 ID=52693 PROTO=TCP SPT=52252 DPT=3389 SEQ=1507783943 ACK=0 WINDOW=1024 RES=0x00 SYN URGP=0 MARK=0x8000000
2019-06-17 17:19:34.000
[BLOCKED - INBOUND] IN=eth0 OUT= MAC=88:d7:f6:1d:46:08:00:01:5c:6d:22:46:08:00 SRC=185.176.26.101 DST=[RO.UT.ER.IP] LEN=40 TOS=0x00 PREC=0x00 TTL=237 ID=63003 PROTO=TCP SPT=52697 DPT=4920 SEQ=2799161548 ACK=0 WINDOW=1024 RES=0x00 SYN URGP=0 MARK=0x8000000
2019-06-17 17:19:33.000
[BLOCKED - INBOUND] IN=eth0 OUT= MAC=88:d7:f6:1d:46:08:00:01:5c:6d:22:46:08:00 SRC=185.176.27.58 DST=[RO.UT.ER.IP] LEN=40 TOS=0x00 PREC=0x00 TTL=238 ID=63319 PROTO=TCP SPT=55350 DPT=7026 SEQ=2658523594 ACK=0 WINDOW=1024 RES=0x00 SYN URGP=0 MARK=0x8000000
2019-06-17 17:19:14.000
[BLOCKED - INBOUND] IN=eth0 OUT= MAC=88:d7:f6:1d:46:08:00:01:5c:6d:22:46:08:00 SRC=111.182.39.109 DST=[RO.UT.ER.IP] LEN=28 TOS=0x00 PREC=0x00 TTL=107 ID=52789 PROTO=ICMP TYPE=8 CODE=0 ID=3445 SEQ=54887 MARK=0x8000000
 
I am getting data (system messages) from the router into Splunk via syslog just fine however data from Skynet appears to be getting forked out into it’s own log which is not being sent. So I don’t think Splunk is a factor it seems to be more a scribe/syslog-ng issue. This did start after I installed syslog-ng and scribe. My original intention was to filter out all of the "protocol 0800 is buggy...” messages.
I don't know how Splunk works ... if Skynet is detected when installing scribe, it adds a filter to put the Skynet messages into /opt/var/log/skynet-0.log, along with a number of other filters. All of the log files are put into /opt/var/log, you can only see /tmp/syslog.log because it is symlinked to /opt/var/log/messages, which is the default logging file.

I'm guessing, because I know zero about Splunk, that it is only looking in /tmp/syslog.log for messages, so it's only seeing messages that are not being filtered elsewhere. If you look in /opt/var/log you will see a number of these other logs. You need to figure out how to point Splunk at the logs you want in /opt/var/log.

You might be tempted to just delete the skynet filter so the Skynet files will go into /opt/var/log/messages and thus be visible in syslog.log. Let me save you the trouble, it won't work. :) The problem is Skynet will wipe out the symlink the first time it runs its hourly cleanup and you won't see anything else in /tmp/syslog.log because the symlink will be gone.
 
Still unclear on how things go to splunk, but if you want the skynet messages to drop to the symlink messages file, then delete the part in the skynet configuration file that says "flags(final)". The message will then continue on to the messages file, and if that file is going to splunk then the other messages will too.

You can do the opposite to filter out the protocol 0800 messages. Use a filter that matches that, and either file the statement or drop it using flags(final). It won't go to messages then.
 
Still unclear on how things go to splunk, but if you want the skynet messages to drop to the symlink messages file, then delete the part in the skynet configuration file that says "flags(final)". The message will then continue on to the messages file, and if that file is going to splunk then the other messages will too.

You can do the opposite to filter out the protocol 0800 messages. Use a filter that matches that, and either file the statement or drop it using flags(final). It won't go to messages then.
Heh, oh yeah, forgot about dropping the flags(final) so it would go to both places. Good point.
 
Is there a way to hide the IOT - BLOCKED events from showing in the syslog?
(moved here to keep it on topic)

I don't have any IoT devices, so I don't know exactly what the log messages look like, but syslog-ng can easily filter them out. The thing I don't know is if or how they're related to Skynet. Do they show up only with or only without Skynet installed? Or do they show up regardless. It doesn't make a huge difference but if they're related to Skynet one way or another, I'd like to just add the filters to the appropriate filter instead of creating a new filter.
 
Code:
Do you want to continue installation of scribe [y|n]? y

 fetching scribe from GitHub master branch ...
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
curl: (60) SSL certificate problem: certificate is not yet valid
More details here: https://curl.haxx.se/docs/sslcerts.html

curl failed to verify the legitimacy of the server and therefore could not
establish a secure connection to it. To learn more about this situation and
how to fix it, please visit the web page mentioned above.

 scribe GitHub repository is unavailable!  -- Aborting.
 
Code:
Do you want to continue installation of scribe [y|n]? y

 fetching scribe from GitHub master branch ...
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
curl: (60) SSL certificate problem: certificate is not yet valid
More details here: https://curl.haxx.se/docs/sslcerts.html

curl failed to verify the legitimacy of the server and therefore could not
establish a secure connection to it. To learn more about this situation and
how to fix it, please visit the web page mentioned above.

 scribe GitHub repository is unavailable!  -- Aborting.
Check your system time
 
Check your system time
You are correct... I think I may have found the culprit too the clock was not getting set between the amount of time it took for the router to set the time, and the time the mounting of the usb disk starts. I have amtm with the disk check feature added and for some reason it forces the router to use router default time and completely skip getting the correct time, if the wait time to get the correct time takes too long.
Note this is on a router that is not using ntpmerlin, but the default ntp client.
 
My solution to the issue was simply adding a sufficient enough sleep time to the pre-mount that was dependent on whether ntp was ready on the nvram variable.
 
My solution to the issue was simply adding a sufficient enough sleep time to the pre-mount that was dependent on whether ntp was ready on the nvram variable.
If you have another box on your network that can serve time, you can add the script at https://gist.github.com/cynicastic/ced78fac27de4394b67977802d76c0d9 to /jffs/scripts, make it executable, and call it from init-start as described in the comments in the script. That will set the time as early as possible. :)
 
If you have another box on your network that can serve time, you can add the script at https://gist.github.com/cynicastic/ced78fac27de4394b67977802d76c0d9 to /jffs/scripts, make it executable, and call it from init-start as described in the comments in the script. That will set the time as early as possible. :)
I have seen these types of scripts, but never used them. Will the ntp ready variable reach 1 if a script like this is used way early on in the boot?
my solution was to simple create a wait loop around the ntp ready variable and only starting the disk check if the parameters were true.
 
I have seen these types of scripts, but never used them. Will the ntp ready variable reach 1 if a script like this is used way early on in the boot?
my solution was to simple create a wait loop around the ntp ready variable and only starting the disk check if the parameters were true.
Yes, the script sets it at line 33 once it successfully syncs the time. :)
 
i will give both routes a test, thank you for the suggestion.
 
Yes, the script sets it at line 33 once it successfully syncs the time. :)
Though your script works well,
i chose to go with this because it allows me to keep my hands out of actually modifying the nvram, with to many unknowns and what ifs, i feel more comfortable with a wait loop that breaks away when the condition is met.
Code:
#!/bin/sh
logger -t "$(basename $0)" "checking if NTP is Ready, before starting Disk Check"
if [ "$(nvram get ntp_ready)" = "1" ]; then
logger -t "$(basename $0)" "NTP is Ready, Starting Disk Check"
. /jffs/scripts/disk-check # Added by amtm
else
while [ "$(nvram get ntp_ready)" = "0" ]
do
sleep 1
       if [ "$(nvram get ntp_ready)" = "1" ]; then
       break
       . /jffs/scripts/disk-check # Added by amtm
       logger -t "$(basename $0)" "NTP is Ready, Starting Disk Check"
       fi
done
fi
 
Though your script works well,
i chose to go with this because it allows me to keep my hands out of actually modifying the nvram, with to many unknowns and what ifs, i feel more comfortable with a wait loop that breaks away when the condition is met.
Code:
#!/bin/sh
logger -t "$(basename $0)" "checking if NTP is Ready, before starting Disk Check"
if [ "$(nvram get ntp_ready)" = "1" ]; then
logger -t "$(basename $0)" "NTP is Ready, Starting Disk Check"
. /jffs/scripts/disk-check # Added by amtm
else
while [ "$(nvram get ntp_ready)" = "0" ]
do
sleep 1
       if [ "$(nvram get ntp_ready)" = "1" ]; then
       break
       . /jffs/scripts/disk-check # Added by amtm
       logger -t "$(basename $0)" "NTP is Ready, Starting Disk Check"
       fi
done
fi
This is already built into amtm disk-check...
 

Latest threads

Sign Up For SNBForums Daily Digest

Get an update of what's new every day delivered to your mailbox. Sign up here!
Top