Notifications (E-Mail)

[sfx2000]

That's pretty much consistent across the board these days - and even then, one should have a valid from header, or it may get snatched by spam filters if nothing else.

 

[RMerlin]

That's pretty much consistent across the board these days - and even then, one should have a valid from header, or it may get snatched by spam filters if nothing else.

It's something new around here. Most local ISPs here never had any problem allowing outbound mail from a different domain. Actually, that actual ISP's mobile network would even block outbound ports 25 AND 587, and forced you to use their SMTP when sending mail over 3G/HSPA. It was a pain for me back when my email was hosted on a standard cPanel server, it only got resolved when I migrated to GApps. They whitelist some of those big providers such as Office365 and GApps...

I left them for a different mobile provider last year. This wasn't the only thing that sucked about their mobile network.

 

[RMerlin]

Rogers also started to implement a similar policy a few months ago.

 

[Jean-Roch Blais]

But is the From: field also using an address @videotron.ca? This is a new requirement (and I suspect the address must also exist on their servers).

Well I'll be darned !!! you were partly right you guru ! , I was using:
"sendmail -S smtp.videotron.ca -f root@firewall <blablalba>@gmail.com < /tmp/MyIP1.log" which didn't work, but using :
"sendmail -S smtp.videotron.ca -f root@firewall <blablabla>@videotron.ca < /tmp/MyIP1.log" works very well indeed !
It looks like the FROM field is not important, but the RECIPIENT_EMAIL has to be <blablabla>@videotron.ca

I wrote an e-mail to Videotron, asking them what they changed, I received the usual useless answer : "Please visit http://soutien.videotron.com/residentiel/internet/configurer-courriel-videotron" ... what a joke! , which brings me to ask you how you knew about this new requirement or changes that were progressively deploying over the past couple of months... if you feel like telling me of course !
Once again, thanks for pointing me in the right direction !
jrb

 

[Jean-Roch Blais]

Here's my full SASL config for postfix

# See /usr/share/postfix/main.cf.dist for a commented, more complete version

# TLS parameters
smtp_use_tls = yes
smtp_tls_CAfile = /etc/ssl/certs/ca-certificates.crt

myhostname = testbox
relayhost = [smtp.gmail.com]:587

# SASL Parameters
smtp_sasl_auth_enable = yes
smtp_sasl_password_maps = hash:/etc/postfix/sasl_passwd
smtp_sasl_security_options =

Again, we're using Gmail, so note that the username/pass aren't in main.cf, they'll be in another file…

sudo nano /etc/postfix/sasl_passwd

add the following line

[smtp.gmail.com]:587    gmailuser@gmail.com:gmailpassword

gmailuser/gmailpassword is the userid/pass for the gmail account that we're using

Now we fix perms and update the postfix config to use the sasl_passwd file

sudo chmod 400 /etc/postfix/sasl_passwd
sudo postmap /etc/postfix/sasl_passwd

And we then reload postfix

sudo service postfix reload

To test - the following should generate an email from your server to your gmail inbox - yes, this <line> is deliberately <broken> for a <reason> - if you don't remove the <brackets> - you'll get an error

echo "Test mail from postfix" | mail -s "Test Postfix" <gmailuser>@gmail.com

So having this info, someone should be able to create a script around this for AsusWRT...

Thanks for this answer, but I'm using sendmail in my bering-uclibc firewall (http://leaf.sourceforge.net/bering-uclibc/), there is no postfix package available in this distro.
jrb.

 

[sfx2000]

True - but there may be other SMTP MTA's (Postfix is a bit heavy for embedded devices) that would work similar..

Growl would be an interesting alternative perhaps... no email needed there - just a growl agent and client SW.. some of the VPN providers use growl for example to send various alerts to users.

 

[sfx2000]

Thanks for this answer, but I'm using sendmail in my bering-uclibc firewall (http://leaf.sourceforge.net/bering-uclibc/), there is no postfix package available in this distro.
jrb.

Just saw that Entware does have a postfix package that support musl... should be able to run musl and uclibc together perhaps...

https://github.com/Entware/openwrt-packages/tree/master/mail/postfix

 

[Jean-Roch Blais]

Vidéotron no longer allows relaying emails from domain that aren't @videotron.ca. The change was progressively deployed over the past couple of months.

Note that GMail also offers app-specific passwords. You could use that capacity to generate a password dedicated to your outgoing notifications. I don't know if they still require you to globally enable access from less-secure applications, or if it's a selective bypass.

for your info, this is the email I get ....

De: root@firewall.videotron.ca
Objet: l'adresse IP a changee !
Date: 17 juin 2016 13:24:07 UTC−4
À: <blablabla>@videotron.ca

Date: Fri, 17 Jun 2016 13:26:00 -0400
nouvel IP: 11.22.33.44
de la part de ton gentil Routeur !!!

 

[RMerlin]

which brings me to ask you how you knew about this new requirement or changes that were progressively deploying over the past couple of months... if you feel like telling me of course !

I work in IT, it started affecting some of my customers over these past few months. Some of them had their POP3 work address configured with Vidéotron's SMTP for outgoing mail.

 

[RMerlin]

Just saw that Entware does have a postfix package that support musl... should be able to run musl and uclibc together perhaps...

https://github.com/Entware/openwrt-packages/tree/master/mail/postfix

Postfix wouldn't really help, as he'd need to configure a Smart Host, which would bring back his original issue. Outbound port 25 is blocked for any SMTP other than the ISP's own.

 

[sfx2000]

Postfix wouldn't really help, as he'd need to configure a Smart Host, which would bring back his original issue. Outbound port 25 is blocked for any SMTP other than the ISP's own.

So Port 25 is blocked, Port 587 probably isn't, otherwise Gmail wouldn't work on SmartPhones

 

[RMerlin]

So Port 25 is blocked, Port 587 probably isn't, otherwise Gmail wouldn't work on SmartPhones

Which would STILL require a smart host... Postfix does nothing to resolve his root problem.

 

[sfx2000]

Which would STILL require a smart host

Perhaps I'm missing a point here - how is setting up a local smarthost a problem?

 

[sfx2000]

That's the power of a smarthost on the local machine - just bounce the output from scripts (bash, php, perl, python, etc...) to localhost sendmail, and postfix configured as a smarthost MTA sends it to the gmail server, which is authenticated via SASL, so they accept it.

My ISP also blocks port 25 outbound to other servers, one must use theirs, that was the whole impetus behind setting up a local smarthost in the first place.. once it gets to gmail, one can set up filter rules to do whatever... (example would be to pipe the output to an SMTP to SMPP gateway to send me a text message)

 

[RMerlin]

That's the power of a smarthost on the local machine - just bounce the output from scripts (bash, php, perl, python, etc...) to localhost sendmail, and postfix configured as a smarthost MTA sends it to the gmail server, which is authenticated via SASL, so they accept it.

His problem is strictly with his router's outbound notifications. He doesn't need to run a full-fledged SMTP server just for that.

 

[Jean-Roch Blais]

I work in IT, it started affecting some of my customers over these past few months. Some of them had their POP3 work address configured with Vidéotron's SMTP for outgoing mail.

Ok I'll admit that, but why did Videotron suddenly made those changes, would you know what their motives, reasons are, or guess what this is all about ? It also took you by surprise I notice. Shouldn't they at least advise their customers, of those changes, unless they don't know what they're doing .... which is what I suspect according to the email they sent me. Anyway I'm not sure this discussion is helping the forum, but knowledge is power !

 

[RMerlin]

Ok I'll admit that, but why did Videotron suddenly made those changes, would you know what their motives, reasons are, or guess what this is all about ?

On more than one occasion, Videotron got their SMTP blacklisted due to spam relaying. Preventing sending mail with a reply address outside of the videotron.ca clamps down on that spam, as spammers would typically use an external domain.

And in general, I've never been impressed by Videotron's sysadmins. Quite a few dubious decisions on their part over the years. I remember once having to deal with the fact that their caching nameservers flat out ignored TTL values. Migrating a customer's server to a different IP was a PITA, as Videotron's DNS kept resolving to the old IP for well over a day, despite the domain TTL being only a few hours. Hopefully they've fixed that since (it was 12-14 years ago).

Also the fact that their SMTP will give up on temporary delivery failure after only one day, while the majority of servers out there will try for at least 5 days. That means a weekend outage for someone's mail server that would only be resolved on Monday morning will result in lost mail.

The fact that you need to use a different mail server to use TLS in 2016 is also a bit mind boggling.

I suppose it's part for the course when it comes to ISPs. Many of them are managed by sysadmins who ignore best practices or RFCs, and do just what they feel like is "best", without thinking any further than their personal, corporate bottom line. </mini rant>

 

[sfx2000]

On more than one occasion, Videotron got their SMTP blacklisted due to spam relaying. Preventing sending mail with a reply address outside of the videotron.ca clamps down on that spam, as spammers would typically use an external domain.

BEEN THERE - and it's not a nice place to be, when one has 16 SMTP gateways, and they all got RBL'ed by SpamHaus, and that relayed to every other real-time black list - took a lot of work to get off those, and SpamHaus is not sympathetic in the least little bit... having 20M customers not being able to send email with any level of confidence is a real problem - so I get why Videotron started clamping down on things...

It was something I inherited, not designed - so the problem was already there, just waiting to happen - ended up redoing the entire platform to be more consistent with best practices and had to do it on the fly - what a mess, and two very long weeks, working almost constantly (sleeping under the conference table more than a couple of times...)

 

[Jean-Roch Blais]

On more than one occasion, Videotron got their SMTP blacklisted due to spam relaying. Preventing sending mail with a reply address outside of the videotron.ca clamps down on that spam, as spammers would typically use an external domain.

And in general, I've never been impressed by Videotron's sysadmins. Quite a few dubious decisions on their part over the years. I remember once having to deal with the fact that their caching nameservers flat out ignored TTL values. Migrating a customer's server to a different IP was a PITA, as Videotron's DNS kept resolving to the old IP for well over a day, despite the domain TTL being only a few hours. Hopefully they've fixed that since (it was 12-14 years ago).

Also the fact that their SMTP will give up on temporary delivery failure after only one day, while the majority of servers out there will try for at least 5 days. That means a weekend outage for someone's mail server that would only be resolved on Monday morning will result in lost mail.

The fact that you need to use a different mail server to use TLS in 2016 is also a bit mind boggling.

I suppose it's part for the course when it comes to ISPs. Many of them are managed by sysadmins who ignore best practices or RFCs, and do just what they feel like is "best", without thinking any further than their personal, corporate bottom line. </mini rant>

<minipraise> I finally received an email from Videotron admitting they were blacklisted, and telling me they now use smtp.videotron.ca using ssl on port 465 and with auth. </minipraise>

Inspired by your smtp.gmail.com example I tried this:

********************************************************

firewall# sendmail -v -H"exec openssl s_client -quiet -CAfile Certificats.pem -tls1 -connect smtp.videotron.ca:465 -p
ause" </tmp/mail.txt -froot@firewall -auxxxxxxx -apyyyyyyy
blablabla@gmail.com
sendmail: send:'NOOP'
depth=2 C = US, O = "VeriSign, Inc.", OU = VeriSign Trust Network, OU = "(c) 2006 VeriSign, Inc. - For authorized use only", CN = VeriSign Class 3 Public Primary Certification Authority - G5
verify return:1
depth=1 C = US, O = Symantec Corporation, OU = Symantec Trust Network, CN = Symantec Class 3 Secure Server CA - G4
verify return:1
depth=0 C = CA, ST = Quebec, L = Montreal, O = Videotron s.e.n.c., OU = Ingenierie, CN = smtp.videotron.ca
verify return:1
read:errno=0
sendmail: NOOP failed

*********************************************************

but once in about every 5 tries with the same command, it works and I get the email to gmail...

*****************************

firewall# sendmail -v -H"exec openssl s_client -quiet -CAfile Certificats.pem -t
ls1 -connect smtp.videotron.ca:465 -pause" </tmp/mail.txt -froot@firewall -auxxxxxxxxx -apyyyyyyyy blablabla@gmail.com
sendmail: send:'NOOP'
depth=2 C = US, O = "VeriSign, Inc.", OU = VeriSign Trust Network, OU = "(c) 2006 VeriSign, Inc. - For authorized use only", CN = VeriSign Class 3 Public Primary Certification Authority - G5
verify return:1
depth=1 C = US, O = Symantec Corporation, OU = Symantec Trust Network, CN = Symantec Class 3 Secure Server CA - G4
verify return:1
depth=0 C = CA, ST = Quebec, L = Montreal, O = Videotron s.e.n.c., OU = Ingenierie, CN = smtp.videotron.ca
verify return:1
sendmail: recv:'220 smtp.videotron.ca Videotron ESMTP server ready'
sendmail: recv:'250 2.0.0 OK'
sendmail: send:'EHLO firewall'
sendmail: recv:'250-smtp.videotron.ca hello [11.22.33.44], pleased to meet you'
sendmail: recv:'250-HELP'
sendmail: recv:'250-AUTH LOGIN PLAIN'
sendmail: recv:'250-SIZE 35840000'
sendmail: recv:'250-ENHANCEDSTATUSCODES'
sendmail: recv:'250-8BITMIME'
sendmail: recv:'250 OK'
sendmail: send:'AUTH LOGIN'
sendmail: recv:'334 VXNlcm5hbWU6'
sendmail: send:''
sendmail: recv:'334 UGFzc3dvcmQ6'
sendmail: send:''
sendmail: recv:'235 2.7.0 ... authentication succeeded'
sendmail: send:'MAIL FROM:<root@firewall>'
sendmail: recv:'250 2.1.0 <root@firewall> sender ok'
sendmail: send:'RCPT TO:<blablabla@gmail.com>'
sendmail: recv:'250 2.1.5 EMZ1booskWjC4EMZ4bADps <blablabla@gmail.com> recipient ok'
sendmail: send:'DATA'
sendmail: recv:'354 OK'
sendmail: send:'Subject: WAN state notification'
sendmail: send:'From: Firewall blablabla@videotron.ca'
sendmail: send:'Date: Sat, 18 Jun 2016 15:25:33 -0400'
sendmail: send:'To: blablabla@gmail.com'
sendmail: send:''
sendmail: send:'I just got connected to the internet.'
sendmail: send:''
sendmail: send:'My WAN IP is: '
sendmail: send:''
sendmail: send:'---- '
sendmail: send:'Your friendly router.'
sendmail: send:''
sendmail: send:'.'
sendmail: recv:'250 2.0.0 EMZ1booskWjC4EMZ4bADps Message accepted for delivery'
sendmail: send:'QUIT'
sendmail: recv:'221 2.0.0 smtp.videotron.ca Videotron closing connection'

***************************

It partly works, there are these two lines:
...
read:errno=0
sendmail: NOOP failed

at the end of the failed dialog... Openssh gives this read:errno=0 which might cause sendmail to become confused... I'm at a loss here. This is why I used the </minipraise> earlier !
My Certificats.pem were exported from my mac, it is a large file about 195 certificats.... I guess it shows I'm new to this ...

 

[Jean-Roch Blais]

<minipraise> I finally received an email from Videotron admitting they were blacklisted, and telling me they now use smtp.videotron.ca using ssl on port 465 and with auth. </minipraise>

Inspired by your smtp.gmail.com example I tried this:

********************************************************
firewall# sendmail -v -H"exec openssl s_client -quiet -CAfile Certificats.pem -tls1 -connect smtp.videotron.ca:465 -p
ause" </tmp/mail.txt -froot@firewall -auxxxxxxx -apyyyyyyy
blablabla@gmail.com
sendmail: send:'NOOP'
depth=2 C = US, O = "VeriSign, Inc.", OU = VeriSign Trust Network, OU = "(c) 2006 VeriSign, Inc. - For authorized use only", CN = VeriSign Class 3 Public Primary Certification Authority - G5
verify return:1
depth=1 C = US, O = Symantec Corporation, OU = Symantec Trust Network, CN = Symantec Class 3 Secure Server CA - G4
verify return:1
depth=0 C = CA, ST = Quebec, L = Montreal, O = Videotron s.e.n.c., OU = Ingenierie, CN = smtp.videotron.ca
verify return:1
read:errno=0
sendmail: NOOP failed
*********************************************************

but once in about every 5 tries with the same command, it works and I get the email to gmail...

*****************************
firewall# sendmail -v -H"exec openssl s_client -quiet -CAfile Certificats.pem -t
ls1 -connect smtp.videotron.ca:465 -pause" </tmp/mail.txt -froot@firewall -auxxxxxxxxx -apyyyyyyyy blablabla@gmail.com
sendmail: send:'NOOP'
depth=2 C = US, O = "VeriSign, Inc.", OU = VeriSign Trust Network, OU = "(c) 2006 VeriSign, Inc. - For authorized use only", CN = VeriSign Class 3 Public Primary Certification Authority - G5
verify return:1
depth=1 C = US, O = Symantec Corporation, OU = Symantec Trust Network, CN = Symantec Class 3 Secure Server CA - G4
verify return:1
depth=0 C = CA, ST = Quebec, L = Montreal, O = Videotron s.e.n.c., OU = Ingenierie, CN = smtp.videotron.ca
verify return:1
sendmail: recv:'220 smtp.videotron.ca Videotron ESMTP server ready'
sendmail: recv:'250 2.0.0 OK'
sendmail: send:'EHLO firewall'
sendmail: recv:'250-smtp.videotron.ca hello [11.22.33.44], pleased to meet you'
sendmail: recv:'250-HELP'
sendmail: recv:'250-AUTH LOGIN PLAIN'
sendmail: recv:'250-SIZE 35840000'
sendmail: recv:'250-ENHANCEDSTATUSCODES'
sendmail: recv:'250-8BITMIME'
sendmail: recv:'250 OK'
sendmail: send:'AUTH LOGIN'
sendmail: recv:'334 VXNlcm5hbWU6'
sendmail: send:''
sendmail: recv:'334 UGFzc3dvcmQ6'
sendmail: send:''
sendmail: recv:'235 2.7.0 ... authentication succeeded'
sendmail: send:'MAIL FROM:<root@firewall>'
sendmail: recv:'250 2.1.0 <root@firewall> sender ok'
sendmail: send:'RCPT TO:<blablabla@gmail.com>'
sendmail: recv:'250 2.1.5 EMZ1booskWjC4EMZ4bADps <blablabla@gmail.com> recipient ok'
sendmail: send:'DATA'
sendmail: recv:'354 OK'
sendmail: send:'Subject: WAN state notification'
sendmail: send:'From: Firewall blablabla@videotron.ca'
sendmail: send:'Date: Sat, 18 Jun 2016 15:25:33 -0400'
sendmail: send:'To: blablabla@gmail.com'
sendmail: send:''
sendmail: send:'I just got connected to the internet.'
sendmail: send:''
sendmail: send:'My WAN IP is: '
sendmail: send:''
sendmail: send:'---- '
sendmail: send:'Your friendly router.'
sendmail: send:''
sendmail: send:'.'
sendmail: recv:'250 2.0.0 EMZ1booskWjC4EMZ4bADps Message accepted for delivery'
sendmail: send:'QUIT'
sendmail: recv:'221 2.0.0 smtp.videotron.ca Videotron closing connection'
***************************

It partly works, there are these two lines:
...
read:errno=0
sendmail: NOOP failed

at the end of the failed dialog... Openssh gives this read:errno=0 which might cause sendmail to become confused... I'm at a loss here. This is why I used the </minipraise> earlier !
My Certificats.pem were exported from my mac, it is a large file about 195 certificats.... I guess it shows I'm new to this ...

Looking at busybox sendmail source code I found this:

...
// connection helper ordered? ->
   if (opts & OPT_H) {
     const char *args[] = { "sh", "-c", opt_connect, NULL };
     // plug it in
     launch_helper(args);
     // Now:
     // our stdout will go to helper's stdin,
     // helper's stdout will be available on our stdin.

     // Wait for initial server message.
     // If helper (such as openssl) invokes STARTTLS, the initial 220
     // is swallowed by helper (and not repeated after TLS is initiated).
     // We will send NOOP cmd to server and check the response.
     // We should get 220+250 on plain connection, 250 on STARTTLSed session.
     //
     // The problem here is some servers delay initial 220 message,
     // and consider client to be a spammer if it starts sending cmds
     // before 220 reached it. The code below is unsafe in this regard:
     // in non-STARTTLSed case, we potentially send NOOP before 220
     // is sent by server.
     // Ideas? (--delay SECS opt? --assume-starttls-helper opt?)
     code = smtp_check("NOOP", -1);
     if (code == 220)
       // we got 220 - this is not STARTTLSed connection,
       // eat 250 response to our NOOP
       smtp_check(NULL, 250);
     else
     if (code != 250)

What can we do about this, ask the busybox maintainer to fix it ?
:-(