Email problems

Please use template to report bugs and problems. Post here your questions when not sure where else to post
Only for bugs in the Domoticz application! other problems go in different subforums!

Moderators: leecollings, remb0

Forum rules
Before posting here, make sure you are on the latest Beta or Stable version.
If you have problems related to the web gui, clear your browser cache + appcache first.

Use the following template when posting here:

Version: xxxx
Platform: xxxx
Plugin/Hardware: xxxx
Description:
.....

If you are having problems with scripts/blockly, always post the script (in a spoiler or code tag) or screenshots of your blockly

If you are replying, please do not quote images/code from the first post

Please mark your topic as Solved when the problem is solved.
willemd
Posts: 621
Joined: Saturday 21 September 2019 17:55
Target OS: Raspberry Pi / ODroid
Domoticz version: 2024.1
Location: The Netherlands
Contact:

Re: Email problems

Post by willemd »

in short: the Ziggo SMTP servers don't accept SMTPUTF8 anymore, only 8BITMIME. This is also visible in the debug info received from the server.

I now might have an entry to the systems people at Ziggo. I left them all the info and hope I will receive some feedback.
glsf91
Posts: 58
Joined: Tuesday 14 November 2017 21:56
Target OS: Linux
Domoticz version:
Contact:

Re: Email problems

Post by glsf91 »

willemd wrote: Wednesday 17 January 2024 19:34 in short: the Ziggo SMTP servers don't accept SMTPUTF8 anymore, only 8BITMIME. This is also visible in the debug info received from the server.
Why do you think it is sending with SMTPUTF8 ?

I have my doubts (sorry).
I have a postfix mailserver installed relaying to smtp.ziggo.nl. When sending from domoticz to this mailserver it is working again through ziggo.
And when I disable SMTPUTF8 (to domoticz) on this mailserver it is still working !
willemd
Posts: 621
Joined: Saturday 21 September 2019 17:55
Target OS: Raspberry Pi / ODroid
Domoticz version: 2024.1
Location: The Netherlands
Contact:

Re: Email problems

Post by willemd »

When I tried sending via ziggo using mail option smtputf8 I received an error message that the server does not support it (unlike gmail)


When I sent via ziggo I have to use 8bitmime currently.
glsf91
Posts: 58
Joined: Tuesday 14 November 2017 21:56
Target OS: Linux
Domoticz version:
Contact:

Re: Email problems

Post by glsf91 »

willemd wrote: Wednesday 17 January 2024 22:27 When I tried sending via ziggo using mail option smtputf8 I received an error message that the server does not support it (unlike gmail)
When I sent via ziggo I have to use 8bitmime currently.
Ok, but why do you think domoticz/libcurl is using smtputf8 ?
willemd
Posts: 621
Joined: Saturday 21 September 2019 17:55
Target OS: Raspberry Pi / ODroid
Domoticz version: 2024.1
Location: The Netherlands
Contact:

Re: Email problems

Post by willemd »

glsf91 wrote: Thursday 18 January 2024 7:23
willemd wrote: Wednesday 17 January 2024 22:27 When I tried sending via ziggo using mail option smtputf8 I received an error message that the server does not support it (unlike gmail)
When I sent via ziggo I have to use 8bitmime currently.
Ok, but why do you think domoticz/libcurl is using smtputf8 ?
I had the impression that if the message is sent as plain text (line 318 in SMTPclinet.cpp) that as a consequence it is sent using SMPTUTF8.

I can see gmail is supporting that but ziggo is not.

Anyway, the core of the problem still seems to be that Ziggo is rejecting a from-adress although it is correct, and then is expecting to retrieve it from the mime message instead.
glsf91
Posts: 58
Joined: Tuesday 14 November 2017 21:56
Target OS: Linux
Domoticz version:
Contact:

Re: Email problems

Post by glsf91 »

willemd wrote: Thursday 18 January 2024 9:18 I had the impression that if the message is sent as plain text (line 318 in SMTPclinet.cpp) that as a consequence it is sent using SMPTUTF8.
I don't think so. The test message for example has this format:

Code: Select all

From: <email address>
To: <email address>
Subject: Domoticz test
Date: Wed, 17 Jan 2024 18:14:12 +0100
MIME-Version: 1.0
Content-Type: multipart/alternative;
        boundary="--------------601143517647029104953591"
This is a multipart message in MIME format.
----------------601143517647029104953591
Content-type: text/html; charset=utf-8
Content-Transfer-Encoding: 8bit

<html>
<body>
<b>Domoticz test message!</body>
</html>

----------------601143517647029104953591--
With curl there is no problem sending this.
I also plan to try to compile domoitcz by myself so I can add debug info
willemd
Posts: 621
Joined: Saturday 21 September 2019 17:55
Target OS: Raspberry Pi / ODroid
Domoticz version: 2024.1
Location: The Netherlands
Contact:

Re: Email problems

Post by willemd »

Now I am lost ... since all my testing showed the mime messages were sent correctly.
So why did this domoticz test message fail?
Or was it only the domoticz.notify() that failed, which is without a mail body?
glsf91
Posts: 58
Joined: Tuesday 14 November 2017 21:56
Target OS: Linux
Domoticz version:
Contact:

Re: Email problems

Post by glsf91 »

I found the cause of the problem. The ziggo mailser is responding on a test messages from domoticz with:

Code: Select all

Message contains bare LF and is violating 822.bis section 2.3
It turns out that the HTMLBody contains only LF instead of CRLF. That is this part:

Code: Select all

<html>
<body>
<b>Domoticz test message!</body>
</html>
When I replace the LF with CRLF the messages is accepted.
There is also a PlainBody possible. I don't know how to trigger this and if this have the same problem of only LF.

Again: I'm testing this on version 2023.1.
willemd
Posts: 621
Joined: Saturday 21 September 2019 17:55
Target OS: Raspberry Pi / ODroid
Domoticz version: 2024.1
Location: The Netherlands
Contact:

Re: Email problems

Post by willemd »

Below is my python test code.
It tries to send three mails of which the first two are accepted and the third one is not.
In the past, the third one was sent without problems.
Is that explained by your finding? What to change to get the last one working? I can only get it to work by adding the "From:[email protected]" to the message
The error feedback is that the From address is missing or malformed, although the debug also shows that the From address and the To address were accepted.

Maybe they have implemented a more strict protocol compliance check at Ziggo (which explains your finding of \r\n need) and in the process got something else wrong (which explains my finding that the processing of the From address fails)

Code: Select all

import smtplib
from email.mime.multipart import MIMEMultipart
emailData = MIMEMultipart()
emailData['Subject'] = "test mime"
emailData['To'] = "[email protected]"
emailData['From'] = "[email protected]"
smtpserver=smtplib.SMTP('smtp.ziggo.nl',587)
smtpserver.set_debuglevel(1)
smtpserver.starttls()
print("--------------\n",emailData.as_string(),"---------------\n")
smtpserver.login("[email protected]","passwordsender")
smtpserver.sendmail("[email protected]","[email protected]", emailData.as_string())   # this mail is sent without problems
smtpserver.sendmail("[email protected]","[email protected]", "From: [email protected]\r\nTo: [email protected]\r\nSubject: test string with From\r\n") #this mail is sent without problems
smtpserver.sendmail("[email protected]","[email protected]", "test string without From\r\n") # this mail fails with message that From address is missing or malformed
smtpserver.quit()
glsf91
Posts: 58
Joined: Tuesday 14 November 2017 21:56
Target OS: Linux
Domoticz version:
Contact:

Re: Email problems

Post by glsf91 »

willemd wrote: Thursday 18 January 2024 20:56 The error feedback is that the From address is missing or malformed, although the debug also shows that the From address and the To address were accepted.
Maybe there is more changed at ziggo which is causing your python problem now. But this has nothing to do with the problem in domoticz. Domoticz is always adding a From in the message so your third example is not applicable.

Domoticz is using libcurl for sending email notifications. Behaviour in the communication with the smtp server in python can be (and probably will) different. So using python to try to solve a problem in Domoticz makes not much sense in this case.

With all respect for your effort of course.
willemd
Posts: 621
Joined: Saturday 21 September 2019 17:55
Target OS: Raspberry Pi / ODroid
Domoticz version: 2024.1
Location: The Netherlands
Contact:

Re: Email problems

Post by willemd »

glsf91 wrote: Friday 19 January 2024 9:58
willemd wrote: Thursday 18 January 2024 20:56 The error feedback is that the From address is missing or malformed, although the debug also shows that the From address and the To address were accepted.
Maybe there is more changed at ziggo which is causing your python problem now. But this has nothing to do with the problem in domoticz. Domoticz is always adding a From in the message so your third example is not applicable.

Domoticz is using libcurl for sending email notifications. Behaviour in the communication with the smtp server in python can be (and probably will) different. So using python to try to solve a problem in Domoticz makes not much sense in this case.

With all respect for your effort of course.
I used python because Domoticz was not he only system that suddenly had issues.
As a final test I did the following:
Within the mime-message I used a different sender and recipient from the ones I used in the send command, so something like
sendmail(fromaddressnr1,toaddressnr1,"mime-message with fromaddressnr2 and toadressnr2")
Guess what happens?
The mail arrives at the mailbox of toaddressnr1, with in the header fromaddressnr2 and toadressnr2....

Anyway, probably time for you to create an issue on Github with your finding so it can be solved?
glsf91
Posts: 58
Joined: Tuesday 14 November 2017 21:56
Target OS: Linux
Domoticz version:
Contact:

Re: Email problems

Post by glsf91 »

mm10
Posts: 5
Joined: Monday 06 April 2015 18:50
Target OS: Raspberry Pi / ODroid
Domoticz version:
Contact:

Re: Email problems

Post by mm10 »

I too have this issue, but with 2023.2 on debian 11.1

WillemD opened https://github.com/domoticz/domoticz/issues/5989 on the domoticz GitHub but closed it again.
I do not agree that the issue does not need involvement from Domoticz.

While it might be true that other mailservers accept the domoticz mail still, this does not mean that ziggo is incorrectly rejecting them.

Until the real reason is found, this should remain an opportunity for Domoticz to become even better.

I can add some experiments if needed, but what is the current consensus about what happens here in all technical detail?

2Bcontinued...
mm10
Posts: 5
Joined: Monday 06 April 2015 18:50
Target OS: Raspberry Pi / ODroid
Domoticz version:
Contact:

Re: Email problems

Post by mm10 »

typical crossover of messages.
When I prepared my message, glsf91 just opened a GitHub issue.
Sorry for the fork.
glsf91
Posts: 58
Joined: Tuesday 14 November 2017 21:56
Target OS: Linux
Domoticz version:
Contact:

Re: Email problems

Post by glsf91 »

Solution is committed to the development branch of domoticz.
mm10
Posts: 5
Joined: Monday 06 April 2015 18:50
Target OS: Raspberry Pi / ODroid
Domoticz version:
Contact:

Re: Email problems

Post by mm10 »

Great work!

Any idea when this is included in a formal release?
BobKooi
Posts: 4
Joined: Sunday 15 May 2022 13:10
Target OS: Raspberry Pi / ODroid
Domoticz version:
Location: Netherlands
Contact:

Re: Email problems

Post by BobKooi »

Hello, I have a similar problem with Vimexx mail (mail.zxcs.nl).
2024-08-06 16:55:01.816 Error: libcurl: (67)
2024-08-06 16:55:01.816 Error: Login denied
I am running the latest Domoticz version: 2024.07 on the latest Raspberry Pi OS (based on Debian 12 ' Bookworm” ).
Should this problem be solved in this latest Domoticz formal release?
User avatar
waltervl
Posts: 5149
Joined: Monday 28 January 2019 18:48
Target OS: Linux
Domoticz version: 2024.7
Location: NL
Contact:

Re: Email problems

Post by waltervl »

BobKooi wrote: Tuesday 06 August 2024 17:15
Should this problem be solved in this latest Domoticz formal release?
Yes, it was already in 2024.3 or 2024.4
Domoticz running on Udoo X86 (on Ubuntu)
Devices/plugins: ZigbeeforDomoticz (with Xiaomi, Ikea, Tuya devices), Nefit Easy, Midea Airco, Omnik Solar, Goodwe Solar
BobKooi
Posts: 4
Joined: Sunday 15 May 2022 13:10
Target OS: Raspberry Pi / ODroid
Domoticz version:
Location: Netherlands
Contact:

Re: Email problems

Post by BobKooi »

Ok Waltervl, thanks for your response. I will continue to look for the cause in my configuration.

Found the problem, it was an authentication problem. The data that was entered automatically turned out to be incorrect. Problem solved!
User avatar
frank666
Posts: 17
Joined: Monday 07 December 2020 10:11
Target OS: Linux
Domoticz version: 2024.7
Location: San Marino

Re: Email problems

Post by frank666 »

  • email does not work if I use gmail or hotmail SMTP.
  • On the other hand, it works if I use other mail handlers.
  • So I deduce that the mail servers that do not accept domoticz have implemented some devilry to block traffic that does not come from mail clients they do not like.
  • Has anyone managed to solve or work around this problem or get more information about the new system adopted by the SMTP servers?
Domoticz running on Docker,Orange Pi Zero Plus
Post Reply

Who is online

Users browsing this forum: No registered users and 1 guest