Page 2 of 2
Re: python code suddenly not working
Posted: Sunday 22 January 2023 22:19
by lambertmular
Did you raise it as an issue on Github yet?
As advised, I raised my problem as an issue on github. Nevertheless the issue was closed 50 min after being opened, with the only response
Please use the forum
Is he referring to this forum or does it exists a forum on github? (sorry for the stupid questions)
here is the link to the issue raised on github
https://github.com/domoticz/domoticz/issues/5559
I must confess that I am a bit lost. Did I do something wrong when raising the issue? I not sure that I followed the right steps since I am new to github. What should I do next? insist on github?
Re: python code suddenly not working
Posted: Monday 23 January 2023 8:26
by willemd
He is often not very open to new issues ......I think you should let him know it has been raised on the forum and no solution was found, while the problems seems to be within domoticz itself, therefore the next step was github. Let him know you did a full rebuild as well.
Re: python code suddenly not working
Posted: Monday 23 January 2023 18:59
by waltervl
Seems to be an issue when used in the def and return as in the first statement re is used without errors.
Code: Select all
es=list(map(float,re.findall("(\d+\.?\d+);(\d+);(\d+)",DE.Devices['Temp Chambre'].s_value)[0]))
What happens if you change this
Code: Select all
def data_meteo(name):
return list(map(float,re.findall("(\d+\.?\d+);(\d+);(\d+)",DE.Devices[name].s_value)[0]))
Into this
Code: Select all
def data_meteo(name):
testvalue = list(map(float,re.findall("(\d+\.?\d+);(\d+);(\d+)",DE.Devices[name].s_value)[0]))
return testvalue
I am not a Python programmer so this could be nonsense....
Perhaps remove the def from the code completely and see if it works. Not optimal but if it works it could be a workaround. Then it could be a issue in python event scripts (rewritten in 2022 due to incompatibility for newer python versions)
Edit: what I also remembered from another issue was to import the module in the def again to prevent errors.
So
Code: Select all
def data_meteo(name):
import re
return list(map(float,re.findall("(\d+\.?\d+);(\d+);(\d+)",DE.Devices[name].s_value)[0]))
Re: python code suddenly not working
Posted: Tuesday 24 January 2023 14:56
by lambertmular
Thank you for your help waltervl! below I answer your questions and report another bug that also seems related to "def
What happens if you change this
Code: Select all
def data_meteo(name):
return list(map(float,re.findall("(\d+\.?\d+);(\d+);(\d+)",DE.Devices[name].s_value)[0]))
Into this
Code: Select all
def data_meteo(name):
testvalue = list(map(float,re.findall("(\d+\.?\d+);(\d+);(\d+)",DE.Devices[name].s_value)[0]))
return testvalue
It does not change anything and the output is still
Code: Select all
2023-01-24 13:27:23.932 Python:[16.2, 64.0, 3.0]
2023-01-24 13:27:23.933 Error: Traceback (most recent call last):
2023-01-24 13:27:23.933 Error: File "Script #1", line 11, in <module>
2023-01-24 13:27:23.933 Error: File "Script #1", line 8, in data_meteo
2023-01-24 13:27:23.933 Error: NameError: name 're' is not defined
what I also remembered from another issue was to import the module in the def again to prevent errors.
So
Code: Select all
def data_meteo(name):
import re
return list(map(float,re.findall("(\d+\.?\d+);(\d+);(\d+)",DE.Devices[name].s_value)[0]))
The error is still present but interestingly it change the type of error (name 'DE' is not defined)
Code: Select all
2023-01-24 13:40:17.540 Python:[15.9, 65.0, 3.0]
2023-01-24 13:40:17.541 Error: Traceback (most recent call last):
2023-01-24 13:40:17.541 Error: File "Script #1", line 11, in <module>
2023-01-24 13:40:17.541 Error: File "Script #1", line 9, in data_meteo
2023-01-24 13:40:17.541 Error: NameError: name 'DE' is not defined
Perhaps remove the def from the code completely and see if it works. Not optimal but if it works it could be a workaround. Then it could be a issue in python event scripts (rewritten in 2022 due to incompatibility for newer python versions)
In this code it is already working without the def. I could remove all def however as you mention it is not optimal at all: my actual code is much more complex with more than 20 def. This simple example was chosen only for illustration.
Finally this bug with "def " and "re" is not the only one I have in my code (that was perfectly working for years): I can report at least another bug which is not related to "re". This is the code
Code: Select all
import DomoticzEvents as DE
import datetime,time
time1=datetime.datetime.now() # heure actuelle
date_retour=datetime.datetime(2023,1,2,15)
DE.Log("Python:"+str(date_retour))
def thermostat_chauffage_chambre():
duree=date_retour-time1
seuil_temperature_chambre=12
return seuil_temperature_chambre
thermostat_chauffage_chambre()
and the output shows that even if "date_retour" is known it gives an error when called inside the def
Code: Select all
2023-01-24 13:45:34.404 Python:2023-01-02 15:00:00
2023-01-24 13:45:34.406 Error: Traceback (most recent call last):
2023-01-24 13:45:34.406 Error: File "Script #2", line 13, in <module>
2023-01-24 13:45:34.406 Error: File "Script #2", line 9, in thermostat_chauffage_chambre
2023-01-24 13:45:34.406 Error: NameError: name 'date_retour' is not defined
Nothing seems to work inside a def, this is crazy
Re: python code suddenly not working
Posted: Tuesday 24 January 2023 15:39
by willemd
Hmm.... strange. I tested your script again, the one at the beginning of this thread.
In python from commandline: no problem.
In python from domoticz: same error as yours.
Then I played around a bit with the code and just added "global re" within the def definition (even though it does not make sense...)
The error disappeared and after that the script ran normally, even if I removed the "global re" line again.
Worth a try.
Re: python code suddenly not working
Posted: Tuesday 24 January 2023 15:45
by willemd
And doing the same in your latest script also solves the problem.
So my conclusion is than when python is run from domoticz, there is something wrong with the global scope of the variables. The def's used within domoticz don't import the globally defined variables unless specifically defined.
the below script just run's fine when the global defnition is included and continues to run when it is removed again, but the initial install fails because the global scope is not recognised properly.
I will add my finding to the github issue report.
Code: Select all
import DomoticzEvents as DE
import datetime,time
time1=datetime.datetime.now() # heure actuelle
date_retour=datetime.datetime(2023,1,2,15)
DE.Log("Python:"+str(date_retour))
def thermostat_chauffage_chambre():
global date_retour,time1
duree=date_retour-time1
seuil_temperature_chambre=12
return seuil_temperature_chambre
DE.Log("from def" + str(thermostat_chauffage_chambre()))
Re: python code suddenly not working
Posted: Tuesday 24 January 2023 16:49
by lambertmular
Many many thanks willemd.
As I told in a previous post, my actual code is very long and complex, however after treating almost all variables with your recipe: using "global" each time they throw me an error, I managed to make my code work again!
Great great job, I can't thank you enough!
I must say that I do not understand the solution nor the problem. Do you think it will remain stable or that I will need to use the "global" trick from time to time?
Re: python code suddenly not working
Posted: Tuesday 24 January 2023 17:22
by willemd
lambertmular wrote: ↑Tuesday 24 January 2023 16:49
I must say that I do not understand the solution nor the problem. Do you think it will remain stable or that I will need to use the "global" trick from time to time?
Let's wait and see what the experts say about a solution. The "global" statement should not be necessary, it should be a temporary workaround.
Re: python code suddenly not working
Posted: Tuesday 24 January 2023 17:37
by lambertmular
I must say it is not completely working.
For example without the global statement it seems that some variable inside "def" are not updated as you can see in this code
Code: Select all
import DomoticzEvents as DE
import datetime,time
time1=datetime.datetime.now() # heure actuelle
DE.Log("Python script1: time1"+str(time1))
def fun():
DE.Log("Python script1: time1"+str(time1))
fun()
whose output is
Code: Select all
2023-01-24 16:34:04.795 Python script1: time12023-01-24 16:34:04.795070
2023-01-24 16:34:04.795 Python script1: time12023-01-24 16:28:17.774032
Clearly "time1" is not updated inside the "def.
Instead if I add a "global:
Code: Select all
import DomoticzEvents as DE
import datetime,time
time1=datetime.datetime.now() # heure actuelle
DE.Log("Python script1: time1"+str(time1))
def fun():
global time1
DE.Log("Python script1: time1"+str(time1))
fun()
Then it works normally
Code: Select all
2023-01-24 16:35:13.695 Python script1: time12023-01-24 16:35:13.695776
2023-01-24 16:35:13.696 Python script1: time12023-01-24 16:35:13.695776
It seems that I cannot withdraw all the "global" commands
Re: python code suddenly not working
Posted: Tuesday 24 January 2023 19:45
by willemd
That again proofs that something is wrong with the import of global definitions into the def scope.
Normally globally defined variables should also be known within the def scope and also their values should be taken "from the outside". So if you print time1 inside the def scope it should show the same value as outside (even without the global line).
You would only use a "global" definition within the def scope if you want to update the variable within the def and make the variable value available globally to the total program. If you don't use the "global" line within the def definition, then any updates to the variable within the def are lost when processing returns to the main program.
So there is also a risk if you leave the global statement in place as a workaround. It will export the value of those variables to the main scope, even if you don't want to. Whether this is a problem depends on your program.
Re: python code suddenly not working
Posted: Thursday 26 January 2023 15:56
by rezzalex
Hello everyone,
I would like to know if my issue is related to the recent bug described in this post.
I am using the official Docker DZ image, currently on almost the latest beta :
Version: 2022.2 (build 14975)
Build Hash: 932461e0c
Compile Date: 2023-01-18 17:23:35
dzVents Version: 3.1.8
Python Version: 3.7.3 (default, Oct 31 2022, 14:04:00) [GCC 8.3.0]
I try to use a Python script from DZ script editor, and try to past the following command at the beginning :
and this one later-on in the same script :
Code: Select all
requestsession = requests.Session()
I have installed "requests" in here :
and here :
Code: Select all
/usr/local/lib/python3.7/dist-packages
but the script keep falling saying
Code: Select all
NameError: name 'requests' is not defined
isn't it weird to be on this Python Version ? Should I rebuild the container completely?
Re: python code suddenly not working
Posted: Thursday 26 January 2023 17:23
by willemd
rezzalex wrote: ↑Thursday 26 January 2023 15:56
Hello everyone,
I would like to know if my issue is related to the recent bug described in this post.
Could be the same issue.
Check:
1) is your script running from commandline without problems?
2) Is the requestsession=requests.Session() line within a function definition?
3) If you add the statement "global requests" inside the definition, does this remove the error message?
If so, it is the same issue.
Re: python code suddenly not working
Posted: Thursday 26 January 2023 23:17
by waltervl
In docker be sure you have the extra python module(s) installed in the Domoticz container else it cannot find it. You can use the customstart.sh for that. See the Domoticz docker wiki.
Re: python code suddenly not working
Posted: Friday 27 January 2023 8:22
by rezzalex
waltervl wrote: ↑Thursday 26 January 2023 23:17
In docker be sure you have the extra python module(s) installed in the Domoticz container else it cannot find it. You can use the customstart.sh for that. See the Domoticz docker wiki.
as already written, I have installed the "requests" package in /usr/lib/python3.7/ and in "/usr/local/lib/python3.7/dist-packages"
Re: python code suddenly not working
Posted: Friday 27 January 2023 13:59
by rezzalex
willemd wrote: ↑Thursday 26 January 2023 17:23
rezzalex wrote: ↑Thursday 26 January 2023 15:56
Hello everyone,
I would like to know if my issue is related to the recent bug described in this post.
Could be the same issue.
Check:
1) is your script running from commandline without problems?
2) Is the requestsession=requests.Session() line within a function definition?
3) If you add the statement "global requests" inside the definition, does this remove the error message?
If so, it is the same issue.
1) yes; it nows run smoothly from commandline
2) Yes, it is a function defined in the request Python package :
https://requests.readthedocs.io/en/late ... /advanced/
3) can not edit a python package "Python out of the box" function
My issue seems rather the one described here :
https://github.com/domoticz/domoticz/issues/5339
Re: python code suddenly not working
Posted: Friday 27 January 2023 15:06
by waltervl
rezzalex wrote: ↑Friday 27 January 2023 8:22
waltervl wrote: ↑Thursday 26 January 2023 23:17
In docker be sure you have the extra python module(s) installed in the Domoticz container else it cannot find it. You can use the customstart.sh for that. See the Domoticz docker wiki.
as already written, I have installed the "requests" package in /usr/lib/python3.7/ and in "/usr/local/lib/python3.7/dist-packages"
A docker container is a seperate system. So installing request on your host system will not enable request on your docker environment. Please investigate the Docker environment in general and the Domoticz docker in detail. See also the wiki page:
https://www.domoticz.com/wiki/Docker
Edit: to get into the docker container bash use:
From there install module request and try again.
Re: python code suddenly not working
Posted: Friday 27 January 2023 15:15
by rezzalex
waltervl wrote: ↑Friday 27 January 2023 15:06
rezzalex wrote: ↑Friday 27 January 2023 8:22
waltervl wrote: ↑Thursday 26 January 2023 23:17
In docker be sure you have the extra python module(s) installed in the Domoticz container else it cannot find it. You can use the customstart.sh for that. See the Domoticz docker wiki.
as already written, I have installed the "requests" package in /usr/lib/python3.7/ and in "/usr/local/lib/python3.7/dist-packages"
A docker container is a seperate system. So installing request on your host system will not enable request on your docker environment. Please investigate the Docker environment in general and the Domoticz docker in detail. See also the wiki page:
https://www.domoticz.com/wiki/Docker
I was previously talking about the DZ container, not my Host.
Re: python code suddenly not working
Posted: Friday 27 January 2023 15:26
by waltervl
rezzalex wrote: ↑Friday 27 January 2023 15:15
I was previously talking about the DZ container, not my Host.
Ok, that was not clear but very important to know.