python code suddenly not working

[lambertmular]

Did you raise it as an issue on Github yet?

As advised, I raised my problem as an issue on github. Nevertheless the issue was closed 50 min after being opened, with the only response

Please use the forum

Is he referring to this forum or does it exists a forum on github? (sorry for the stupid questions)
here is the link to the issue raised on github https://github.com/domoticz/domoticz/issues/5559
I must confess that I am a bit lost. Did I do something wrong when raising the issue? I not sure that I followed the right steps since I am new to github. What should I do next? insist on github?

[willemd]

He is often not very open to new issues ......I think you should let him know it has been raised on the forum and no solution was found, while the problems seems to be within domoticz itself, therefore the next step was github. Let him know you did a full rebuild as well.

[waltervl]

Seems to be an issue when used in the def and return as in the first statement re is used without errors.

Code: Select all

es=list(map(float,re.findall("(\d+\.?\d+);(\d+);(\d+)",DE.Devices['Temp Chambre'].s_value)[0]))

What happens if you change this

Code: Select all

def data_meteo(name):
    return list(map(float,re.findall("(\d+\.?\d+);(\d+);(\d+)",DE.Devices[name].s_value)[0]))
    

Into this

Code: Select all

def data_meteo(name):
    testvalue = list(map(float,re.findall("(\d+\.?\d+);(\d+);(\d+)",DE.Devices[name].s_value)[0]))
    return testvalue
    

I am not a Python programmer so this could be nonsense....
Perhaps remove the def from the code completely and see if it works. Not optimal but if it works it could be a workaround. Then it could be a issue in python event scripts (rewritten in 2022 due to incompatibility for newer python versions)

Edit: what I also remembered from another issue was to import the module in the def again to prevent errors.
So

Code: Select all

def data_meteo(name):
    import re
    return list(map(float,re.findall("(\d+\.?\d+);(\d+);(\d+)",DE.Devices[name].s_value)[0]))

[lambertmular]

Thank you for your help waltervl! below I answer your questions and report another bug that also seems related to "def

What happens if you change this

Code: Select all

def data_meteo(name):
    return list(map(float,re.findall("(\d+\.?\d+);(\d+);(\d+)",DE.Devices[name].s_value)[0]))

Into this

Code: Select all

def data_meteo(name):
    testvalue = list(map(float,re.findall("(\d+\.?\d+);(\d+);(\d+)",DE.Devices[name].s_value)[0]))
    return testvalue

It does not change anything and the output is still

Code: Select all

2023-01-24 13:27:23.932 Python:[16.2, 64.0, 3.0]
2023-01-24 13:27:23.933 Error: Traceback (most recent call last):
2023-01-24 13:27:23.933 Error: File "Script #1", line 11, in <module>
2023-01-24 13:27:23.933 Error: File "Script #1", line 8, in data_meteo
2023-01-24 13:27:23.933 Error: NameError: name 're' is not defined 

what I also remembered from another issue was to import the module in the def again to prevent errors.
So

Code: Select all

def data_meteo(name):
    import re
    return list(map(float,re.findall("(\d+\.?\d+);(\d+);(\d+)",DE.Devices[name].s_value)[0]))

The error is still present but interestingly it change the type of error (name 'DE' is not defined)

Code: Select all

2023-01-24 13:40:17.540 Python:[15.9, 65.0, 3.0]
2023-01-24 13:40:17.541 Error: Traceback (most recent call last):
2023-01-24 13:40:17.541 Error: File "Script #1", line 11, in <module>
2023-01-24 13:40:17.541 Error: File "Script #1", line 9, in data_meteo
2023-01-24 13:40:17.541 Error: NameError: name 'DE' is not defined 

Perhaps remove the def from the code completely and see if it works. Not optimal but if it works it could be a workaround. Then it could be a issue in python event scripts (rewritten in 2022 due to incompatibility for newer python versions)

In this code it is already working without the def. I could remove all def however as you mention it is not optimal at all: my actual code is much more complex with more than 20 def. This simple example was chosen only for illustration.
Finally this bug with "def " and "re" is not the only one I have in my code (that was perfectly working for years): I can report at least another bug which is not related to "re". This is the code

Code: Select all

import DomoticzEvents as DE
import datetime,time

time1=datetime.datetime.now() # heure actuelle
date_retour=datetime.datetime(2023,1,2,15) 
DE.Log("Python:"+str(date_retour))

def thermostat_chauffage_chambre():    
    duree=date_retour-time1
    seuil_temperature_chambre=12
    return seuil_temperature_chambre  
    
thermostat_chauffage_chambre()    

and the output shows that even if "date_retour" is known it gives an error when called inside the def

Code: Select all

2023-01-24 13:45:34.404 Python:2023-01-02 15:00:00
2023-01-24 13:45:34.406 Error: Traceback (most recent call last):
2023-01-24 13:45:34.406 Error: File "Script #2", line 13, in <module>
2023-01-24 13:45:34.406 Error: File "Script #2", line 9, in thermostat_chauffage_chambre
2023-01-24 13:45:34.406 Error: NameError: name 'date_retour' is not defined 

Nothing seems to work inside a def, this is crazy

[willemd]

Hmm.... strange. I tested your script again, the one at the beginning of this thread.

In python from commandline: no problem.
In python from domoticz: same error as yours.

Then I played around a bit with the code and just added "global re" within the def definition (even though it does not make sense...)

The error disappeared and after that the script ran normally, even if I removed the "global re" line again.

Worth a try.

[willemd]

And doing the same in your latest script also solves the problem.

So my conclusion is than when python is run from domoticz, there is something wrong with the global scope of the variables. The def's used within domoticz don't import the globally defined variables unless specifically defined.

the below script just run's fine when the global defnition is included and continues to run when it is removed again, but the initial install fails because the global scope is not recognised properly.

I will add my finding to the github issue report.

Code: Select all

import DomoticzEvents as DE
import datetime,time

time1=datetime.datetime.now() # heure actuelle
date_retour=datetime.datetime(2023,1,2,15) 
DE.Log("Python:"+str(date_retour))

def thermostat_chauffage_chambre():    
    global date_retour,time1
    duree=date_retour-time1
    seuil_temperature_chambre=12
    return seuil_temperature_chambre  
    
DE.Log("from def" + str(thermostat_chauffage_chambre()))  

[lambertmular]

Many many thanks willemd.
As I told in a previous post, my actual code is very long and complex, however after treating almost all variables with your recipe: using "global" each time they throw me an error, I managed to make my code work again!
Great great job, I can't thank you enough!

I must say that I do not understand the solution nor the problem. Do you think it will remain stable or that I will need to use the "global" trick from time to time?

[willemd]

I must say that I do not understand the solution nor the problem. Do you think it will remain stable or that I will need to use the "global" trick from time to time?

Let's wait and see what the experts say about a solution. The "global" statement should not be necessary, it should be a temporary workaround.

[lambertmular]

I must say it is not completely working.
For example without the global statement it seems that some variable inside "def" are not updated as you can see in this code

Code: Select all

import DomoticzEvents as DE
import datetime,time
time1=datetime.datetime.now() # heure actuelle
DE.Log("Python script1: time1"+str(time1))

def fun():
    DE.Log("Python script1: time1"+str(time1))
fun()

whose output is

Code: Select all

2023-01-24 16:34:04.795 Python script1: time12023-01-24 16:34:04.795070
2023-01-24 16:34:04.795 Python script1: time12023-01-24 16:28:17.774032 

Clearly "time1" is not updated inside the "def.
Instead if I add a "global:

Code: Select all

import DomoticzEvents as DE
import datetime,time
time1=datetime.datetime.now() # heure actuelle
DE.Log("Python script1: time1"+str(time1))

def fun():
    global time1
    DE.Log("Python script1: time1"+str(time1))
fun()

Then it works normally

Code: Select all

2023-01-24 16:35:13.695 Python script1: time12023-01-24 16:35:13.695776
2023-01-24 16:35:13.696 Python script1: time12023-01-24 16:35:13.695776 

It seems that I cannot withdraw all the "global" commands

[willemd]

That again proofs that something is wrong with the import of global definitions into the def scope.

Normally globally defined variables should also be known within the def scope and also their values should be taken "from the outside". So if you print time1 inside the def scope it should show the same value as outside (even without the global line).

You would only use a "global" definition within the def scope if you want to update the variable within the def and make the variable value available globally to the total program. If you don't use the "global" line within the def definition, then any updates to the variable within the def are lost when processing returns to the main program.

So there is also a risk if you leave the global statement in place as a workaround. It will export the value of those variables to the main scope, even if you don't want to. Whether this is a problem depends on your program.

[rezzalex]

Hello everyone,
I would like to know if my issue is related to the recent bug described in this post.

I am using the official Docker DZ image, currently on almost the latest beta :

Version: 2022.2 (build 14975)
Build Hash: 932461e0c
Compile Date: 2023-01-18 17:23:35
dzVents Version: 3.1.8
Python Version: 3.7.3 (default, Oct 31 2022, 14:04:00) [GCC 8.3.0]

I try to use a Python script from DZ script editor, and try to past the following command at the beginning :

Code: Select all

import requests

and this one later-on in the same script :

Code: Select all

requestsession = requests.Session()

I have installed "requests" in here :

Code: Select all

/usr/lib/python3.7/

and here :

Code: Select all

/usr/local/lib/python3.7/dist-packages

but the script keep falling saying

Code: Select all

NameError: name 'requests' is not defined

isn't it weird to be on this Python Version ? Should I rebuild the container completely?

[willemd]

Hello everyone,
I would like to know if my issue is related to the recent bug described in this post.

Could be the same issue.

Check:
1) is your script running from commandline without problems?
2) Is the requestsession=requests.Session() line within a function definition?
3) If you add the statement "global requests" inside the definition, does this remove the error message?

If so, it is the same issue.

[waltervl]

In docker be sure you have the extra python module(s) installed in the Domoticz container else it cannot find it. You can use the customstart.sh for that. See the Domoticz docker wiki.

[rezzalex]

In docker be sure you have the extra python module(s) installed in the Domoticz container else it cannot find it. You can use the customstart.sh for that. See the Domoticz docker wiki.

as already written, I have installed the "requests" package in /usr/lib/python3.7/ and in "/usr/local/lib/python3.7/dist-packages"

[rezzalex]

Hello everyone,
I would like to know if my issue is related to the recent bug described in this post.

Could be the same issue.

Check:
1) is your script running from commandline without problems?
2) Is the requestsession=requests.Session() line within a function definition?
3) If you add the statement "global requests" inside the definition, does this remove the error message?

If so, it is the same issue.

1) yes; it nows run smoothly from commandline
2) Yes, it is a function defined in the request Python package : https://requests.readthedocs.io/en/late ... /advanced/
3) can not edit a python package "Python out of the box" function

My issue seems rather the one described here :
https://github.com/domoticz/domoticz/issues/5339

[waltervl]

In docker be sure you have the extra python module(s) installed in the Domoticz container else it cannot find it. You can use the customstart.sh for that. See the Domoticz docker wiki.

as already written, I have installed the "requests" package in /usr/lib/python3.7/ and in "/usr/local/lib/python3.7/dist-packages"

A docker container is a seperate system. So installing request on your host system will not enable request on your docker environment. Please investigate the Docker environment in general and the Domoticz docker in detail. See also the wiki page: https://www.domoticz.com/wiki/Docker

Edit: to get into the docker container bash use:

Code: Select all

docker exec -it domoticz 

From there install module request and try again.

[rezzalex]

In docker be sure you have the extra python module(s) installed in the Domoticz container else it cannot find it. You can use the customstart.sh for that. See the Domoticz docker wiki.

as already written, I have installed the "requests" package in /usr/lib/python3.7/ and in "/usr/local/lib/python3.7/dist-packages"

A docker container is a seperate system. So installing request on your host system will not enable request on your docker environment. Please investigate the Docker environment in general and the Domoticz docker in detail. See also the wiki page: https://www.domoticz.com/wiki/Docker

I was previously talking about the DZ container, not my Host.

[waltervl]

I was previously talking about the DZ container, not my Host.

Ok, that was not clear but very important to know.