Persistent data

[Toulon7559]

A database or a file is an appropriate medium to store information, manipulating it for reading and writing,
and it 'survives' when a function or computer stops.
As element in Domoticz the user variables are a means to keep values, with the nice aspect of individual adressing.
Also, after initiation they automatically become part of Domoticz' background database.
.
Is in Python 2.x and Python 3.x an equivalent available for those Domoticz' user variables?
Or required to mimick (howto?)?

[Number8]

Are you looking for something like data pickle?

[Toulon7559]

Never heard of pickling, but the description looks promising, because really simple structure and very short scriptlines.
Have also been thinking of similar application of a Python dictionary as auxiliary file, but then script quickly expands.

[Number8]

yes I use it in a python script, pretty easy to set-up

[Toulon7559]

Meanwhile (with help from Google, and side-looking to pickle) also found a setup making use of
- a default & working dictionary,
- at end of script-cycle saving the dictionary for backup as json-file, and
- at start of next script-cycle after read-back reconstruction as reference dictionary.

The script-segment in the spoiler below demonstrates an application of such setup for read-out for a weatherstation, and generation of some derived values.

BE AWARE THAT THIS IS NOT A RUNNING SCRIPT BY ITSELF, BUT A SEGMENT FOR DEMONSTRATION OF THE SETUP,
extracted from this Python3.x application!!!

Spoiler: show

#!/usr/bin/python3

# THIS IS A SEGMENT OF A PYTHON SCRIPT JUST DEMONSTRATING MANIPULATION OF A DICTIONARY AS EXTERNAL, WORKING 'MEMORY'
# IT CANNOT BE RUN STAND-ALONE, BUT NEEDS HEAD & TAIL TO BECOME USEFUL!!!!!!!
# [Other functions of script, inculding preset & definition for variables]

# -------------------------------------------------
# Line xxx = Rain_Cumulation & Extreme values & Times
# -------------------------------------------------
print (' Functions for processing of Rain-values & Extremes ToBeDefined/Added')
# Define default list for cumulations and extremes
# Line xxx = Setup of framework for dictionary
# Initial contents in the dictionary to be tuned with realistic values at date&time of start-up !!!!!!!
Aux_dict = {'ZDate': 7, 'ZTime': 1330, 'Rain_Today': 0.0, 'Rain_Yesterday': 0.7, 'Rain_ThisMonth': 5.5, 'Rain_ThisYear': 283.1, 'TempCMax_Val': 11.2, 'TempCMax_Time': 0, 'TempCMin_Val': 25.2, 'TempCMin_Time': 0, 'RHumMax_Val': 60, 'RHumMax_Time': 0, 'RHumMin_Val': 99, 'RHumMin_Time': 0, 'WGustMax_Val': 0, 'WGustMax_Time': 0, 'SolRadMax_Val': 0, 'SolRadMax_Time': 0, 'Test_Value': 45.00}
# Test the layout
print(Aux_dict)
# Reference for test on single value
print(Aux_dict['Test_Value'])
# Test change of single value
Aux_dict['Test_Value'] = 2
print(Aux_dict['Test_Value'])
print(Aux_dict)

# Line xxx = Read the dedicated, earlier saved version of the dictionary

# reading the data from the file
with open('/home/pi/Veurne_dict.json') as f:
data = f.read()

print("Data type before reconstruction : ", type(data))

# reconstructing the data as a dictionary
js = json.loads(data)
print("Data type after reconstruction : ", type(js))
print(js)
# Replace working dictionary by read copy
Aux_dict = js

# Line xxx = Calculate Rain-values and adjust Extreme Values & Times of Occurrence
# Line xxx = Setup for calculation of Rain_Today, Rain_ThisMonth and Rain_ThisYear
# Rain_Today = Rain accumulated today by addition of each 10minute-value
# Rain_Yesterday = Rain accumulated this Month at end of Yesterday
# Rain_LastMonth = Rain accumulated this Year at end of Last Month
# Read from datafile the actual values for Date, Time & Rain_since_midnight
# Line xxx = Check NewYear change and reset values

print('NewYear transition to be added')

# RainRate = Rain/lastHour = 6 * Rain10min
# if (New_Day = (Date <> ZDate)) then
# Rain_Today = 0
# if (New_Month = (Date < ZDate)) and (Time < 0010) then
# Rain_LastMonth = Rain_ThisYear
# Rain_Today = 0, Rain_Yesterday = 0, Rain_ThisMonth = 0
# if Time > 0010 and Time < 2355 then
# Rain_Today = Rain_since_midnight
# Rain_ThisMonth = Rain_Yesterday + Rain_Today
# Rain_ThisYear = Rain_ThisYear + Rain_Today
# store values
# else
# Rain_Yesterday = Rain_ThisMonth
# store value
# Line xxx = Setup to determine new extreme value & time since midnight
# if Time < 0010 then
# Read values and store values&times
# else
# if Value > Val_Max then
# Val_Max = Value, Time_Max = Time, store value&time
# if Value < Val_Min then
# Val_Min = Value, Time_Min = Time, store value&time
#
# Line xxx = Realization of calculation of Rain_Today
# in line with setup mentioned above, is reset of Rain_Today at change of date

# Line xxx = Check midnight change and reset Rain_Today

if Rain_incl == 1:
# Rainrate by extrapolation
RainRate1 = 6 * Rain10min
print('RainRate1 = ',RainRate1)
# Rainrate by cumulation of last 6 10minute-values
RainRate2 = round(float( obj_test[17][4] + obj_test[17][4] +obj_test[16][4] + obj_test[15][4] + obj_test[14][4] + obj_test[13][4]), 1)
print('RainRate2 = ',RainRate2)

if ( int(DD_CET) != int(Aux_dict['ZDate']) ):
Rain_Today = 0
Aux_dict['Rain_Today'] = 0

# Line xxx = Realization of calculation of Rain_ThisMonth
# Line xxx = Check start of Month and backup/reset values

if ( int(DD_CET) < int(Aux_dict['ZDate']) ):
RainLastMonth = float(Aux_dict['Rain_ThisYear'])
Aux_dict['Rain_LastMonth'] = RainLastMonth
Aux_dict['Rain_Today'] = 0
Aux_dict['Rain_Yesterday'] = 0
Aux_dict['Rain_ThisMonth'] = 0
Aux_dict['ZDate'] = DD_CET
Aux_dict['ZTime'] = HHMM_CET

# Line xxx = Update/refresh the Rain-values

if ( int(HHMM_CET > 10) and int(HHMM_CET < 2355)) and (Rain_incl == 1):
RainIncrease = float(Rain10min)
RainToday = float(Aux_dict['Rain_Today']) + RainIncrease
Aux_dict['Rain_Today'] = RainToday
RainThisMonth = float(Aux_dict['Rain_Yesterday']) + RainIncrease
Aux_dict['Rain_ThisMonth'] = RainThisMonth
RainThisYear = float(Aux_dict['Rain_ThisYear']) + RainIncrease
Aux_dict['Rain_ThisYear'] = RainThisYear
Aux_dict['ZDate'] = DD_CET
Aux_dict['ZTime'] = HHMM_CET
else:
Aux_dict['Rain_Yesterday'] = Aux_dict['Rain_ThisMonth']
Aux_dict['ZDate'] = DD_CET
Aux_dict['ZTime'] = HHMM_CET

print('Date&Time + Rain values included')

# Line xxx = Realization of adjustment of Extremes
# in line with setup mentioned above

if ( int(HHMM_CET) < 10):
Aux_dict['TempCMax_Val'] = TempC_SI
Aux_dict['TempCMax_Time'] = str(dattim7)
Aux_dict['TempCMin_Val'] = TempC_SI
Aux_dict['TempCMin_Time'] = str(dattim7)
Aux_dict['RHumMax_Val'] = RHum_SI
Aux_dict['RHumMax_Time'] = str(dattim7)
Aux_dict['RHumMin_Val'] = RHum_SI
Aux_dict['RHumMin_Time'] = str(dattim7)
Aux_dict['WGustMax_Val'] = WindG_SI
Aux_dict['WGustMax_Time'] = str(dattim7)
Aux_dict['SolRadMax_Val'] = SolRad_SI
Aux_dict['SolRadMax_Time'] = str(dattim7)
Aux_dict['ZDate'] = str(DD_CET)
Aux_dict['ZTime'] = str(HHMM_CET)

if ( float(TempC_SI) > float(Aux_dict['TempCMax_Val']) ):
Aux_dict['TempCMax_Val'] = str(TempC_SI)
Aux_dict['TempCMax_Time'] = str(dattim7)

if ( float(TempC_SI) < float(Aux_dict['TempCMin_Val']) ):
Aux_dict['TempCMin_Val'] = float(TempC_SI)
Aux_dict['TempCMin_Time'] = str(dattim7)

if ( float(RHum_SI) > float(Aux_dict['RHumMax_Val']) ):
Aux_dict['RHumMax_Val'] = str(RHum_SI)
Aux_dict['RHumMax_Time'] = str(dattim7)

if ( float(RHum_SI) < float(Aux_dict['RHumMin_Val']) ):
Aux_dict['RHumMin_Val'] = float(RHum_SI)
Aux_dict['RHumMin_Time'] = str(dattim7)

if ( float(WindG_SI) > float(Aux_dict['WGustMax_Val']) ):
Aux_dict['WGustMax_Val'] = float(WindG_SI)
Aux_dict['WGustMax_Time'] = str(dattim7)

if ( float(SolRad) > float(Aux_dict['SolRadMax_Val']) ):
Aux_dict['SolRadMax_Val'] = str(SolRad_SI)
Aux_dict['SolRadMax_Time'] = str(dattim7)

# Line xxx = Make backup copy with filename tuned to station_name
Veurne_dict = Aux_dict
print
print('Backup file')
print(Veurne_dict)

# Line xxx = Save station dictionary
# Make JSON-file
with open('/home/pi/Veurne_dict.json', 'w') as outfile:
json.dump(Veurne_dict, outfile)

print ('End of calculation of Rain-Values & Extreme Values')

# [More other functions of script]

[Number8]

Good to know, thanks

[Toulon7559]

Rummaging with Google for a completely different aspect, found this contribution by Xorfor:
is a setup which more directly links Python to the facilities & services of Domoticz, including the uservariables and devices.
IMHO it even enables that a Python-script and lua/dzVents-script share common data from Domoticz, avoiding seperate specific constructions & translations.

Seems quite useful, but although already existing a few years, must admit that never seen an application.