GarbageCalendar (new version) lua scripts

[Xenomes]

Ok we gonna try!

[jvdz]

What about when I write a generic module that wil take a CSV file as input that contains GarbageDate en GarbageType? This way the CSV file can contain the whole years data and the nightly "Web update" will then read it and convert it into de required DATA file.

Jos

[Xenomes]

I managed to make a dummy and created a data file (made from another zip code) and alter it to my garbage service dates.

2020-09-18 15_20_38-Domoticz - Overige.png (11.39 KiB) Viewed 1649 times

What about when I write a generic module that wil take a CSV file as input that contains GarbageDate en GarbageType? This way the CSV file can contain the whole years data and the nightly "Web update" will then read it and convert it into de required DATA file.

That will be a better solution! Now i had to change the paths in the config and create a dummy file (org m_mijnafvalwijzer.lua) where i commit out the 'web update' section.

Code: Select all

websitemodule = "m_dummy"

-- Switch on mydebug in case of issues and initially and check the domoticz log for any issues or missing
mydebug      = false  -- (true/false)
testdataload = false  -- (true/false)

-- Specify the appropriate directories (Raspberry Pi example)
datafilepath     = '/home/pi/domoticz/scripts/lua'                           -- specify the directory where the garbagecalendar.data & garbagecalendar.log will be stored
scriptpath       = '/home/pi/domoticz/scripts/lua'      -- specify the directory for the main "script_time_garbagecalendar.lua" script
domoticzjsonpath = '/home/pi/domoticz/scripts/lua'      -- specify the path to domoticz where the JSON.lua file can be found

[jvdz]

The module m_csv_file is available in the Github Development branch. The module itself contains the in expected input format and the variable that needs to be defined in the garbagecalendarconfig.lua file.:

Code: Select all

ver="20200916-1600"
websitemodule="m_csv_file"
--[[
This module requires an inputfile defined by this variable in the configfile:
input_csv_file = "garbagecalendar/garbage_input.csv"
--With the following format:--
garbagedate;garbagetype
20-8-2020;papier
23-8-2020;GFT
1-9-2020;Rest
10-9-2020;papier
21-9-2020;GFT
30-9-2020;Rest
3-10-2020;papier
4-10-2020;GFT
5-10-2020;Rest
--]]

Have a try and see whether that works for you as well.

Jos

[martin82]

wauw , can you make it work for OMRIN also?

in leeuwarden works only OMRIN i think

[Xenomes]

Have a try and see whether that works for you as well.

Thnx, I have set is all up, tomorrow the first notification will be triggered.

[jvdz]

wauw , can you make it work for OMRIN also?

in leeuwarden works only OMRIN i think

I'll have a look later this week to see what is needed to make it work.

[Hann1BaL]

Have a try and see whether that works for you as well.

Thnx, I have set is all up, tomorrow the first notification will be triggered.

Curious, did you get it to work? And did you populate all dates manually in the csv?

Since its broken I realize why I made it work in the first place! I really rely on the light in my room to tell me to put the bin out and which bin that day..

Also, what is the best way to update the script? Just download and write over the files?

[Xenomes]

Thnx, I have set is all up, tomorrow the first notification will be triggered.

Curious, did you get it to work? And did you populate all dates manually in the csv?

Since its broken I realize why I made it work in the first place! I really rely on the light in my room to tell me to put the bin out and which bin that day..

Also, what is the best way to update the script? Just download and write over the files?

Yes i did, but i had made a error in my csv file so i mist the test day . I moved the old files to a backup location and wget the new files (the ones i need) from the develop github and set it up again.

[jvdz]

I've used Excel to edit/update the csv.

[Rolandet]

@jvdz

I gave this script a try. Works like a charm for Omrin
At the first attempt I forgot to add

websitemodule = "m_omrin" to the garbagecalendarconfig.lua

But it's working now.
Only thing I can't get to work, is the icon. It still gives me the default 'text'
any thoughts on that?

[jvdz]

Only thing I can't get to work, is the icon. It still gives me the default 'text'
any thoughts on that?

This is something you need to do in Domoticz yourself with custom ico's.
I haven't played with custom icon myself as I am not working much with the webinterface so didn't bother.

Jos

[Rolandet]

Only thing I can't get to work, is the icon. It still gives me the default 'text'
any thoughts on that?

This is something you need to do in Domoticz yourself with custom ico's.
I haven't played with custom icon myself as I am not working much with the webinterface so didn't bother.

Jos

ok thanks. I'll try figure it out. Thanks for the script

[martin82]

yes thanx for the script en thanx for all the help.


greetz martin

[Xenomes]

Hi jvdz, i have a issue with the m_csv_file module. It is showing two old notifications. i just enable debugging to create a log, normally the script updates 2:30.

2020-10-06 15_47_18-Window.png (47.75 KiB) Viewed 1432 times

Log:

Spoiler: show

Code: Select all

2020-10-06 15:48:00.383 Status: LUA: @GarbageCal(m_csv_file): #### Tue Oct 6 15:48:00 2020 ### Start garbagecalendar script v20200906-1400
2020-10-06 15:48:00.383 Status: LUA: @GarbageCal(m_csv_file): #### Debuging with extra messages because "mydebug=true" in garbagecalendarconfig.lua
2020-10-06 15:48:00.383 Status: LUA: @GarbageCal(m_csv_file): #### please change it back to "mydebug=false" when done testing to avoid growing a big domoticz log.
2020-10-06 15:48:00.383 Status: LUA: @GarbageCal(m_csv_file): domoticzjsonpath: /home/pi/domoticz/scripts/lua/
2020-10-06 15:48:00.384 Status: LUA: @GarbageCal(m_csv_file): datafilepath: /var/tmp/
2020-10-06 15:48:00.384 Status: LUA: @GarbageCal(m_csv_file): scriptpath: /home/pi/domoticz/scripts/lua/
2020-10-06 15:48:00.384 Status: LUA: @GarbageCal(m_csv_file): Loaded /home/pi/domoticz/scripts/lua/garbagecalendar/garbagecalendarconfig.lua.
2020-10-06 15:48:00.385 Status: LUA: @GarbageCal(m_csv_file): Loaded /home/pi/domoticz/scripts/lua/garbagecalendar/generalfuncs.lua.
2020-10-06 15:48:00.386 Status: LUA: @GarbageCal(m_csv_file): Start checking garbagetype_cfg table:
2020-10-06 15:48:00.386 Status: LUA: @GarbageCal(m_csv_file): -> NotificationTime=19:22 Garbagetype=gft
2020-10-06 15:48:00.386 Status: LUA: @GarbageCal(m_csv_file): -> NotificationTime=02:30 Garbagetype=reloaddata
2020-10-06 15:48:00.386 Status: LUA: @GarbageCal(m_csv_file): -> NotificationTime=02:31 Garbagetype=dummy1
2020-10-06 15:48:00.386 Status: LUA: @GarbageCal(m_csv_file): -> NotificationTime=19:22 Garbagetype=pmd
2020-10-06 15:48:00.386 Status: LUA: @GarbageCal(m_csv_file): -> NotificationTime=19:22 Garbagetype=papier
2020-10-06 15:48:00.386 Status: LUA: @GarbageCal(m_csv_file): Start update for text device:
2020-10-06 15:48:00.387 Status: LUA: @GarbageCal(m_csv_file): - Start looping through data from the website to find the first 3 event to show: /var/tmp/garbagecalendar_m_csv_file.data
2020-10-06 15:48:00.388 Status: LUA: @GarbageCal(m_csv_file): - End -----------------
2020-10-06 15:48:00.388 Status: LUA: @GarbageCal(m_csv_file): #!# Warning: These records are missing in your garbagecalendarconfig.lua file, so no notifications will be send!
2020-10-06 15:48:00.388 Status: LUA: @GarbageCal(m_csv_file): #!# -- start -- Add these records into the garbagetype_cfg table and adapt the schedule and text info to your needs :
2020-10-06 15:48:00.388 Status: LUA: @GarbageCal(m_csv_file): #!# -- end ----------------------------
2020-10-06 15:48:00.388 Status: LUA: @GarbageCal(m_csv_file): ==> found schedule:gft: maa 28 sep ; plastic: maa 5 okt ; papier: zat 10 okt ;
2020-10-06 15:48:00.388 Status: LUA: @GarbageCal(m_csv_file): No updated text for TxtDevice.
2020-10-06 15:48:00.389 Status: LUA: @GarbageCal(m_csv_file): #### 15:48:00 End garbagecalendar script v20200906-1400
2020-10-06 15:48:00.411 Status: EventSystem: Script event triggered: /home/pi/domoticz/scripts/lua/script_time_garbagecalendar.lua

[jvdz]

I see a couple of strange things in the log that I don't understand, so could you Email me the following files in a zip file, so I can have a closer look? :

• Garbagecalendarconfig.lua
• Your input datafile with the csv dates as defined in the config.
• /var/tmp/garbagecalendar_m_csv_file.data
• /var/tmp/garbage_run_m_csv_file.log
• /var/tmp/garbage_run_update_m_csv_file.log
• /var/tmp/garbage_web_m_csv_file.log

.. but it feels like something is going wrong in the nightly run. This info is located in the garbage_web_m_csv_file.log.

Thanks, Jos

[iTDB]

Because my Domoticz no longer worked properly, I reinstalled it completely. I took a test raspberry for the Garbage calendar (m_recycleapp-be is now used) and it doesn't work at all anymore.
I get the following error:
2020-10-07 14:38:00.927 Error: EventSystem: in /home/pi/domoticz/scripts/lua/script_time_garbagecalendar.lua: ...domoticz/scripts/lua/garbagecalendar/m_recycleapp-be.lua:78: attempt to index a nil value (field '?')


2020-10-07 14:38:00.338 Status: LUA: @GarbageCal(m_recycleapp-be): #### Wed Oct 7 14:38:00 2020 ### Start garbagecalendar script v20200906-1400
2020-10-07 14:38:00.338 Status: LUA: @GarbageCal(m_recycleapp-be): #### Debuging with extra messages because "mydebug=true" in garbagecalendarconfig.lua
2020-10-07 14:38:00.338 Status: LUA: @GarbageCal(m_recycleapp-be): #### please change it back to "mydebug=false" when done testing to avoid growing a big domoticz log.
2020-10-07 14:38:00.339 Status: LUA: @GarbageCal(m_recycleapp-be): domoticzjsonpath: /home/pi/domoticz/scripts/lua/
2020-10-07 14:38:00.339 Status: LUA: @GarbageCal(m_recycleapp-be): datafilepath: /home/pi/domoticz/logs/
2020-10-07 14:38:00.339 Status: LUA: @GarbageCal(m_recycleapp-be): scriptpath: /home/pi/domoticz/scripts/lua/
2020-10-07 14:38:00.339 Status: LUA: @GarbageCal(m_recycleapp-be): Loaded /home/pi/domoticz/scripts/lua/garbagecalendar/garbagecalendarconfig.lua.
2020-10-07 14:38:00.341 Status: LUA: @GarbageCal(m_recycleapp-be): Loaded /home/pi/domoticz/scripts/lua/garbagecalendar/generalfuncs.lua.
2020-10-07 14:38:00.341 Status: LUA: @GarbageCal(m_recycleapp-be): Start checking garbagetype_cfg table:
2020-10-07 14:38:00.342 Status: LUA: @GarbageCal(m_recycleapp-be): -> NotificationTime=19:22 Garbagetype=papier
2020-10-07 14:38:00.342 Status: LUA: @GarbageCal(m_recycleapp-be): -> NotificationTime=02:31 Garbagetype=dummy1
2020-10-07 14:38:00.342 Status: LUA: @GarbageCal(m_recycleapp-be): -> NotificationTime=19:22 Garbagetype=pmd
2020-10-07 14:38:00.342 Status: LUA: @GarbageCal(m_recycleapp-be): -> NotificationTime=02:30 Garbagetype=reloaddata
2020-10-07 14:38:00.342 Status: LUA: @GarbageCal(m_recycleapp-be): -> NotificationTime=19:22 Garbagetype=gft
2020-10-07 14:38:00.342 Status: LUA: @GarbageCal(m_recycleapp-be): Start update for text device:
2020-10-07 14:38:00.342 Status: LUA: !!! table.load error:cannot open /home/pi/domoticz/logs/garbagecalendar_m_recycleapp-be.data: No such file or directory
2020-10-07 14:38:00.343 Status: LUA: !!! unable to read Data from /home/pi/domoticz/logs/garbagecalendar_m_recycleapp-be.data
2020-10-07 14:38:00.343 Status: LUA: @GarbageCal(m_recycleapp-be): ### Warning: Datafile not found:/home/pi/domoticz/logs/garbagecalendar_m_recycleapp-be.data . Start webupdate now.
2020-10-07 14:38:00.343 Status: LUA: @GarbageCal(m_recycleapp-be): 14:38:00 start new foreground webupdate for module m_recycleapp-be of file /home/pi/domoticz/logs/garbagecalendar_m_recycleapp-be.data
2020-10-07 14:38:00.350 Status: LUA: -H "x-secret: Qp4KmgmK2We1ydc9Hxso5D6K0frz3a9raj2tqLjWN5n53TnEijmmYz78pKlcma54sjKLKogt6f9WdnNUci6Gbujnz6b34hNbYo4DzyYRZL5yzdJyagFHS15PSi2kPUc4v2yMck81yFKhlk2aWCTe93" -H "x-consumer: recycleapp.be"
2020-10-07 14:38:00.927 Error: EventSystem: in /home/pi/domoticz/scripts/lua/script_time_garbagecalendar.lua: ...domoticz/scripts/lua/garbagecalendar/m_recycleapp-be.lua:78: attempt to index a nil value (field '?')

[jvdz]

@iTDB,
So it tries to do a web update since the data files doesn't exists yet so I need to see the output for that: What is the contents of the garbage_web_m_recycleapp-be.log?

Jos

[iTDB]

@m_recycleapp-be: #### Wed Oct 7 14:56:00 2020 ### Start garbagecalendar module m_recycleapp-be (v202008018-1700)
@m_recycleapp-be: Loaded JSON.lua.
@m_recycleapp-be: !!! perform background update to /home/pi/domoticz/logs/garbagecalendar_m_recycleapp-be.data for Zipcode 9000 - 59 (optional) Host$
@m_recycleapp-be: ---- web update ----------------------------------------------------------------------------
@m_recycleapp-be: sQuery=curl -k -H "x-secret: Qp4KmgmK2We1ydc9Hxso5D6K0frz3a9raj2tqLjWN5n53TnEijmmYz78pKlcma54sjKLKogt6f9WdnNUci6Gbujnz6b34hNbYo4Dz$
@m_recycleapp-be: ---- web data ----------------------------------------------------------------------------
@m_recycleapp-be: {"accessToken":"eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpYXQiOjE2MDIwNzUzNjAsImV4cCI6MTYwMjA3ODk2MCwiYXVkIjoicmVjeWNsZWFwcC5iZSJ9.n$
@m_recycleapp-be: ---- web err ------------------------------------------------------------------------
@m_recycleapp-be: Web_Err= % Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
^M 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0^M100 175 100 175 0 0 1325 0 --:--:-- --:--:-- --:$

@m_recycleapp-be: ---- end web data ------------------------------------------------------------------------
@m_recycleapp-be: accessToken:eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpYXQiOjE2MDIwNzUzNjAsImV4cCI6MTYwMjA3ODk2MCwiYXVkIjoicmVjeWNsZWFwcC5iZSJ9.nqKMh$
@m_recycleapp-be: sQuery=curl -k -H "x-secret: Qp4KmgmK2We1ydc9Hxso5D6K0frz3a9raj2tqLjWN5n53TnEijmmYz78pKlcma54sjKLKogt6f9WdnNUci6Gbujnz6b34hNbYo4Dz$
@m_recycleapp-be: ---- web data ----------------------------------------------------------------------------
@m_recycleapp-be: {"items":[{"city":{"names":{"nl":"Gent","fr":"Gand","de":"Gent","en":"Gent"},"zipcodes":["9000-44021","9030-44021","9031-44021","90$
@m_recycleapp-be: ---- web err ------------------------------------------------------------------------
@m_recycleapp-be: Web_Err= % Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
^M 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0^M100 789 0 789 0 0 5759 0 --:--:-- --:--:-- --:$

@m_recycleapp-be: ---- end web data ------------------------------------------------------------------------
@m_recycleapp-be: postcode_id:9000-44021
@m_recycleapp-be: sQuery=curl -k -H "x-secret: Qp4KmgmK2We1ydc9Hxso5D6K0frz3a9raj2tqLjWN5n53TnEijmmYz78pKlcma54sjKLKogt6f9WdnNUci6Gbujnz6b34hNbYo4Dz$
@m_recycleapp-be: ---- web data ----------------------------------------------------------------------------
@m_recycleapp-be: {"items":[],"total":0,"pages":0,"page":1,"size":20,"self":"http://datatank-datatank-http:8080/v1/s ... odes%5B0%5$
@m_recycleapp-be: ---- web err ------------------------------------------------------------------------
@m_recycleapp-be: Web_Err= % Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
^M 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0^M100 469 0 469 0 0 3692 0 --:--:-- --:--:-- --:$

@m_recycleapp-be: ---- end web data ------------------------------------------------------------------------

[jvdz]

I really need the whole file as attachment in an EMail as I can only see the first 150 characters.
My current guess is that the streetname isn't found, but I can only determine that when I have access to the whole file.

see next post
Jos