Large domoticz implementation

[JanVlietland]

Hi, I just wanted to share the results of my domotics adventure over the last 15 months for fun and insight. I implemented a rather extensive Domoticz setup, and here are some key characteristics and lessons learned from the project:

Zigbee: I set up 3 Zigbee networks, each with its own Zigbee2MQTT service, supporting a total of 725 Zigbee entities and 89 devices. Due to Zigbee devices (routers) being unable to handle more than 15 messages per second, I had to split them across different networks. Lua scripts in Domoticz connect the motion sensors to lights, dynamically adjusting based on light conditions and time.

Z-Wave: The sunscreens are managed via Zwave2MQTT, which integrates with Domoticz, controlled by Lua depending on rain, sun and light conditions.

MQTT: The Zigbee networks communicate over MQTT, and Domoticz listens for messages. I was unable to create a persistent MQTT-Domoticz autodiscovery setup, so I also used the Manual MQTT connector in Domoticz, with 2 created adapters (Python) that run as SystemD services to handle Zigbee dimmers and thermostat data.

Heating: The heating system (thermostats, floor valves, and central heater) is controlled by six ESP32 units, for which I wrote custom C software. The hardware units consist of 8-channel relay kits with optocouplers, powered by 24V and 5V supplies for the ESPs. All parts is readily available at Aliexpress. Domoticz controls the heating based on thermostats, interfacing with the ESPs via MySensors. For the temperature dependent central heating, I fed back the outdoor temperature to the heater by developing a temperature/current table with an ESP's A/D converter.

Air Conditioning: I used the reverse-engineered Mitsubishi protocol (mel-ac from GitHub) and wrote an adapter in dzVents to sync Domoticz virtual switches with the AC settings. The air conditioning is switched by motion sensors and temperature.

Airflow: For the 3 Orcon airflow units, I used Lolin D1 Mini's (another GitHub project) and wrote custom software in C since the controller was based on ESPhome, which Domoticz doesn’t support.

Gate and Bells: An Arduino Nano with custom software controls the gate and bell switches. Lua scripts in Domoticz detect bell signals, triggering emails with images from the cameras and controls the gate.

Kodi: I used the Kodi API and a Python script to control Kodi via Domoticz.

Domoticz: I've embraced the saying “Automate Everything.” The system operates autonomously with almost none manual input needed.
Though wall switches can still be used in conjunction with Zigbee devices. Control is optimized based on time, temperature, light conditions, motion, sun power, sun position, rain (knmi), and phone detection. The fact that the lady of the house hasn't complained lately is a testament to the system's success !

Some Key Takeaways:

Integrating off-the-shelf commercial products with Domoticz can be frustrating, but when they use open-source protocols, it becomes much easier.

The strength of Domoticz lies in its ability to combine various data streams. For example, incorporating weather forecasts into the heating system can prevent overshoot and reduce energy consumption. I've achieved a 50% reduction in energy use through smart control.

Combining camera, email, and bell data lets you build a “Ring”-like system in no time. The possibilities with Domoticz are almost limitless.

I love Domoticz’s Lua and dzVents webIDE. Its simplicity and power made it a clear choice over Home Assistant (HA). While HA is great for standard setups, YAML is not ideal for complex conditional programming. Additionally, integrating with other software in HA tends to complicate the system unnecessarily. My current setup — Domoticz (Docker), MQTT, 3 Zigbee2MQTT services, and ZWave2MQTT (Docker) — runs on a single RPI at only 4% CPU usage.

Split Zigbee networks into a maximum of 20-25 devices, especially if you use a lot of motion sensors, which can generate up to one MQTT message per second (I have 22 motion sensors in my setup!).

Use the manual MQTT interface if you need full control, and don’t rely too much on autodiscovery. Autodiscovery can miss devices and entities. The same is true with MySensors autodiscovery, where it’s crucial to ensure that all sensors and outputs present themselves correctly during initialization.

I'm now building a database with 12 months (4 seasons) of data to feed it into a AI with the objective to further optimize energy consumption (I had a setback due to a corrupt Domoticz.db file that resulted in memory leaks, see my other posts for more detail).

I'm considering to upload all programming to GitHub. If you're interested, please let me know, as personally I'm fine with archiving them as a zip file. I'll only invest the time in setting it up if there's enough interest. Cheers !!

[psubiaco]

Wow, very huge system!!
I have only half of your entities (about 400 active), running in a Raspberry PI3 with 1GB ram, and I can say that:
* Domoticz is really strong and reliable
* LUA script is really really good: with a little LUA coding I made a complete alarm system that sends short video taken from the external IP cam when an external motion sensor goes ON (typically, when neighbour cats enters my property)
* LUA script is also very good for heating/cooling system (I have a heat pump with Modbus interface + radiant system + photovoltaic), so it's possible to regulate the heat pump using as inputs the difference Temp-Setpoint (I don't use hardware thermostats, because I want to overheat/overcool in case of extra power from photovoltaic), dT/dt (derivate of temperature), power from photovoltaic, other loads (for example the electric vehicle charging has a higher priority than heating/cooling if indoor temperature is near the setpoint and battery level is much lower than desired value), ...)
* In my case Domoticz CPU is much higher than 4%: EventSystemQueue process is active continuously to manage entities that change their state/value. I have several energy meters, temperature+humidity sensors that fill the Domoticz event queue, so Domoticz call each time the LUA script named script_device*.lua.
To mitigate this overload, I have only one script_device_master.lua that checks what entities have changed, and call the script associated to those entities (bypassing useless entities). For example, if a device named PIR* MCS* SIREN* changes, script_device_master.lua will call the alarm.lua script managing those entities. In this way Domoticz does not run all LUA scripts when any entity changed. I've collected some LUA scripts in https://github.com/CreasolTech/domoticz_lua_scripts , and I suggest you to do the same for your scripts: they can be very useful for anyone.
I also suggest you to send the grabbed picture, when someone ring the bell, to a Telegram channel (within your family), so you can see it immediately.
Keep us informed about your AI way to manage the heating/cooling system: it will be very nice to save lot of energy!
Thanks for your explanation and good luck.

[HvdW]

I love Domoticz’s Lua and dzVents webIDE. Its simplicity and power made it a clear choice over Home Assistant (HA). While HA is great for standard setups, YAML is not ideal for complex conditional programming.

Use the manual MQTT interface if you need full control, and don’t rely too much on autodiscovery. Autodiscovery can miss devices and entities.

I'm considering to upload all programming to GitHub. If you're interested, please let me know, as personally I'm fine with archiving them as a zip file. I'll only invest the time in setting it up if there's enough interest. Cheers !!

The fact that the lady of the house hasn't complained lately is a testament to the system's success !

Indeed. Using some old KlikAan-KlikUit switches that communicate with Domoticz for the lights and the blinds to give it the feel of buttons instead of using the web interface or the Domoticz app.
I have a HomeAssistant system running too, but in my case dzVents with a tiny shot of lua makes Domoticz the prefered system.

[FlyingDomotic]

In my case, I have 450+ entities, with 120 ZigBee devices on one unique Z2M, with less than 4% RPi4 CPU usage, and instant response on actions.

IMHO, it may be a good idea to reduce a bit (drastically?) your motion sensor messages increasing interval (to 1 minute for example), unless you have very specific needs. You can do it changing minimum reporting interval in Z2M. Your coordinator and routes will thank you

This remark may also apply to energy reporting devices, that very often report data with very short interval (3 to 5 seconds), which is useless (unless again you have very specific needs).

You may also (as I did) filter ZigBee entites you don't need (for example temperature from motion sensor/window switches or pressure from temperature sensors, as well as redundant ones like low_bat and bat voltage). This will reduce significantly number of reported entities. I do this with Zigbee2MQTT Plugin, in device blacklist, using regex.