Domoticz

Hi I have the following problem I use a Raspberry B with msata disk running Domoticz so far no problem. But when I install Nefit Easy(it works great, all 10 sensors) the processor activity is higher and higher beginning with 1.5 % and every day 1 % higher. I have looked to the tasks and the Node task is making the high activity. I am not a Unix expert, and 69 years old, never to old to learn something. But I would ask, have someone an idea?

pa3gmi wrote: ↑Monday 11 June 2018 21:03 Hi I have the following problem I use a Raspberry B with msata disk running Domoticz so far no problem. But when I install Nefit Easy(it works great, all 10 sensors) the processor activity is higher and higher beginning with 1.5 % and every day 1 % higher. I have looked to the tasks and the Node task is making the high activity. I am not a Unix expert, and 69 years old, never to old to learn something. But I would ask, have someone an idea?

node is Java based and Java based applications are "famous" of taking a lot of CPU and memory. But I run on Ubuntu and have 4Gb memory and 2 Atom CPU's. I have been running the easy-server for as long as it exists, but do not see any significant CPU increase for days/weeks/months. Maybe search the internet or node user forums if this is a known issue. At least is does not sound as a Domoticz issue.
My 2cents, Harry

Hi Harry,

You are right that it is not an Domoticz item, with only Domotics and a my sensors gateway an RF-link Gateway no problems at all, so there must be something in the easy-server that make my pi upset. Is there a Nefit easy server forum? i can't find it.

SweetPants wrote: ↑Tuesday 12 June 2018 13:39

pa3gmi wrote: ↑Monday 11 June 2018 21:03 Hi I have the following problem I use a Raspberry B with msata disk running Domoticz so far no problem. But when I install Nefit Easy(it works great, all 10 sensors) the processor activity is higher and higher beginning with 1.5 % and every day 1 % higher. I have looked to the tasks and the Node task is making the high activity. I am not a Unix expert, and 69 years old, never to old to learn something. But I would ask, have someone an idea?

node is Java based and Java based applications are "famous" of taking a lot of CPU and memory. But I run on Ubuntu and have 4Gb memory and 2 Atom CPU's. I have been running the easy-server for as long as it exists, but do not see any significant CPU increase for days/weeks/months. Maybe search the internet or node user forums if this is a known issue. At least is does not sound as a Domoticz issue.
My 2cents, Harry

I have sinds last month also problems with memory ans CPU use then Nefit-easy services is running.
After 1 of 2 day the services crasht and must be restart with "PM2 restart 0"


(On 8 jun the services crasht, 10 jun I reboot the RBP, 14 Jun the services crasht also)

I use node v6.14.3 and easy-server 4.0.1 and pm2 v2.10.4

Have somebody a idea to resolve this problem?

pa3gmi wrote: ↑Wednesday 13 June 2018 8:13 Is there a Nefit easy server forum? i can't find it.

This is the author of the easy-server, maybe he can help you https://github.com/robertklep/nefit-easy-http-server
BTW, i'm using the same node version and easy-server version as you do and have no problems on Ubuntu Linux. You are on Raspberry, maybe there is an issue there?

Hi,

I see more people having problems with that, I use node -v (versie 10.4.1)
npm -v (6.1.0) easy-server -v (4.0.1) pm2 -v=2.10.4 and Raspbian stretch kernel 4.14

Another thing I see today, in the hour gas log I see 0.012 m3 gas at 7 o clock in the morning, further no usage. In the day gas log I See 2.203 m3

Rien

i have the same memory and cpu issues as mentioned above

when i open putty and connect to my rpi, and type:
pm2 logs 0 --lines 1000
i get the eror message below several times

does anybody else also see these errors?

could this be a reason why the memory and cpu consumption is increasing?

0|nefiteas | (node:571) UnhandledPromiseRejectionWarning: TypeError: Cannot read property 'write' of null
0|nefiteas | at Promise (/usr/lib/node_modules/nefit-easy-http-server/node_modules/@xmpp/connection/index.js:307:19)
0|nefiteas | at new Promise (<anonymous>)
0|nefiteas | at Client.write (/usr/lib/node_modules/nefit-easy-http-server/node_modules/@xmpp/connection/index.js:305:12)
0|nefiteas | at Client.send (/usr/lib/node_modules/nefit-easy-http-server/node_modules/@xmpp/connection/index.js:292:17)
0|nefiteas | at Client.Connection.send (/usr/lib/node_modules/nefit-easy-http-server/node_modules/nefit-easy-core/lib/index.js:15:15)
0|nefiteas | at Client.send (/usr/lib/node_modules/nefit-easy-http-server/node_modules/@xmpp/client-core/lib/Client.js:18:18)
0|nefiteas | at Object.queue.add [as promiseGenerator] (/usr/lib/node_modules/nefit-easy-http-server/node_modules/nefit-easy-core/lib/index.js:125:19)
0|nefiteas | at Queue._dequeue (/usr/lib/node_modules/nefit-easy-http-server/node_modules/promise-queue/lib/index.js:153:30)
0|nefiteas | at /usr/lib/node_modules/nefit-easy-http-server/node_modules/promise-queue/lib/index.js:109:18
0|nefiteas | at new Promise (<anonymous>)
0|nefiteas | at Queue.add (/usr/lib/node_modules/nefit-easy-http-server/node_modules/promise-queue/lib/index.js:94:16)
0|nefiteas | at NefitEasyCommands.queueMessage (/usr/lib/node_modules/nefit-easy-http-server/node_modules/nefit-easy-core/lib/index.js:122:23)
0|nefiteas | at NefitEasyCommands.send (/usr/lib/node_modules/nefit-easy-http-server/node_modules/nefit-easy-core/lib/index.js:197:17)
0|nefiteas | at queueMessage.catch.e (/usr/lib/node_modules/nefit-easy-http-server/node_modules/nefit-easy-core/lib/index.js:201:19)
0|nefiteas | at <anonymous>
0|nefiteas | (node:571) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). (rejection id: 6405)

Can you start the easy-server without pm2? I do not use pm2 and do not experience these problem(s)

i used the instruction in the wiki to install nefiteasy (https://www.domoticz.com/wiki/NefitEasy)
how should i start it without pm2
(if pm2 is not the method to use, should the wiki be modified?)

peerke72 wrote: ↑Sunday 17 June 2018 18:25 (if pm2 is not the method to use, should the wiki be modified?)

just start you easy-server using:

/usr/bin/easy-server --host=127.0.0.1 --serial=<your serial> --access-key=<your access key> --password=<your password>

thanks! i'll try that.

to autostart nefit easy server after a reboot i've added it to /etc/rc.local with:

sudo nano /etc/rc.local
and added the command you provided
/usr/bin/easy-server --host=127.0.0.1 --serial=<your serial> --access-key=<your access key> --password=<your password>

now fingers crossed and wait if the cpu and memory consumption get back to normal

edit:

unfortunately the memory consumption is still increasing

but the error i posted before (UnhandledPromiseRejectionWarning: Unhandled promise rejection.)
seems to be investigated by robert klep (author of the easy server)
https://github.com/robertklep/nefit-easy-core/issues/14

I have had contact with the author Robert Klep and he wrote me the following message:

""I'm afraid someone familiar with Domoticz internals has to take a look at this, because I'm all out of ideas.
To give an idea on where I'm at now:
• I've ruled out that merely running the HTTP-server is the problem, because I've been running one for weeks now and haven't noticed any increasing CPU-usage;
• Having the HTTP-server perform periodic (30s) updates: the HTTP-server mentioned in the previous item did just that. Not a problem.
• Running the core code for a long time on restricted hardware: I have a Homey that's running the Nefit Easy app, which also performs periodic requests to the backend (using the same core code as the HTTP server). My Homey typically has an uptime measured in weeks. Not a problem. Homey is also more restricted in terms of hardware than an RPi, so that's not a cause either.
• All issues regarding increasing CPU usage are from people running Domoticz, so I think it's fair to say that there's some weird interaction between Domoticz and the HTTP server that's causing this.
I'm not familiar with Domoticz at all, so I have no idea how it handles requests to the HTTP-server (including timeouts, scheduling, error-handling, etc).
""
So Are there more people who has the problem of getting higher an higher cpu usage? and what is your configuration.
Mine is:
Raspberrypi B with a lan connection to my router.
Raspbian stretch with desktop release 2018-04-18
node -v (versie 10.4.1)
npm -v (6.1.0)
easy-server -v (4.0.1)
libpython3.4-dev
I have pm2 completely removed.
When I disconnect the easy-server in the hardware the processor usage is getting direct to normal, for me 1.5%

I'm also still experiencing the same problems, and hoped the investigations of robert klep would fix the issue

configuration

Raspberry Pi reference 2018-04-18 (9 (stretch))
node -v v8.11.3
npm -v 6.1.0
easy-server -v (4.0.1)

as mentioned above in my previous post i don't use pm2 to run the nefit easy server (but i didn't remove pm2)



when i look at the pi memory historychart the increase of memory (and also the increased cpu usage) started around 29th may
so i asume something changed around that date in domoticz or in the nefit easy server package or at bosch/nefits webservice (or a combination of them)?

@others when started the memory and cpu increase for you?

when i look at the release date of easy-server 4.0.0 was 29th may and 4.0.1 was 30th may
it looks there could be a relation ...

maybe the reason for the new easy-server releases was a change at bosch/nefits webservice.
and somehow the fix doesn't work in combination with domoticz

what todo to get this fixed?

edit:
i also tried what happened when i disable easy server in the hardware menu.
and indeed the cpu percentage and memory consumption are returning back to normal values

My memory and cpu also goes up. I never used pm2, so that's not the problem I think. I did start to use pm2 as my internet connection isn't perfect, and when it's down I need to restart the easy server, which I'm able to do now with a lua script and pm2.

i added as temporay workaround a dummy switch
with an 'on action': http://127.0.0.1:8080/json.htm?type=com ... tem_reboot
and scheduled that this switch is enabled each night at 4 o'clock to reboot my RPi
offcourse it's a uggly workaround but i hope this will avoid my RPi / domoticz run out off resources

Any solutions to this problem? I have this issue for some weeks now, and follow these threads curiously. However solving seems to stagnate now? I experience the CPU spikes now and then, and every half hour or so the "nefit hardware thread has stopped" message in log. Every 2-3 weeks domoticz crashes.
I have updated Node, Nefit server and latest beta of Domoticz. Anyone working on this issue still?

i (temporay) removed nefit easy http server from my raspberry pi and disabled it also in domoticz in the hardware menu.
domoticz is running as smooth as before ...

but i still wonder why this problem only seems to occur with the combination of
- domoticz
- nefit easy http server
- raspberry pi

and further based on the information of robert klep (see https://github.com/robertklep/nefit-easy-core/issues/14)
it could also depends on how often a Nefit Easy is offline.

maybe it's depending on the (in)stability of the internet/modem connection
so users with a very stable internet connection maybe don't have any issues.

btw my ziggo ubee wifi modem isn't known for it's stability ...
my nefit easy is connected to a seperate wifi router
but maybe there is a relationship with the issue.

for now i don't need to use my heating (today 30 degrees celcius outside pfff) so i can disable nefit easy server
that way domoticz stays running very stable without strange behaviour caused by nefit easy
but before we now it it's freezing again ...

i'm out of options ...
who can help?

Maybe I've found something in the errorlog of PM2 we can use to solve the issue.
The errorlog of PM2 is flooded with the following message:
No Idea what the problem is... but something isn't going as expected.
I also experience more and more CPU and memory usage over time with domoticz. Maybe nodejs is encoutering a loop which takes more and more resources like a memory leak?

-Update-
Currently running a modified debug version of nefit-easy-core as provided by Robert. I'll let it know if things improve.

i've been on holiday for a few weeks.
did installing the 'debug version' improve things or give some usefull log information ?

i see on robert klep's site https://github.com/robertklep/nefit-easy-core/issues/14
that he's still looking into some issues.
maybe they are related to our problem.

My memory and cpu usage is also extreme lately.

I use a workaround to fix this until robert or domoticz has a solution.

Every night at 03:00 I start a script to restart the nefit easy server with LUA:

(script_time_blabla)
The script contains: This way my domoticz won't hang after 2-3 weeks.