No more P1 gas readings after upgrade to 3.8153

[BlueMoon]

Since I upgraded my Domoticz installation to 3.8153 I have no more gas reading from my P1 Smart meter. My current situation:

Rpi 2 running Raspbian Jessie
Domoticz 3.8452(beta)
P1 smart meter Electricity + Gas (KAIFA) connected via GPIO serial pins
The Electricity part is still working like a charm.

I tried rebooting, upgrading Raspbian, upgrading to latest Domoticz beta, dis/enabling the gas meter device, dis/enabling the P1 hardware entry, but the gas value stays zero.

Eventually I removed the gas meter device, hoping for it to reappear after a few days. But it doesn't, the only way to get it back is restore a previous backup, but still with no values.

the output of

Code: Select all

cat /dev/ttyAMA0

still shows complete telegrams, including correct gas values. What stands out that both the most recent telegrams and one that i happened to have kept from last year or so state that the gas meter is on channel 2

Code: Select all

0-1:24.1.0(003)
0-1:96.1.0()
0-2:24.1.0(003)
0-2:96.1.0([S/N])
0-2:24.2.1([...])

while the domoticz log shows

Code: Select all

P1 Smart Meter: Found gas meter on M-Bus channel 1

Is this a Domoticz issue, or am I doing something wrong. The 'gas season' is slowly beginning and I like to have proper logging again..

[BlueMoon]

I know, but as far as I know it always has. In a telegram I capturered last year i found the same ghost (and the actual meter on ch. 2) and Domoticz never complained and showed me the values.

Is there a way to force Domoticz to use ch.2, or to force the meter to use ch. 1?

[BlueMoon]

Ah, that is very useful information. Good to see some proper data checking added to the code.

Compiling my own Domoticz version is indeed perfectly doable, but it is better to tackle the real problem. When the meter was installed, the fist gas meter appeared to be faulty, so the they took a new one, appearanlty without decoupling the first. I never thought about it until you mentioned there is supposed to be just 1 gas meter.

So I called the grid operator (who owns the meter) with my problem and they were actually quite helpful. First, they will try to remove the obsolete gas meter remotely (which will take a night) and if that doesn't work they send someone to do it on premises.

I'll keep you posted

Edit: oh wow, the gas meter is already showing up in the devices tab, so it was actually fixed within the hour. Now I just have to find out how to merge the data of the new meter with the old readings. Thanks for the help

[Edgar]

Compiling my own Domoticz version is indeed perfectly doable, but it is better to tackle the real problem. When the meter was installed, the fist gas meter appeared to be faulty, so the they took a new one, appearanlty without decoupling the first. I never thought about it until you mentioned there is supposed to be just 1 gas meter.

Very interesting.

I guess the same happened to me. My gas meter stopped working somewhere mid august. The analogue meter kept counting whereas the electronic version stopped. So I was like 18m3 behind and had 6 straight weeks without any gas usage (digitally). When I asked the grid operator about it, they switched both gasmeter and electricity to newer models (previous where 4 years old).

And when the new gasmeter was installed, apparently he used the wrong one and had to change for a second unit. This, after the first one was connected already. He said that they would correct the mistake in a few days, but I guess I have to remember them about this.

Code: Select all

0-1:96.1.0(00000000000000000000000000000000)
0-2:24.1.0(003)
0-2:96.1.0(473030333235631323930353531xxxxxxxxx)
0-2:24.2.1(17100922000S)(00004.617*m3)

Above my datagram. Looks like yours!

@bluemoon; who is your grid operator? If you had a case-id assigned to this, could you PM that to me?

[Edgar]

Fabulous! May still look into allowing the channel to be manually overridden but it's nice to see big companies like that be so helpful.

I vote for a possibility to select a channel. But I'll call the grid operator also

[BlueMoon]

Hmm, with only one gas meter at the same time, there is no real need for channel selection.. It might be easier to just check whether the gas serial > 0 or > "" and otherwise skip it nontheless. I don't know if that is sufficient though. My meter and Edgar's seem to handle the ghost meter differently. Maybe there are even more implementations, which do keep sending their S/N.

@Edgar, you've got mail, but since you have a different meter (output), I guess we don't have the same operator

[Edgar]

Gordon, I have to think this over a bit.

Channel 2: is that 0-1:xx or 0-2:xx?
As you say it, it would look at the data at 0-2:xx by your quote. Maybe I understand it wrong.

Why wouldn't it pick up the device at 0-2 because it has a device identifier (0-2:24.1.0) and a complete set of data for it? I added the P1 meter again in domoticz, but it wouldn't show up after a while. Am I doing something wrong?

If it's because of the dead meter data at 0-1:24 :

I'm not into this compiling stuff. I did take a peek in the sources for P1meter, but it's out of my league.

Gordon, would it be very difficult to parameterize this in the GUI (although it's not a common problem). I guess that in near future when more devices are connected to the smart meter bus, enumerating problems will be occuring more.

I'm also in touch with the grid operator (Liander) but they're not very keen on acting. At least it seems. But I'll give it a few days.

Thanks,

Edgar

[Edgar]

@Edgar, you've got mail, but since you have a different meter (output), I guess we don't have the same operator

Actually, they're quite equal. I forgot to paste one line; 0-1:24

Code: Select all

0-1:24.1.0(003)
0-1:96.1.0(00000000000000000000000000000000)
0-2:24.1.0(003)
0-2:96.1.0(473030333235631323930353531xxxxxxxxx)
0-2:24.2.1(17100922000S)(00004.617*m3)

[Edgar]

Gordon,

If you look closely at my last datagram snippet, you'll see two 24.1.0 record id's with a value 3. The first one has no accompanying 24.2.1 gas reading line, the second has. So it's not a case of one gasmeter entry with two gas readings, or a 24.2.1 record id without any relation to a meter, but one "orphaned" meter entry without gas reading data and one which is complete with a gas reading data entry.

Anyway, it would be nice if the code would be a little bit more robust to cater for these situations. I can imagine that the simplest solution is to present each found / detected 24.1.0 device (so to speak) in the datagram as a domoticz device and let the user decide which one he wants to use.

Also, I got a call back from the grid operator and finally it seems that they understand what went wrong. They made some corrections and my gasmeter is device 1 again without any stray data. I'll try to get more info on the correct procedure / info to give to the grid provider so they know what to do right away.

[BlueMoon]

Good to see it working again Edgar. The funny thing is that the grid operator seems to think that the gas meter not being at ch. 1 is the problem, while the specification (and Domoticz) is perfectly fine with another channel, as long as there is just 1 gas meter.

We (or better: I) simply did not expect that the recorded gas meter channel might need to hold more than just one value. Call it an oversight or too much trust in people that are supposed to follow procedures actually doing so.

Just curious: is it just your assumption (and common sense) that there should be only one gas meter present, or is it actually specified? I can't find any reference to the maximum number of gas meters in de DSMR (v4.2) Is there any other specification that defines this?

If it is specified, there is no need for Domoticz to be more robust. Any other 'victims' should just complain to the grid operator. Besides, I couldn't find any reference to this problem so it might just be the two of us

[BenAW]

Voor Domoticz stable op de RaspberryPi v bij een vriend:
sinds 8 september geen gas meter waardes meer. Weet niet of dit met een Domoticz upgrade of een update van de Liander apparatuur komt.
De data die via de P1 kabel binnenkont laat 2 gasmeters zien. De meter op kanaal 2 heeft de actuele gasmeter stand.

Code: Select all

/ISk5\2MT382-1004

0-0:96.1.1(5A424556303035313631363437383133)
1-0:1.8.1(08340.208*kWh)
1-0:1.8.2(10191.226*kWh)
1-0:2.8.1(00006.283*kWh)
1-0:2.8.2(00029.048*kWh)
0-0:96.14.0(0002)
1-0:1.7.0(0000.14*kW)
1-0:2.7.0(0000.00*kW)
0-0:17.0.0(0999.00*kW)
0-0:96.3.10(1)
0-0:96.13.1()
0-0:96.13.0()
0-1:24.1.0(3)
0-1:96.1.0(3234313537303033303436333030383133)
0-1:24.3.0(171107160000)(00)(60)(1)(0-1:24.2.1)()
(00000000)
0-1:24.4.0(1)
0-2:24.1.0(3)
0-2:96.1.0(3234313537303033303631313932383133)
0-2:24.3.0(171107160000)(00)(60)(1)(0-2:24.2.1)(m3)
(07917.420)
0-2:24.4.0(1)
!

Alle advies / hulp welkom

[mosjonathan]

Looks like your gasmeter changed to channel 2.
In beta there is now a option to set a uservariable to select gasmeter channel.

[Edgar]

Yup, they fucked up (again).

In my case it took a lot of persuasion to the contact person of the Liander helpdesk to accept the problem lays in their system. After the first few conversations Liander was still convinced it was a fault in my device, because they could remote read the values. Even after mailing a good and a wrong P1 data chunk with full explanation of every line!! And accompaning with the full reference of the P1 standard). But luckily the woman I spoke to, was unhappily with the outcome and looked further. She called back after a few days saying they completely understand the problem and they were tryiing to fix it.

I asked how I should have described this problem and possible department to send it to to get it solved the first time. This for future case(s).

So if you are in the same boat and see an extra gasmeter on a extra channel, you can call your grid operator (in my case Liander) and when describing this problem, say that this is for the smart meter division. The people working there know how to solve this. They do it remotely with a minor minor chance that they have to pay a visit when it goes wrong. It's done in a matter of hours.

I had to fight a lot of misunderstanding, misinterpretation and suspision (why can he read these values, where does he get them from, what do those values mean, is our meter hacked / comprosmised, your device is wrong (when actually I don't do anything to the datastream from the smartmeter)) because of the lack of experience with smart meter devices like Toon etc. I guess there are lots of victims of this bad installation practise. Luckily the lady I had contact with, went further and asked other divisions about this when in fact she would not have to pursue any further. To them it was a clear case of a user apparatus malfunctioning.

TL;DR:

you can call your grid operator (in my case Liander) and when describing this problem, say that this is a problem which can be solved by the smart meter division

If they are reluctant or don't know how to solve this, I can give you via private chat my Liander case numbers. But with above information it should be solved by Liander.

[BenAW]

@mosjonathan

In beta there is now a option to set a uservariable to select gasmeter channel.

Thanks. I'll keep this in mind, but for now I try to avoid beta's at my friends setup.

[BenAW]

@edgar

So if you are in the same boat and see an extra gasmeter on a extra channel, you can call your grid operator (in my case Liander) and when describing this problem, say that this is for the smart meter division. The people working there know how to solve this. They do it remotely with a minor minor chance that they have to pay a visit when it goes wrong. It's done in a matter of hours.

Thanks. I'll have a word with Liander and see if they will solve this.
Any idea where this change came from? Up to september 8th the gas meter showed up fine.

[Edgar]

Yes, I watched it happening.

See my messages earlier in this thread. In message #7 I explained how it happened at my site.

Basically: gasmeter got changed, but engineer made mistake (wrong capacity meter). Takes it off just after pairing with electricity smart meter, but doesn’t unpair. Installs correct gasmeter and pairs again. Electricity meter thinks it has two gasmeters connected.

To make things worse, it looks like my gasmeter is the type with the short connection threads. Lately there’re rumors this type can spring a leak on the connection with the residential gas piping. Last saturday an item about this was on dutch television. I think I recognized the same short threads... So maybe I’m in for a new meter again.

[BenAW]

@gordonb3
Thanks for your explanation, also earlier in this thread. You already made it very clear that a second (phantom) gas meter on the M bus created this problem.
My question was about how this second gas meter could pop up out of nowhere. No physical changes to the 2 meters.
Software update for the smartmeter(s) ?

[BlueMoon]

Well, in my case the phantom meter output has always been there. since the initial smart meter installation. As gordonb3 explained, before version 3.8153, domoticz would just accept any value from an external meter, so I had no problems reading the gas values:

Yes. That would be me. As it stands, the original code would accept any value from any remote device attached to the smart meter. While so far these appear to be gas meters exclusively, the DSMR standard clearly states that the interface is meant to support water and heat meters (at some time) as well. The new code features an automatic channel select for the gas meter that will only accept the input if it is a gas meter but can be fooled if the smart meter advertises a phantom gas meter, i.e. one that does not actually exist.

Since version 3.8153 does some more checking on the input, and that's where this issue comes up.
So the expected physical changes could perfectly well have been applied way earlier.

[BenAW]

@BlueMoon

Since version 3.8153 does some more checking on the input, and that's where this issue comes up.
So the expected physical changes could perfectly well have been applied way earlier.

Thanks. I had dismissed 3.8153 as a possible cause since it had been around since july and the gasmeter provided data until september 8th.
Maybe my friend hadn't upgraded Domoticz all that time. That would explain the disappearing of the gasmeter values.

[BenAW]

Installed latest beta and setup the Uservariable for the gasmeter on channel 2.
Gasmeter shows up fine again.
Thanks for all the help.