X2D decoding attempt
Moderator: leecollings
- ervplecter
- Posts: 18
- Joined: Monday 02 March 2015 9:47
- Target OS: Raspberry Pi / ODroid
- Domoticz version: 2.2284
- Location: Paris, France
- Contact:
X2D decoding attempt
Greetings, first post. Newcomer + Noob to home automation, recently got "corrupted" by a coworker and decided to dive deep.
Aside the regular install of domoticz on a RPi 1, then recently a v2, I was only doing some basic things with domoticz, my (former) main goal being to log data about the house and see how it performs.
Here's what I have done far, as an intro to the actual topic :
Arduino + ethernet shield built as a generic hub for sensors of any kind. I wanted to have things like one wire or DHT22 style sensors to be hooked to an actual microcontroller, then send data to the json web interface of domoticz. Tested with a dallas + DHT22 locally first, then I've deployed several sensors in the house, from the basement to the attic, using the existing telephone line cable. Works like a charm.
My next step was to meter the electricity, asked the local (private) company to change my old meter (rotating disc), hooked up the Teleinfo cable after passing (with a lot of issues) 7 meters of it to the meter which is outside the house, USB serial thingie + opto isolator, done.
My next step was to play with existing HW in the house. The RFXcomm provided everything I needed for my oregon scientific probes, DIO switches and what not. I'll probably control the house ventilation with it very soon, as it's now controlled by a simple calendar static module installed in the electric panel of the house (7 days periodicity, automatic daylight saving, several time slices for each day, but still not versatile enough).
The real challenge comes next. I've first renovated the house with radio "pilot wires" to turn the heaters on and off and select comfort, economy or non freezing (old heaters have 3 orders, new ones have 6 as you know). Logging the house is one thing, but at some point, I'd like also to control the heaters with something more versatile than the provided wireless control unit, which is already doing a great job (7 days day per day, 1/2 hr resolution, long absence, manual control, and 3 zones). Just like the ventilation, I'd like to have a global calendar to decide, to include vacations and stuff like that.
the unit is a Delta Dore Deltia, 3 zones, using the X2D protocol. I've read a lot of things about, just to find that it's not supported by most systems, except one home automation box (pi base ? can't remember). The proprietary nature of the protocol is also mentioned everywhere, along with rolling codes, but I've not seen much of an attempt to at least understand how it works, so I thought I'd tackle that.
This is also going along with another radio hack attempt, with a Cyble water meter, that I haven't seen logged neither, which has in common with X2D to be also in 868 MHz. More to be posted in another topic when time comes, in the water / meter section.
I started by looking at a module that could allow me to both receive and log the frames from my deltia unit
Looking up for 868 MHz module, most of them either :
- are fixed RF channel
- or have a variable channel but have a dedicated packet control system
- provide access to the raw output of the data slicer, BUT require a hell of a parts for the radio section (similar to what you have in a RFXcomm) and no breakout available.
Digging the datasheet of the CC1101 (TI) which is quite popular for having a ton of 433 MHz unit all over ebay, I realized that you can bypass the fifo and export the raw rx data to some of the GPIO (3 of them). However, the 868 MHz version of the module (as a chip breakout) are rare, but I finally found one on aliexpress.
The 868 MHz version is different on the RF section topology to avoid harmonics, so you can't just use a 433 MHz on which you change caps and inductors and antenna.
So I hooked that up to an arduino and started playing. I'm still waiting for my RF Explorer unit that should show up this week, so I didn't have any accurate hints about the modulation used, just found 868.3 MHz referenced in a few documents, so I tuned my CC1101 to that freq (home made helical antena)
I then assumed 2-FSK and started wondering about the deviation, based on the channel occupation and modulation data rate, supposed to be in the range of 1200 Baud (modem side, not over the air side, which was unknown).
I started receiving things, capture by single shot triggering on the scope, and furthermore with my logic analyzer but I couldn't see any good data there.
Not being sure of the modulation type, data rate (which has an impact on the configuration of the receiver), I finally cracked the deltia open to have a look at the RF section, found a melexis chip there (no surprise), a 72035 which has both ASK and FSK modulations.
Looking at the PCB shows that only the ASK pin is used along with the Tx enable pin, so I tapped those lines and hooked it to my logic analyzer, bingo.
(I left the tap interface in there, just in case)
so far, here are the specs of the transport layer of X2D
868.3 MHz (will be confirmed by the RF spectrum analyzer when I get it, along with the channel occupancy)
ASK (or OOK)
5 kbps over the air rate (2.5 kbps usable rate)
manchester encoded, probably the regular one, not the inverted one, no reason for that, it's not 802.3. I've tried bi phase mark recognition, no go, I'm assuming it's straight manchester.
0.2 to 0.3 ms transmission, huge packet (75 bytes)
Several transmissions, 2 or 3, spaced by 700 ms to 1 second (ish). I'm not sure, I've read that some modules might echo what they have received to propagate to neighbors, but that would mean the heater's unit are both emitters and receivers, not sure of that and the age of the tech (X2D) doesn't suggest bi directional exchange over ASK though it's technically possible.
here's a set of log I've made on the Rx side after tuning the CC1101 to the right modulation and data rate (see attached file). For now, I don't get perfect reception each time, it seems that AGC doesn't tune properly to set the slicing threshold at the right place, hence missing a few bytes, but as long as I now know what I need to look for at the beginning, I know when it works or when it fails.
Sync word (or what looks like it, can be called a preamble as well) : 170, 254. No surprise, 170 is 0xAA, which along with 0x55 are the 2 tokens maximizing alternating 0 and 1 and are often used for sync.
I haven't got the time to look further in the packet. I think I can see there several data "sub chucks", the contents also changes from one packet to another, but I first need to log only eco frames then only comfort frames and see what's constant and what's not, then start looking a a possible protocol interpretation.
In order to do that, I've started writing an new arduino sketch that is now using the chip packet handling, so that I can log and print in clear ASCII the packets and start a differential analysis. So far I was exporting the asynchronous data of the slicer and the carrier detection to trigger the capture, now I'd like the chip to provide a fully formated packet that is populated only when the sync word / frame preamble is detected. It's not working totally yet, I'm still fighting with proper FIFO access, which is only 64 byte deep while the message is more like 75.
to be continued. Any ideas or feedback welcome. Apologies if I've posted in the wrong section, please move it if I've made a mistake
if someone has info about the X2D frames format and can highlight it in the provide excel logs, please post your founds !
Aside the regular install of domoticz on a RPi 1, then recently a v2, I was only doing some basic things with domoticz, my (former) main goal being to log data about the house and see how it performs.
Here's what I have done far, as an intro to the actual topic :
Arduino + ethernet shield built as a generic hub for sensors of any kind. I wanted to have things like one wire or DHT22 style sensors to be hooked to an actual microcontroller, then send data to the json web interface of domoticz. Tested with a dallas + DHT22 locally first, then I've deployed several sensors in the house, from the basement to the attic, using the existing telephone line cable. Works like a charm.
My next step was to meter the electricity, asked the local (private) company to change my old meter (rotating disc), hooked up the Teleinfo cable after passing (with a lot of issues) 7 meters of it to the meter which is outside the house, USB serial thingie + opto isolator, done.
My next step was to play with existing HW in the house. The RFXcomm provided everything I needed for my oregon scientific probes, DIO switches and what not. I'll probably control the house ventilation with it very soon, as it's now controlled by a simple calendar static module installed in the electric panel of the house (7 days periodicity, automatic daylight saving, several time slices for each day, but still not versatile enough).
The real challenge comes next. I've first renovated the house with radio "pilot wires" to turn the heaters on and off and select comfort, economy or non freezing (old heaters have 3 orders, new ones have 6 as you know). Logging the house is one thing, but at some point, I'd like also to control the heaters with something more versatile than the provided wireless control unit, which is already doing a great job (7 days day per day, 1/2 hr resolution, long absence, manual control, and 3 zones). Just like the ventilation, I'd like to have a global calendar to decide, to include vacations and stuff like that.
the unit is a Delta Dore Deltia, 3 zones, using the X2D protocol. I've read a lot of things about, just to find that it's not supported by most systems, except one home automation box (pi base ? can't remember). The proprietary nature of the protocol is also mentioned everywhere, along with rolling codes, but I've not seen much of an attempt to at least understand how it works, so I thought I'd tackle that.
This is also going along with another radio hack attempt, with a Cyble water meter, that I haven't seen logged neither, which has in common with X2D to be also in 868 MHz. More to be posted in another topic when time comes, in the water / meter section.
I started by looking at a module that could allow me to both receive and log the frames from my deltia unit
Looking up for 868 MHz module, most of them either :
- are fixed RF channel
- or have a variable channel but have a dedicated packet control system
- provide access to the raw output of the data slicer, BUT require a hell of a parts for the radio section (similar to what you have in a RFXcomm) and no breakout available.
Digging the datasheet of the CC1101 (TI) which is quite popular for having a ton of 433 MHz unit all over ebay, I realized that you can bypass the fifo and export the raw rx data to some of the GPIO (3 of them). However, the 868 MHz version of the module (as a chip breakout) are rare, but I finally found one on aliexpress.
The 868 MHz version is different on the RF section topology to avoid harmonics, so you can't just use a 433 MHz on which you change caps and inductors and antenna.
So I hooked that up to an arduino and started playing. I'm still waiting for my RF Explorer unit that should show up this week, so I didn't have any accurate hints about the modulation used, just found 868.3 MHz referenced in a few documents, so I tuned my CC1101 to that freq (home made helical antena)
I then assumed 2-FSK and started wondering about the deviation, based on the channel occupation and modulation data rate, supposed to be in the range of 1200 Baud (modem side, not over the air side, which was unknown).
I started receiving things, capture by single shot triggering on the scope, and furthermore with my logic analyzer but I couldn't see any good data there.
Not being sure of the modulation type, data rate (which has an impact on the configuration of the receiver), I finally cracked the deltia open to have a look at the RF section, found a melexis chip there (no surprise), a 72035 which has both ASK and FSK modulations.
Looking at the PCB shows that only the ASK pin is used along with the Tx enable pin, so I tapped those lines and hooked it to my logic analyzer, bingo.
(I left the tap interface in there, just in case)
so far, here are the specs of the transport layer of X2D
868.3 MHz (will be confirmed by the RF spectrum analyzer when I get it, along with the channel occupancy)
ASK (or OOK)
5 kbps over the air rate (2.5 kbps usable rate)
manchester encoded, probably the regular one, not the inverted one, no reason for that, it's not 802.3. I've tried bi phase mark recognition, no go, I'm assuming it's straight manchester.
0.2 to 0.3 ms transmission, huge packet (75 bytes)
Several transmissions, 2 or 3, spaced by 700 ms to 1 second (ish). I'm not sure, I've read that some modules might echo what they have received to propagate to neighbors, but that would mean the heater's unit are both emitters and receivers, not sure of that and the age of the tech (X2D) doesn't suggest bi directional exchange over ASK though it's technically possible.
here's a set of log I've made on the Rx side after tuning the CC1101 to the right modulation and data rate (see attached file). For now, I don't get perfect reception each time, it seems that AGC doesn't tune properly to set the slicing threshold at the right place, hence missing a few bytes, but as long as I now know what I need to look for at the beginning, I know when it works or when it fails.
Sync word (or what looks like it, can be called a preamble as well) : 170, 254. No surprise, 170 is 0xAA, which along with 0x55 are the 2 tokens maximizing alternating 0 and 1 and are often used for sync.
I haven't got the time to look further in the packet. I think I can see there several data "sub chucks", the contents also changes from one packet to another, but I first need to log only eco frames then only comfort frames and see what's constant and what's not, then start looking a a possible protocol interpretation.
In order to do that, I've started writing an new arduino sketch that is now using the chip packet handling, so that I can log and print in clear ASCII the packets and start a differential analysis. So far I was exporting the asynchronous data of the slicer and the carrier detection to trigger the capture, now I'd like the chip to provide a fully formated packet that is populated only when the sync word / frame preamble is detected. It's not working totally yet, I'm still fighting with proper FIFO access, which is only 64 byte deep while the message is more like 75.
to be continued. Any ideas or feedback welcome. Apologies if I've posted in the wrong section, please move it if I've made a mistake
if someone has info about the X2D frames format and can highlight it in the provide excel logs, please post your founds !
Rpi v2 + Domoticz
Re: X2D decoding attempt
Hi !
Keep the good work ! Were you able to get any improvement since your first post ? This is (also) my first post on this forum, I found this thread by googling around ...
I was desperate to see somebody trying to reverse engineer this protocol used in Deltia Dore products (mostly across France & Europe).
I have the same base station as the one in your picture, and 16 receivers for my home heaters, and I am dreaming about automation of my heating system (more advanced than the basic features provided by their base controller - already have a RPI 2 for light, videosurveillance & Electric Meter monitoring - ie. Teleinfo).
FYI, the only 3rd party product where X2D was implemented was the Zibase (and I think as part of a partnership with Deltia Dore ...). I have not tried to dig around the zibase source code to see if we can get some more infos ...
Thanks to share any progress !
Bon courage,
Guillaume.
Keep the good work ! Were you able to get any improvement since your first post ? This is (also) my first post on this forum, I found this thread by googling around ...
I was desperate to see somebody trying to reverse engineer this protocol used in Deltia Dore products (mostly across France & Europe).
I have the same base station as the one in your picture, and 16 receivers for my home heaters, and I am dreaming about automation of my heating system (more advanced than the basic features provided by their base controller - already have a RPI 2 for light, videosurveillance & Electric Meter monitoring - ie. Teleinfo).
FYI, the only 3rd party product where X2D was implemented was the Zibase (and I think as part of a partnership with Deltia Dore ...). I have not tried to dig around the zibase source code to see if we can get some more infos ...
Thanks to share any progress !
Bon courage,
Guillaume.
-
- Posts: 9
- Joined: Tuesday 16 June 2015 14:15
- Target OS: Linux
- Domoticz version:
- Contact:
Re: X2D decoding attempt
I personally use the CUL V3 usb key to decode DeltaDore frames (http://busware.de/tiki-index.php?page=CUL)
I am able to decode the device ID and that's all ...
I use biphase decoding.
Here is part of my python code (start at Decode with the pulse width as input)
In short:
1- first detect a minimum of 7 long pulses
2- detect 12 short pulses
3- start decoding the frame with biphase policy
- 2 successive short pulse = 1 (when you receive a short pulse you have to wait for the next pulse in order to decide)
- 1 long pulse = 0
class X2DDecoder(DecodeOOK):
#Deltadore use biphase mark coding
#see patent EP 1160752 B1
#http://www.google.com/patents/EP1160752B1?cl=fr
ENDFRAME=0xFE
FRAME_STATE_UNKNOWN="unknown"
FRAME_STATE_FRAME_END_FOUND="ended"
def __init__(self):
DecodeOOK.__init__(self)
self.Reset()
self.syncword_detector = 0x00
self.oneCounter=0
def Restart(self):
self.pos = 0
self.data = []
self.data.append(0)
self.state = DecodeState.T1
self.syncword_detector = 0x00
self.doaverage = False
self.flip = 1
self.oneCounter=0
def Reset(self):
DecodeOOK.Reset(self)
self.syncword_detector=0x00
self.longpulse_average = 0
self.shortpulse_average = 0
self.longmax = 460
self.longmin = 307
self.shortmax = 250
self.shortmin = 134
self.nbFrames = 0
self.doaverage = True
self.oneCounter=0
def TryInterpret(self):
ret = 0
self.message = None
if (self.total_bits == 0):
nbbits=self.data[0]
if nbbits>=20:
ret = 1
#Champ Adresse Maison 2 octets 0 à FFFFh
#Champ Adresse Source 1 octet 0 à FFh
#Champ Adresse Destinataire 1 octet 0 à FFh
#Champ Adresse Transmetteur 1 octet 0 à FFh
#Champ de contrôle 1 octet
#Champ d'information N octet(s) N compris entre 0 et 8
#Champ de vérification 2 octets
id = (self.data[3]&0x07)<<16 | self.data[2]<<8 | self.data[1]
self.message = DecodedMessage(vendorName='X2D',productId=id,deviceId=0)
if nbbits>=48:
data = "%01x"%((self.data[3]&0xf8))
data = data + "%02x%02x%02x"%(self.data[4],self.data[5],self.data[6])
self.message.append(DecodedInfo(type=MessageType.UNKNOWN,val=data))
return ret
def Decode (self,width):
#if self.state == DecodeState.T0 or self.state == DecodeState.OK:
#print "at %d have 0x%x" % (self.pos,self.data[self.pos])
interpret_res = 0;
if (self.shortmin <= width and width < self.shortmax) or (self.longmin <= width and width < self.longmax):
longpulse = (self.longmin <= width and width < self.longmax)
if self.state == DecodeState.UNKNOWN:
#first detect a minimum of 7 long pulses
# __ __ __
# __| |__| |__| |__
if (longpulse == 1):
if (self.flip == 0):
self.longpulse_average = width
else:
self.longpulse_average = ((self.longpulse_average*3)+width)/4
# long pulse
self.flip += 1
#print 'have unknown long pulse ' + str(self.flip)
elif (7 <= self.flip):
#if we have at leat 7 long pulse and then a short pulse, start receiving short pulses
# __ __ __ _
#__| |__| |__| |__| |
#print 'have unknown short pulse '
# Long pulse, start 12 short
self.flip = 1
self.state = DecodeState.T1
self.shortpulse_average = width
else:
#Reset decoder
#print 'reset unknown decoder'
return -1
elif self.state == DecodeState.T1:
#now count short pulse
if (longpulse == 0):
# short pulse
self.flip += 1
if self.doaverage == True:
self.shortpulse_average = ((self.shortpulse_average*3)+width)/4
#print 'have T1 short pulse ' + str(self.flip)
elif (12 <= self.flip):
#once we had 12 short pulses, a X2D sync pattern is here
RfLogger.debug('Have X2D start bit at ' + str(datetime.datetime.now()))
RfLogger.debug("long=%d short=%d" % ( self.longpulse_average, self.shortpulse_average ))
if self.doaverage == True:
self.longmax = self.longpulse_average * 1.25
self.longmin = self.longpulse_average * 0.75
self.shortmax = self.shortpulse_average * 1.28
self.shortmin = self.shortpulse_average * 0.72
self.flip = 1
self.state = DecodeState.OK
else:
#Reset decoder
#print 'reset unknown d
return -1
elif self.state == DecodeState.OK:
#decode thanks to Biphase mark coding
#2 successive short pulse = 1
#1 long pulse = 0
#now decode bits
if (longpulse == 0):
# Short pulse
self.state = DecodeState.T0
else:
# Long pulse
#print 'X2D add 1 ' + str(width)
self.GotBit(0)
interpret_res = self.TryInterpret()
elif self.state == DecodeState.T0:
if (longpulse == 0):
#print 'X2D add 0 ' + str(width)
# Second short pulse
self.GotBit(1)
interpret_res = self.TryInterpret()
else:
RfLogger.debug('X2D reset ' + str(width))
# Reset decoder
return -1
else:
if self.state == DecodeState.T0 or self.state == DecodeState.OK or self.state == DecodeState.OK:
RfLogger.debug('reset ' + str(self.state) + ' / ' + str(width))
return -1
return interpret_res
def syncWordDetector(self,value):
self.syncword_detector = value | (self.syncword_detector << 1)
self.syncword_detector &= 0xFF
#RfLogger.debug("sync: %d %02x"%(value,self.syncword_detector))
if self.syncword_detector == X2DDecoder.ENDFRAME:
return X2DDecoder.FRAME_STATE_FRAME_END_FOUND
return X2DDecoder.FRAME_STATE_UNKNOWN
def GotBit(self,value):
if self.oneCounter==5:
RfLogger.debug("remove a 0 bit after 5 successive one")
self.state = DecodeState.OK
self.oneCounter=0
return
if self.syncWordDetector(value) == X2DDecoder.FRAME_STATE_FRAME_END_FOUND:
self.nbFrames += 1
self.total_bits -= 15 #remove 16 bits, since the last one is not pushed remove 15 bits
self.data.insert(0,self.total_bits)
self.total_bits = 0
self.bits = 0
self.pos = 0
RfLogger.debug('End Frame %d'%(self.nbFrames))
else:
self.total_bits += 1
olddata = self.data[self.pos]
#self.data[self.pos] = value | (olddata << 1)
self.data[self.pos] = value<<7 | (olddata >> 1)
#RfLogger.debug("data: %d %02x"%(value,self.data[self.pos]))
self.bits += 1
if (self.bits >= 8):
self.bits = 0
self.data.append(0)
self.pos += 1
RfLogger.debug("new byte from X2D @ %d: 0x%02x" % ((self.pos - 1),self.data[self.pos - 1]))
self.state = DecodeState.OK
if value==1:
self.oneCounter+=1
else:
self.oneCounter=0
def Done(self):
self.state = DecodeState.DONE
I am able to decode the device ID and that's all ...
I use biphase decoding.
Here is part of my python code (start at Decode with the pulse width as input)
In short:
1- first detect a minimum of 7 long pulses
2- detect 12 short pulses
3- start decoding the frame with biphase policy
- 2 successive short pulse = 1 (when you receive a short pulse you have to wait for the next pulse in order to decide)
- 1 long pulse = 0
class X2DDecoder(DecodeOOK):
#Deltadore use biphase mark coding
#see patent EP 1160752 B1
#http://www.google.com/patents/EP1160752B1?cl=fr
ENDFRAME=0xFE
FRAME_STATE_UNKNOWN="unknown"
FRAME_STATE_FRAME_END_FOUND="ended"
def __init__(self):
DecodeOOK.__init__(self)
self.Reset()
self.syncword_detector = 0x00
self.oneCounter=0
def Restart(self):
self.pos = 0
self.data = []
self.data.append(0)
self.state = DecodeState.T1
self.syncword_detector = 0x00
self.doaverage = False
self.flip = 1
self.oneCounter=0
def Reset(self):
DecodeOOK.Reset(self)
self.syncword_detector=0x00
self.longpulse_average = 0
self.shortpulse_average = 0
self.longmax = 460
self.longmin = 307
self.shortmax = 250
self.shortmin = 134
self.nbFrames = 0
self.doaverage = True
self.oneCounter=0
def TryInterpret(self):
ret = 0
self.message = None
if (self.total_bits == 0):
nbbits=self.data[0]
if nbbits>=20:
ret = 1
#Champ Adresse Maison 2 octets 0 à FFFFh
#Champ Adresse Source 1 octet 0 à FFh
#Champ Adresse Destinataire 1 octet 0 à FFh
#Champ Adresse Transmetteur 1 octet 0 à FFh
#Champ de contrôle 1 octet
#Champ d'information N octet(s) N compris entre 0 et 8
#Champ de vérification 2 octets
id = (self.data[3]&0x07)<<16 | self.data[2]<<8 | self.data[1]
self.message = DecodedMessage(vendorName='X2D',productId=id,deviceId=0)
if nbbits>=48:
data = "%01x"%((self.data[3]&0xf8))
data = data + "%02x%02x%02x"%(self.data[4],self.data[5],self.data[6])
self.message.append(DecodedInfo(type=MessageType.UNKNOWN,val=data))
return ret
def Decode (self,width):
#if self.state == DecodeState.T0 or self.state == DecodeState.OK:
#print "at %d have 0x%x" % (self.pos,self.data[self.pos])
interpret_res = 0;
if (self.shortmin <= width and width < self.shortmax) or (self.longmin <= width and width < self.longmax):
longpulse = (self.longmin <= width and width < self.longmax)
if self.state == DecodeState.UNKNOWN:
#first detect a minimum of 7 long pulses
# __ __ __
# __| |__| |__| |__
if (longpulse == 1):
if (self.flip == 0):
self.longpulse_average = width
else:
self.longpulse_average = ((self.longpulse_average*3)+width)/4
# long pulse
self.flip += 1
#print 'have unknown long pulse ' + str(self.flip)
elif (7 <= self.flip):
#if we have at leat 7 long pulse and then a short pulse, start receiving short pulses
# __ __ __ _
#__| |__| |__| |__| |
#print 'have unknown short pulse '
# Long pulse, start 12 short
self.flip = 1
self.state = DecodeState.T1
self.shortpulse_average = width
else:
#Reset decoder
#print 'reset unknown decoder'
return -1
elif self.state == DecodeState.T1:
#now count short pulse
if (longpulse == 0):
# short pulse
self.flip += 1
if self.doaverage == True:
self.shortpulse_average = ((self.shortpulse_average*3)+width)/4
#print 'have T1 short pulse ' + str(self.flip)
elif (12 <= self.flip):
#once we had 12 short pulses, a X2D sync pattern is here
RfLogger.debug('Have X2D start bit at ' + str(datetime.datetime.now()))
RfLogger.debug("long=%d short=%d" % ( self.longpulse_average, self.shortpulse_average ))
if self.doaverage == True:
self.longmax = self.longpulse_average * 1.25
self.longmin = self.longpulse_average * 0.75
self.shortmax = self.shortpulse_average * 1.28
self.shortmin = self.shortpulse_average * 0.72
self.flip = 1
self.state = DecodeState.OK
else:
#Reset decoder
#print 'reset unknown d
return -1
elif self.state == DecodeState.OK:
#decode thanks to Biphase mark coding
#2 successive short pulse = 1
#1 long pulse = 0
#now decode bits
if (longpulse == 0):
# Short pulse
self.state = DecodeState.T0
else:
# Long pulse
#print 'X2D add 1 ' + str(width)
self.GotBit(0)
interpret_res = self.TryInterpret()
elif self.state == DecodeState.T0:
if (longpulse == 0):
#print 'X2D add 0 ' + str(width)
# Second short pulse
self.GotBit(1)
interpret_res = self.TryInterpret()
else:
RfLogger.debug('X2D reset ' + str(width))
# Reset decoder
return -1
else:
if self.state == DecodeState.T0 or self.state == DecodeState.OK or self.state == DecodeState.OK:
RfLogger.debug('reset ' + str(self.state) + ' / ' + str(width))
return -1
return interpret_res
def syncWordDetector(self,value):
self.syncword_detector = value | (self.syncword_detector << 1)
self.syncword_detector &= 0xFF
#RfLogger.debug("sync: %d %02x"%(value,self.syncword_detector))
if self.syncword_detector == X2DDecoder.ENDFRAME:
return X2DDecoder.FRAME_STATE_FRAME_END_FOUND
return X2DDecoder.FRAME_STATE_UNKNOWN
def GotBit(self,value):
if self.oneCounter==5:
RfLogger.debug("remove a 0 bit after 5 successive one")
self.state = DecodeState.OK
self.oneCounter=0
return
if self.syncWordDetector(value) == X2DDecoder.FRAME_STATE_FRAME_END_FOUND:
self.nbFrames += 1
self.total_bits -= 15 #remove 16 bits, since the last one is not pushed remove 15 bits
self.data.insert(0,self.total_bits)
self.total_bits = 0
self.bits = 0
self.pos = 0
RfLogger.debug('End Frame %d'%(self.nbFrames))
else:
self.total_bits += 1
olddata = self.data[self.pos]
#self.data[self.pos] = value | (olddata << 1)
self.data[self.pos] = value<<7 | (olddata >> 1)
#RfLogger.debug("data: %d %02x"%(value,self.data[self.pos]))
self.bits += 1
if (self.bits >= 8):
self.bits = 0
self.data.append(0)
self.pos += 1
RfLogger.debug("new byte from X2D @ %d: 0x%02x" % ((self.pos - 1),self.data[self.pos - 1]))
self.state = DecodeState.OK
if value==1:
self.oneCounter+=1
else:
self.oneCounter=0
def Done(self):
self.state = DecodeState.DONE
Last edited by poloalexis on Tuesday 16 June 2015 14:34, edited 2 times in total.
Re: X2D decoding attempt
Hi everybody,
Sorry for my english (I am french )
About the protocol named X2D, I have some information.
I use the USB key (emulate a port COM) with delta dore reference : Tydom360.
This key was sold by the company Myxyty (an other french company) with in reference : MyDomokit.
About this protocol, all french companies signed a special commitment to limit the sending information.
In the attach file, you can see the package with source code to use test this key. I tested it an it's work fine.
I search some people to create a new package with all parameters :
#Champ Adresse Maison 2 octets 0 à FFFFh => House adress
#Champ Adresse Source 1 octet 0 à FFh => Source adress
#Champ Adresse Destinataire 1 octet 0 à FFh => Receiver adress
#Champ Adresse Transmetteur 1 octet 0 à FFh => Sender adress
and to manage a lot of devices.
Best regards
Jimmy
Sorry for my english (I am french )
About the protocol named X2D, I have some information.
I use the USB key (emulate a port COM) with delta dore reference : Tydom360.
This key was sold by the company Myxyty (an other french company) with in reference : MyDomokit.
About this protocol, all french companies signed a special commitment to limit the sending information.
In the attach file, you can see the package with source code to use test this key. I tested it an it's work fine.
I search some people to create a new package with all parameters :
#Champ Adresse Maison 2 octets 0 à FFFFh => House adress
#Champ Adresse Source 1 octet 0 à FFh => Source adress
#Champ Adresse Destinataire 1 octet 0 à FFh => Receiver adress
#Champ Adresse Transmetteur 1 octet 0 à FFh => Sender adress
and to manage a lot of devices.
Best regards
Jimmy
- Attachments
-
- X2dLib v0.4.zip
- (189.4 KiB) Downloaded 370 times
Re: X2D decoding attempt
Thanks Jimmy,
I had a fast look to the SDK, and decompiled the JAR.
With this in the pocket, it won't be an issue to get the format of the messages that are sent to the USB Key (and quite all message type can be generated by the SDK - ie. Heating Control, On/Off ...). The SDK seems to directly generate binary messages, not human readable serial message to the USB KEY.
The low level 868Mhz encoding seems to have been reversed engineering by poloalexis (biphase mark coding).
The last thing to do would be to understand if the USB KEY is just acting as a passthrough for the binary message received from the API (= USB KEY only doing biphase encoding and 868Mhz transmitting) ... Unfortunately, if the KEY is generating a new message based on the API request, it won't help a lot ...
Just my 0.02$, unfortunately, I have limited knowledge on radio stuff and no test gears, so it will be hard for me to help more ...
Guillaume.
I had a fast look to the SDK, and decompiled the JAR.
With this in the pocket, it won't be an issue to get the format of the messages that are sent to the USB Key (and quite all message type can be generated by the SDK - ie. Heating Control, On/Off ...). The SDK seems to directly generate binary messages, not human readable serial message to the USB KEY.
Code: Select all
public static final FunctioningMode Reduced = new FunctioningMode(0, "Reduced", 0);
public static final FunctioningMode Moderato = new FunctioningMode(1, "Moderato", 1);
public static final FunctioningMode Medio = new FunctioningMode(2, "Medio", 2);
public static final FunctioningMode Comfort = new FunctioningMode(3, "Comfort", 3);
public static final FunctioningMode Stop = new FunctioningMode(4, "Stop", 4);
public static final FunctioningMode AntiFrost = new FunctioningMode(5, "AntiFrost", 5);
public static final FunctioningMode Special = new FunctioningMode(6, "Special", 6);
public static final FunctioningMode Auto = new FunctioningMode(7, "Auto", 7);
public static final FunctioningMode Centralized = new FunctioningMode(8, "Centralized", 8);
The last thing to do would be to understand if the USB KEY is just acting as a passthrough for the binary message received from the API (= USB KEY only doing biphase encoding and 868Mhz transmitting) ... Unfortunately, if the KEY is generating a new message based on the API request, it won't help a lot ...
Just my 0.02$, unfortunately, I have limited knowledge on radio stuff and no test gears, so it will be hard for me to help more ...
Guillaume.
Re: X2D decoding attempt
Hi Guillaume,
The USB X2D key has a same principe as the RS232 modem.
When I plug my USB X2D key, my computer recognize it as the COM device.
I send all message in the COM port and I receive the same type of message coming from emulate COM port.
I also attach the picture of this USB device.
Jimmy
The USB X2D key has a same principe as the RS232 modem.
When I plug my USB X2D key, my computer recognize it as the COM device.
I send all message in the COM port and I receive the same type of message coming from emulate COM port.
I also attach the picture of this USB device.
Jimmy
- Attachments
-
- In box device
- DSC_0041.JPG (202.83 KiB) Viewed 16622 times
-
- device
- DSC_0040.JPG (168.32 KiB) Viewed 16622 times
Re: X2D decoding attempt
Thx Jimmy,
Based on my initial look, the USB transmitter does not (unfortunately) seems to be a "dumb" 868Mhz encoder/transmitter.
It looks like the API only sends the message portion of the frame to the transmitter (and the transmitter then adds all the headers, like Source, Destination, control bytes ...).
And as such, no indication on how the control bytes are generated ...
So unfortunately, nothing more helpfull than what Poloalexis & Ervplecter already did by reverse engineering the Heater Controler ...
And even more annoying, the API does not seems to support the "3 zones pilot wire heater adapters" (which was the initial request of this thread), so no indication on the message format for those devices.
G.
Based on my initial look, the USB transmitter does not (unfortunately) seems to be a "dumb" 868Mhz encoder/transmitter.
It looks like the API only sends the message portion of the frame to the transmitter (and the transmitter then adds all the headers, like Source, Destination, control bytes ...).
And as such, no indication on how the control bytes are generated ...
So unfortunately, nothing more helpfull than what Poloalexis & Ervplecter already did by reverse engineering the Heater Controler ...
And even more annoying, the API does not seems to support the "3 zones pilot wire heater adapters" (which was the initial request of this thread), so no indication on the message format for those devices.
G.
Re: X2D decoding attempt
Sorry, I sent the package when I found in the internet.
But I started the modification of this initiale package to update the house code or other information.
I saw in the initiale code, the house code is 1 and the other specific code are initiliazed with value 0.
In my first version of this package rebuild, all information (house adress, transmetter adress, ...) are sent to the USB code and modifyed by me.
Regards
Jimmy
But I started the modification of this initiale package to update the house code or other information.
I saw in the initiale code, the house code is 1 and the other specific code are initiliazed with value 0.
In my first version of this package rebuild, all information (house adress, transmetter adress, ...) are sent to the USB code and modifyed by me.
Regards
Jimmy
Re: X2D decoding attempt
Great Jimmy !
Hum, so I may have missed something in the API, as you seems to confirm that the whole frame is generated (including house code ...), not only the message portion I didn't find the class where the contol bits and house code are generated in the API, but I may have looked badly ...
Based on you initial research, do you confirm that the API generated frames sent to the USP key are consistent with :
#Champ Adresse Maison 2 octets 0 à FFFFh
#Champ Adresse Source 1 octet 0 à FFh
#Champ Adresse Destinataire 1 octet 0 à FFh
#Champ Adresse Transmetteur 1 octet 0 à FFh
#Champ de contrôle 1 octet
#Champ d'information N octet(s) N compris entre 0 et 8
#Champ de vérification 2 octets
This is v0.4 of the API ... There may be an updated version with extended device support (including the missing pilot wire controller ...)
Do not hesitate to share your modified code with us !
G.
Hum, so I may have missed something in the API, as you seems to confirm that the whole frame is generated (including house code ...), not only the message portion I didn't find the class where the contol bits and house code are generated in the API, but I may have looked badly ...
Based on you initial research, do you confirm that the API generated frames sent to the USP key are consistent with :
#Champ Adresse Maison 2 octets 0 à FFFFh
#Champ Adresse Source 1 octet 0 à FFh
#Champ Adresse Destinataire 1 octet 0 à FFh
#Champ Adresse Transmetteur 1 octet 0 à FFh
#Champ de contrôle 1 octet
#Champ d'information N octet(s) N compris entre 0 et 8
#Champ de vérification 2 octets
This is v0.4 of the API ... There may be an updated version with extended device support (including the missing pilot wire controller ...)
Do not hesitate to share your modified code with us !
G.
Re: X2D decoding attempt
Lo, Nice topic.
I have decompiled X2D java library too.
This could help a bit to understand X2D protocol, but it will be far more difficult than decoding low cost temperature/humidity weather station external sensors.
If someone interested, I have decoded Otio SHT-10 temperature/humidity sensor and an Astrell (now probably Innnovalley) temperature sensor.
https://github.com/SixK/Otio_SHT-10_protocol
I have to find an 868Mhz receiver/transmitter to play with X2D.
SixK
I have decompiled X2D java library too.
This could help a bit to understand X2D protocol, but it will be far more difficult than decoding low cost temperature/humidity weather station external sensors.
If someone interested, I have decoded Otio SHT-10 temperature/humidity sensor and an Astrell (now probably Innnovalley) temperature sensor.
https://github.com/SixK/Otio_SHT-10_protocol
I have to find an 868Mhz receiver/transmitter to play with X2D.
SixK
Re: X2D decoding attempt
A Callgraph to try to understand x2d library
i.class seem's to be important.
stove management part : SixK
i.class seem's to be important.
stove management part : SixK
Re: X2D decoding attempt
If I have a look at f.class, I can see this :
arrby[6] = -104;
Then I'm wondering, does X2D protocol have signed or unsigned messages?
SixK
arrby[6] = -104;
Then I'm wondering, does X2D protocol have signed or unsigned messages?
SixK
Last edited by SixK on Monday 23 November 2015 20:55, edited 1 time in total.
-
- Posts: 784
- Joined: Wednesday 10 December 2014 13:06
- Target OS: Linux
- Domoticz version: beta
- Location: Bordeaux France
- Contact:
Re: X2D decoding attempt
Guys, i'm sure RFLink / Stunteam can Help you ! RFLink = rfx like but home made and open source.
Please visit here :
viewforum.php?f=49
and
http://www.nemcon.nl/blog2/
Deltadore va vous detester...
Please visit here :
viewforum.php?f=49
and
http://www.nemcon.nl/blog2/
Deltadore va vous detester...
Domoticz stable 3.5877 for real & Domoticz beta for test
Rfxtrxe / RFLink / Milight / Yeelight / Tasmota / MQTT / BLE / Zigate
http://domo-attitude.fr
Rfxtrxe / RFLink / Milight / Yeelight / Tasmota / MQTT / BLE / Zigate
http://domo-attitude.fr
Re: X2D decoding attempt
Ok, found the following document giving some parts of information on X2D :
http://perso.telecom-paristech.fr/~chol ... mes-V2.doc
Confirming some informations already posted here (sorry document is in french) :
http://perso.telecom-paristech.fr/~chol ... mes-V2.doc
Confirming some informations already posted here (sorry document is in french) :
3.3.4 Un exemple de protocole de courant porteur : le protocole X2D
La société DELTA DORE (Combourg, 35) étudie et fabrique du matériel dans le domaine de la thermique du bâtiment, de l'automobile, de l'électroménager, et des télécommunications. Cette société a acquis aujourd'hui une position de leader dans le domaine des transmissions qui peuvent utiliser les supports filaires, radio ou courant porteur et faire appel à divers protocoles (Batibus, X2D, EIB, EHS... .). Mais l’offre principale de Delta Dore s’appuie sur leur protocole propriétaire dont on trouvera ci-après quelques caractéristiques.
Les modems RF et CPL sont reliés à la centrale par une liaison du type RS232 (1200 bauds, 8 bits de données, 1 bit de start, 1 bit de stop, pas de parité).
Le modem RF permet de recevoir les trames émises par le programmateur selon le protocole X2D. A la réception d’une trame conforme au protocole X2D, le modem RF en délivre une version simplifiée à la centrale.
Inversement, lorsque le modem CPL reçoit de la centrale une trame conforme à la version simplifiée du protocole X2D, celui-ci l’émet sur le réseau électrique.
Les paragraphes suivants décrivent la structure générale d’une trame X2D simplifié avant de détailler le contenu des différents types de trame utiles à l’application.
Structure générale d’une trame X2D simplifiée
Une trame est constituée d'une succession de champs logiques, chacun étant composé d’un ou de plusieurs octets. La structure est la suivante :
Champ Adresse Maison 2 octets 0 à FFFFh
Champ Adresse Source 1 octet 0 à FFh
Champ Adresse Destinataire 1 octet 0 à FFh
Champ Adresse Transmetteur 1 octet 0 à FFh
Champ de contrôle 1 octet
Champ d'information N octet(s) N compris entre 0 et 8
Champ de vérification 2 octets
Seul le champ d’information est optionnel. Sa présence dépend du type de la trame.
La longueur d’une trame est donc comprise entre 8 et 16 octets. Les données codées sur plusieurs octets sont transférées dans le sens poids fort d’abord.
Le champ "Adresse Maison"
L'adresse maison, distribuée par l’élément émetteur à tous les éléments destinataires lors de la phase de mise en service, empêche les interférences entre deux installations voisines.
Celle-ci n’est pas prise en compte si elle est égale à FFFFh. On dispose donc de 65535 adresses de maison différentes.
Généralement, l’adresse maison est fixée aléatoirement en usine dans chaque émetteur. Le processus de configuration mis en œuvre à l’installation du système permet la distribution de cette adresse maison à tous les destinataires d’un même logement.
Les champs "Adresse Source" et "Adresse Destinataire"
Une adresse individuelle est attribuée à chaque élément. Cette adresse permet d'identifier l’émetteur et le destinataire d'une trame.
Les adresses F0h à FEh sont réservées à l'adressage par famille. L’adresse FFh signifie que tous les éléments ayant reçus la trame sont destinataires (s’ils ont la même adresse maison ou si celle figurant dans la trame est égale à FFFFh). On est donc capable d’adresser individuellement 240 destinations différentes.
Généralement, les émetteurs ont une adresse individuelle fixée arbitrairement en usine (00). Par contre l’adresse individuelle des récepteurs leur est distribuée en même temps que l’adresse maison pendant le processus de configuration.
Le champ "Adresse Transmetteur"est utile dans le cas ou la trame émise par la source transite par un ou plusieurs répéteurs avant d’atteindre le destinataire. L’adresse transmetteur identifie l’élément émetteur, qui est donc l’élément source au départ de la trame et qui change ensuite à chaque ré-émission de celle-ci par un répéteur. Ce champ permet donc à un répéteur de savoir quand il doit réémettre une trame. Dans les installations sans répéteurs, l’adresse transmetteur sera toujours égal à l’adresse source.
Le champ de contrôle permet identifier l’action à effectuer à la réception d’une trame et le contenu du champ d’information quand celui-ci est présent. Il contient également des informations en rapport avec la nature de la communication (unidirectionnelle avec ou sans répétition, bidirectionnelle avec demande de réponse ou d’acquittement).
Re: X2D decoding attempt
I've started to have a look at delta dore patents.
We may find usefull information in the following document (It's not X2D itself) :
French version :
http://www.google.ga/patents/EP1160752A1?cl=fr
English Version :
http://www.google.ga/patents/EP1160752A1?cl=en
SixK
We may find usefull information in the following document (It's not X2D itself) :
French version :
http://www.google.ga/patents/EP1160752A1?cl=fr
English Version :
http://www.google.ga/patents/EP1160752A1?cl=en
SixK
Re: X2D decoding attempt
Here is hardware I will probably buy :
RFBee
http://www.seeedstudio.com/depot/RFbee- ... p-614.html
Default firmware can handle 868.299Mhz frequency that is not that far from Theorical X2D protocol frequency (868.35 Mhz), but even if it's not really explained in RFbee documentation, it seem's that RFbee frequency is fully configurable recompiling it's own custom firmware. (configuration parameters can be found using SmartRftm TI tool).
Why trying to have the exact X2D frequency ? Slave device may check the exact frequency and reject message that are not on right frequency.
That means, we will be able to trace messages, but any message sent by RFbee may be ignored.
RFbee seem's to use CC1101 SDR RF chip.
That means it also could catch RF433 Mhz messages if correctly configured
UartSBee
http://www.seeedstudio.com/depot/UartSB ... -1752.html
Small card to acces RFbee from PC.
Should let me tweak RFBee firmware and play with various tools that should help to understand X2D
DFRobot Leonardo with Xbee socket :
http://www.dfrobot.com/index.php?route= ... lylstKrSHs
I don't really like Leonardo arduino board, since it has some problems with Serial initialisation/handling, but with XBee socket integrated, it should be way cleaner and cheaper to handle RFbee.
[UPDATED] probably Useless till RFbee firmware is not modified to understand X2D protocol.
Here is TI tool to reconfigure CC1101 chip (this may help to configure RFbee module to the exact X2D frequency) :
http://www.ti.com/tool/smartrftm-studio ... rfstudio#1
Here are 2 tools that may help to decode X2D protocol (I try to see how this tools can help) :
RF Explorer :
http://j3.rf-explorer.com/downloads
Packet Sniffer :
http://www.ti.com/tool/packet-sniffer?D ... ketsniffer
SixK
RFBee
http://www.seeedstudio.com/depot/RFbee- ... p-614.html
Default firmware can handle 868.299Mhz frequency that is not that far from Theorical X2D protocol frequency (868.35 Mhz), but even if it's not really explained in RFbee documentation, it seem's that RFbee frequency is fully configurable recompiling it's own custom firmware. (configuration parameters can be found using SmartRftm TI tool).
Why trying to have the exact X2D frequency ? Slave device may check the exact frequency and reject message that are not on right frequency.
That means, we will be able to trace messages, but any message sent by RFbee may be ignored.
RFbee seem's to use CC1101 SDR RF chip.
That means it also could catch RF433 Mhz messages if correctly configured
UartSBee
http://www.seeedstudio.com/depot/UartSB ... -1752.html
Small card to acces RFbee from PC.
Should let me tweak RFBee firmware and play with various tools that should help to understand X2D
DFRobot Leonardo with Xbee socket :
http://www.dfrobot.com/index.php?route= ... lylstKrSHs
I don't really like Leonardo arduino board, since it has some problems with Serial initialisation/handling, but with XBee socket integrated, it should be way cleaner and cheaper to handle RFbee.
[UPDATED] probably Useless till RFbee firmware is not modified to understand X2D protocol.
Here is TI tool to reconfigure CC1101 chip (this may help to configure RFbee module to the exact X2D frequency) :
http://www.ti.com/tool/smartrftm-studio ... rfstudio#1
Here are 2 tools that may help to decode X2D protocol (I try to see how this tools can help) :
RF Explorer :
http://j3.rf-explorer.com/downloads
Packet Sniffer :
http://www.ti.com/tool/packet-sniffer?D ... ketsniffer
SixK
Last edited by SixK on Saturday 05 December 2015 0:14, edited 2 times in total.
Re: X2D decoding attempt
Hello everybody
I've tried to understand X2D a few month ago but I failed. I've just stumbled upon this thread and it encouraged me to do further investigations.
I have also decompiled the X2D library and I've started to analyse it, to make it more human readable (I've renamed some variables and classes) and I made some changes to make it compilable so I can use a Java debugger.
At a first glance it seems that the library could manage the whole content of the packets. Unfortunatly when I try to inject the trames from jimmy2cv, the interpretation doesn't work at all.
Does any of you have some other trames so I can make other tests ?
Of course I'll share my finds with you.
Olivier
PS. Sorry for my bad english.
I've tried to understand X2D a few month ago but I failed. I've just stumbled upon this thread and it encouraged me to do further investigations.
I have also decompiled the X2D library and I've started to analyse it, to make it more human readable (I've renamed some variables and classes) and I made some changes to make it compilable so I can use a Java debugger.
At a first glance it seems that the library could manage the whole content of the packets. Unfortunatly when I try to inject the trames from jimmy2cv, the interpretation doesn't work at all.
Does any of you have some other trames so I can make other tests ?
Of course I'll share my finds with you.
Olivier
PS. Sorry for my bad english.
Re: X2D decoding attempt
Great !
I should get my Arduino + RFBee tomorrow or later.
I don't know how long it will take me to catch X2D messages, but then I should be able to provide you some packets.
Which format X2D library expect for messages ? (Binary, Hexa, Decimal, ascii ... ?)
SixK
I should get my Arduino + RFBee tomorrow or later.
I don't know how long it will take me to catch X2D messages, but then I should be able to provide you some packets.
Which format X2D library expect for messages ? (Binary, Hexa, Decimal, ascii ... ?)
SixK
Re: X2D decoding attempt
Hi SixK
The lib takes bytes as input. You can give me the représentation you want (binary/hexa/decimal).
Olivier
The lib takes bytes as input. You can give me the représentation you want (binary/hexa/decimal).
Olivier
Re: X2D decoding attempt
Well, Leonardo&Xbee board (DFR0221) was not exactly the best choice.
In fact I need something like UartSBee to set RFbee with custom parameters and exploit CC1101A chip directly with some various tools.
It will not be a big problem right now, I need to understand how to catch X2D signal with RFBee.
Now I can send AT commands to RFbee, it's a good start.
[Updated] Leonardo&RFbee DFRobot board was definitly not the good choice, I may be able to use it later if I succeed in modifying RFBee module firmware to work with X2D protocol, but otherwise it probably can only talk to another RFbee...
SixK
In fact I need something like UartSBee to set RFbee with custom parameters and exploit CC1101A chip directly with some various tools.
It will not be a big problem right now, I need to understand how to catch X2D signal with RFBee.
Now I can send AT commands to RFbee, it's a good start.
[Updated] Leonardo&RFbee DFRobot board was definitly not the good choice, I may be able to use it later if I succeed in modifying RFBee module firmware to work with X2D protocol, but otherwise it probably can only talk to another RFbee...
SixK
Who is online
Users browsing this forum: No registered users and 1 guest