X2D decoding attempt
Posted: Monday 02 March 2015 11:19
Greetings, first post. Newcomer + Noob to home automation, recently got "corrupted" by a coworker and decided to dive deep.
Aside the regular install of domoticz on a RPi 1, then recently a v2, I was only doing some basic things with domoticz, my (former) main goal being to log data about the house and see how it performs.
Here's what I have done far, as an intro to the actual topic :
Arduino + ethernet shield built as a generic hub for sensors of any kind. I wanted to have things like one wire or DHT22 style sensors to be hooked to an actual microcontroller, then send data to the json web interface of domoticz. Tested with a dallas + DHT22 locally first, then I've deployed several sensors in the house, from the basement to the attic, using the existing telephone line cable. Works like a charm.
My next step was to meter the electricity, asked the local (private) company to change my old meter (rotating disc), hooked up the Teleinfo cable after passing (with a lot of issues) 7 meters of it to the meter which is outside the house, USB serial thingie + opto isolator, done.
My next step was to play with existing HW in the house. The RFXcomm provided everything I needed for my oregon scientific probes, DIO switches and what not. I'll probably control the house ventilation with it very soon, as it's now controlled by a simple calendar static module installed in the electric panel of the house (7 days periodicity, automatic daylight saving, several time slices for each day, but still not versatile enough).
The real challenge comes next. I've first renovated the house with radio "pilot wires" to turn the heaters on and off and select comfort, economy or non freezing (old heaters have 3 orders, new ones have 6 as you know). Logging the house is one thing, but at some point, I'd like also to control the heaters with something more versatile than the provided wireless control unit, which is already doing a great job (7 days day per day, 1/2 hr resolution, long absence, manual control, and 3 zones). Just like the ventilation, I'd like to have a global calendar to decide, to include vacations and stuff like that.
the unit is a Delta Dore Deltia, 3 zones, using the X2D protocol. I've read a lot of things about, just to find that it's not supported by most systems, except one home automation box (pi base ? can't remember). The proprietary nature of the protocol is also mentioned everywhere, along with rolling codes, but I've not seen much of an attempt to at least understand how it works, so I thought I'd tackle that.
This is also going along with another radio hack attempt, with a Cyble water meter, that I haven't seen logged neither, which has in common with X2D to be also in 868 MHz. More to be posted in another topic when time comes, in the water / meter section.
I started by looking at a module that could allow me to both receive and log the frames from my deltia unit
Looking up for 868 MHz module, most of them either :
- are fixed RF channel
- or have a variable channel but have a dedicated packet control system
- provide access to the raw output of the data slicer, BUT require a hell of a parts for the radio section (similar to what you have in a RFXcomm) and no breakout available.
Digging the datasheet of the CC1101 (TI) which is quite popular for having a ton of 433 MHz unit all over ebay, I realized that you can bypass the fifo and export the raw rx data to some of the GPIO (3 of them). However, the 868 MHz version of the module (as a chip breakout) are rare, but I finally found one on aliexpress.
The 868 MHz version is different on the RF section topology to avoid harmonics, so you can't just use a 433 MHz on which you change caps and inductors and antenna.
So I hooked that up to an arduino and started playing. I'm still waiting for my RF Explorer unit that should show up this week, so I didn't have any accurate hints about the modulation used, just found 868.3 MHz referenced in a few documents, so I tuned my CC1101 to that freq (home made helical antena)
I then assumed 2-FSK and started wondering about the deviation, based on the channel occupation and modulation data rate, supposed to be in the range of 1200 Baud (modem side, not over the air side, which was unknown).
I started receiving things, capture by single shot triggering on the scope, and furthermore with my logic analyzer but I couldn't see any good data there.
Not being sure of the modulation type, data rate (which has an impact on the configuration of the receiver), I finally cracked the deltia open to have a look at the RF section, found a melexis chip there (no surprise), a 72035 which has both ASK and FSK modulations.
Looking at the PCB shows that only the ASK pin is used along with the Tx enable pin, so I tapped those lines and hooked it to my logic analyzer, bingo.
(I left the tap interface in there, just in case)
so far, here are the specs of the transport layer of X2D
868.3 MHz (will be confirmed by the RF spectrum analyzer when I get it, along with the channel occupancy)
ASK (or OOK)
5 kbps over the air rate (2.5 kbps usable rate)
manchester encoded, probably the regular one, not the inverted one, no reason for that, it's not 802.3. I've tried bi phase mark recognition, no go, I'm assuming it's straight manchester.
0.2 to 0.3 ms transmission, huge packet (75 bytes)
Several transmissions, 2 or 3, spaced by 700 ms to 1 second (ish). I'm not sure, I've read that some modules might echo what they have received to propagate to neighbors, but that would mean the heater's unit are both emitters and receivers, not sure of that and the age of the tech (X2D) doesn't suggest bi directional exchange over ASK though it's technically possible.
here's a set of log I've made on the Rx side after tuning the CC1101 to the right modulation and data rate (see attached file). For now, I don't get perfect reception each time, it seems that AGC doesn't tune properly to set the slicing threshold at the right place, hence missing a few bytes, but as long as I now know what I need to look for at the beginning, I know when it works or when it fails.
Sync word (or what looks like it, can be called a preamble as well) : 170, 254. No surprise, 170 is 0xAA, which along with 0x55 are the 2 tokens maximizing alternating 0 and 1 and are often used for sync.
I haven't got the time to look further in the packet. I think I can see there several data "sub chucks", the contents also changes from one packet to another, but I first need to log only eco frames then only comfort frames and see what's constant and what's not, then start looking a a possible protocol interpretation.
In order to do that, I've started writing an new arduino sketch that is now using the chip packet handling, so that I can log and print in clear ASCII the packets and start a differential analysis. So far I was exporting the asynchronous data of the slicer and the carrier detection to trigger the capture, now I'd like the chip to provide a fully formated packet that is populated only when the sync word / frame preamble is detected. It's not working totally yet, I'm still fighting with proper FIFO access, which is only 64 byte deep while the message is more like 75.
to be continued. Any ideas or feedback welcome. Apologies if I've posted in the wrong section, please move it if I've made a mistake
if someone has info about the X2D frames format and can highlight it in the provide excel logs, please post your founds !
Aside the regular install of domoticz on a RPi 1, then recently a v2, I was only doing some basic things with domoticz, my (former) main goal being to log data about the house and see how it performs.
Here's what I have done far, as an intro to the actual topic :
Arduino + ethernet shield built as a generic hub for sensors of any kind. I wanted to have things like one wire or DHT22 style sensors to be hooked to an actual microcontroller, then send data to the json web interface of domoticz. Tested with a dallas + DHT22 locally first, then I've deployed several sensors in the house, from the basement to the attic, using the existing telephone line cable. Works like a charm.
My next step was to meter the electricity, asked the local (private) company to change my old meter (rotating disc), hooked up the Teleinfo cable after passing (with a lot of issues) 7 meters of it to the meter which is outside the house, USB serial thingie + opto isolator, done.
My next step was to play with existing HW in the house. The RFXcomm provided everything I needed for my oregon scientific probes, DIO switches and what not. I'll probably control the house ventilation with it very soon, as it's now controlled by a simple calendar static module installed in the electric panel of the house (7 days periodicity, automatic daylight saving, several time slices for each day, but still not versatile enough).
The real challenge comes next. I've first renovated the house with radio "pilot wires" to turn the heaters on and off and select comfort, economy or non freezing (old heaters have 3 orders, new ones have 6 as you know). Logging the house is one thing, but at some point, I'd like also to control the heaters with something more versatile than the provided wireless control unit, which is already doing a great job (7 days day per day, 1/2 hr resolution, long absence, manual control, and 3 zones). Just like the ventilation, I'd like to have a global calendar to decide, to include vacations and stuff like that.
the unit is a Delta Dore Deltia, 3 zones, using the X2D protocol. I've read a lot of things about, just to find that it's not supported by most systems, except one home automation box (pi base ? can't remember). The proprietary nature of the protocol is also mentioned everywhere, along with rolling codes, but I've not seen much of an attempt to at least understand how it works, so I thought I'd tackle that.
This is also going along with another radio hack attempt, with a Cyble water meter, that I haven't seen logged neither, which has in common with X2D to be also in 868 MHz. More to be posted in another topic when time comes, in the water / meter section.
I started by looking at a module that could allow me to both receive and log the frames from my deltia unit
Looking up for 868 MHz module, most of them either :
- are fixed RF channel
- or have a variable channel but have a dedicated packet control system
- provide access to the raw output of the data slicer, BUT require a hell of a parts for the radio section (similar to what you have in a RFXcomm) and no breakout available.
Digging the datasheet of the CC1101 (TI) which is quite popular for having a ton of 433 MHz unit all over ebay, I realized that you can bypass the fifo and export the raw rx data to some of the GPIO (3 of them). However, the 868 MHz version of the module (as a chip breakout) are rare, but I finally found one on aliexpress.
The 868 MHz version is different on the RF section topology to avoid harmonics, so you can't just use a 433 MHz on which you change caps and inductors and antenna.
So I hooked that up to an arduino and started playing. I'm still waiting for my RF Explorer unit that should show up this week, so I didn't have any accurate hints about the modulation used, just found 868.3 MHz referenced in a few documents, so I tuned my CC1101 to that freq (home made helical antena)
I then assumed 2-FSK and started wondering about the deviation, based on the channel occupation and modulation data rate, supposed to be in the range of 1200 Baud (modem side, not over the air side, which was unknown).
I started receiving things, capture by single shot triggering on the scope, and furthermore with my logic analyzer but I couldn't see any good data there.
Not being sure of the modulation type, data rate (which has an impact on the configuration of the receiver), I finally cracked the deltia open to have a look at the RF section, found a melexis chip there (no surprise), a 72035 which has both ASK and FSK modulations.
Looking at the PCB shows that only the ASK pin is used along with the Tx enable pin, so I tapped those lines and hooked it to my logic analyzer, bingo.
(I left the tap interface in there, just in case)
so far, here are the specs of the transport layer of X2D
868.3 MHz (will be confirmed by the RF spectrum analyzer when I get it, along with the channel occupancy)
ASK (or OOK)
5 kbps over the air rate (2.5 kbps usable rate)
manchester encoded, probably the regular one, not the inverted one, no reason for that, it's not 802.3. I've tried bi phase mark recognition, no go, I'm assuming it's straight manchester.
0.2 to 0.3 ms transmission, huge packet (75 bytes)
Several transmissions, 2 or 3, spaced by 700 ms to 1 second (ish). I'm not sure, I've read that some modules might echo what they have received to propagate to neighbors, but that would mean the heater's unit are both emitters and receivers, not sure of that and the age of the tech (X2D) doesn't suggest bi directional exchange over ASK though it's technically possible.
here's a set of log I've made on the Rx side after tuning the CC1101 to the right modulation and data rate (see attached file). For now, I don't get perfect reception each time, it seems that AGC doesn't tune properly to set the slicing threshold at the right place, hence missing a few bytes, but as long as I now know what I need to look for at the beginning, I know when it works or when it fails.
Sync word (or what looks like it, can be called a preamble as well) : 170, 254. No surprise, 170 is 0xAA, which along with 0x55 are the 2 tokens maximizing alternating 0 and 1 and are often used for sync.
I haven't got the time to look further in the packet. I think I can see there several data "sub chucks", the contents also changes from one packet to another, but I first need to log only eco frames then only comfort frames and see what's constant and what's not, then start looking a a possible protocol interpretation.
In order to do that, I've started writing an new arduino sketch that is now using the chip packet handling, so that I can log and print in clear ASCII the packets and start a differential analysis. So far I was exporting the asynchronous data of the slicer and the carrier detection to trigger the capture, now I'd like the chip to provide a fully formated packet that is populated only when the sync word / frame preamble is detected. It's not working totally yet, I'm still fighting with proper FIFO access, which is only 64 byte deep while the message is more like 75.
to be continued. Any ideas or feedback welcome. Apologies if I've posted in the wrong section, please move it if I've made a mistake
if someone has info about the X2D frames format and can highlight it in the provide excel logs, please post your founds !