Zigbee with Python

With my custom made devices I’m using custom data structures and ZigBee is used only as a data transport protocol. However, having more and more standard ZigBee devices available, I started to look for some standard ZigBee features implementation in Python. Unfortunately wasn’t really able to find anything which would fit the purpose – ZigBee feature implementation decoupled from manufacturer specific elements and covering ZDO/ZCL processing/parsing as per ZigBee standard.

There are usually three groups of references coming up in search results:

  • XBEE python package, which provides a tool which handles XBEE api, however has nothing to do with ZigBee as such. I’m already using this library with slight modifications today to handle XBEE api in my application.
  • bellows python package, which gets really close to what I’m looking for, but is very much linked with specific manufacturer device api.
  • custom implementations of specific ZigBee ZCL/ZDO requests for specific devices as part of some project.

So I decided to look a bit more into bellows package. Quite quickly I was able to simulate parsing of ZCL/ZDO messages, but few of the general concepts were not exactly matching my expectations. So to make it work for me either I had to implement EZSP uart protocol in my coordinator device, or rewrite several parts of library which would anyway require quite good understanding of ZDO/ZCL stuff. Unfortunately neither was looking as a quick and simple option.

The only option left was to dig into ZigBee ZDO/ZCL details and create some kind of prototype library by myself to get better understanding what I can get out of standard features, the complexity of implementation, and then decide to have another look on bellows package or continue with own library creation.

Xiaomi Zigbee sensors

Recently I noticed Xiaomi Zigbee sensors being available for quite reasonable price, could not resist and bought few from each of the available types – temperature/humidity sensor, motion sensor, door/window open sensor and button/switch.

All my current devices are custom built and are not using proper ZHA profiles and clusters, as first devices built which are still in use were using XBEE devices, I continued to use the same profile/cluster/endpoint scheme to keep compatiblity between all devices. And it was quite nice surprise that when I tried to link new devices with some Zigbee network, they found my development coordinator and connected to it, and started to send signals, which even reached software but were not processed due to the missing support for standard HA devices.

Sensors are quite nicely made, and to my surprise were using the same NXP JN5168 chips I have chosen for my devices – this should eliminate majority of Zigbee stack incompatibility issues, like disconnects I have been reading about in some forums where these sensors were used with non-Xiaomi gateways.

The only complain I could think about at the moment is missing temperature sensor endpoint on motion sensor – having just one sensor which reports both temperature readings and presence/motion would be really nice feature, as instead on two sensors it would be just one.

XBee speaking with JN5168

After few hours playing with Zigbee sniffer finally managed to get my old XBEE based device using standard API router firmware to communicate with new JN5168 based device and pass the payload data over to the connected PC.

IMG_20150222_150732

Still not sure, but probably will create JN5168 firmware which will emulate XBee API (only functions I use) to keep compatibility with old devices and existing management software, and add new features to support my new custom JN5168 based end devices as well as standard ZigBee devices.

 

 

UI for system control

Some time decided to put an android based tablet on the wall for my house system control purposes. And here is the result:

IMG_9495UI is actually web based, so it looks the same on a tablet and any other browser. Each screen displayed (multiple dashboards are supported) can be fully customized and every element by nature is small widget which can be put in any position/size described in configuration table in database. This is main dashboard for me – it shows temperature outside/inside and forecast for next five days, each button controls appropriate device (lights/heating/gate), and also I included few historical graphs which show temperature sensor readings for 48/72 hours and also heating system on/off graph.

Also I managed to set tablet wake up on motion detected in front camera, which is quite a nice feature, as whenever you pass by device it is ready for use.

 

What changed in last two years

Unfortunately I’ve been quite busy with my work commitments, therefore I wasn’t able to put a lot of effort in my house automation system. Still some things I managed to change…

First big change is the software replacement. The original management software was written in C++ and was running only under MS Windows. Obviously, such implementation required all the time running PC with Windows. So I made decision to change that and rewrote application in Python. As a result it runs on any platform (Windows/Linux) and of course can run on devices like Raspberry Pi, user interface is now web based, and several new features were introduced like ability to use distributed gateways connected via internet and manage all devices as they would be connected in the same home network.

Another change coming is on the device hardware side. While I really like XBee modules, there are many new modules introduced to the market which include not only radio part, but also easy programmable micro controller. Yes, I know there are also programmable XBee modules, but the price is doubled immediately. So I found JN5168 based modules which have all the features I need, less power consumption, less price. While interoperability with standard XBee modules is a challenge here, all other test prototype results seems quite promising. No actual devices created yet, but hope to get some made and deployed in my house by end of year.

Short overview of my House Automation solution

If you are not a technical person, perhaps my previous posts do not provide a clear picture of what is this all about.

We all know that we can switch channels and change volume on a tv without a remote control, however we prefer to use a remote control, as it makes things easier. Even more, remotes get more clever and manage several devices with single click – if you want to watch a tv, it will turn on tv set, receiver, external tv decoder with single click. Systems for house automation are something like a huge and smart universal remote control for almost any electric device at your home. Continue reading

What manages my house today

Since 2009 I have two generations of the devices created, and currently working on third.

What can I do with the system:

  • Devices use ZigBee for communication, so no changes in a current power line wiring needed.
  • Devices can be put behind switches or power sockets (deeper boxes are needed to support an additional device behind a switch mechanics).
  • A single device can control from 4 to 8 inputs, where any input can be analog or digital. Analog inputs are used for analog sensors, like temperature, light or any other analog sensor. Digital inputs are used for switches.
  • Each device can manage from 2 to 5 power lines using relays.
  • Devices do not need a central device and can work independently, so if there is any failure with a central device, end device will still work managing power lines according to configuration stored in a device.
  • Flexible configuration – all inputs can manage the same output and act as paired switch, or one switch can manage several power lines.
  • Custom devices are created for specific purposes, like heating system management device with built in switches or remote inputs, capable to manage 8 power lines.
  • Power lines controled by single device can be extended to virtually any number with custom devices created.
  • The central system is storing and visualising information from devices – state of inputs, values of inputs (for analog sensors), state of outputs.
  • The central system provides an automation capability using any device data to control any other device. For example, having a temperature less than 19 degrees in any room, and if it is saturday and time is between 9AM and 4PM and external light level is less than selected threshold, a heating system can be turned on. Or another example, having an alarm system armed, system will turn of all lights at a house, and when an alarm system is disarmed, it will turn on a light at entrance if an external light level is less than selected threshold.

Continue reading

Main idea = ZigBee + Microcontroller or how it all started at 2009

Idea

The main idea is quite simple – each device should be controllable using PC/smartphone. So there comes requirements:

  • Communication channel between devices – ZigBee features seemed as perfect fit.
  • Some logic processing at the controlling device – microcontroller driven circuit controls, ability to add sensors/switches.
  • Central software to manage all this stuff with/without human help.

Continue reading