Switching out my home automation messaging infrastructure for MQTT

by Mike Linnen 1. June 2012 21:09

I am re-vamping my home automation strategy from a home grown publish/subscribe messaging system to use MQTT instead.  I was using Azure Service Bus to connect remote devices such as my phone with devices in my home such as my lawn irrigation system.  This worked well as a messaging infrastructure for remote devices but I wanted to have a more standard messaging infrastructure that could work in my home network without connectivity to the outside world. 

A few reasons why I switched to MQTT:

  • Light weight
  • Many Clients already exist for many platforms and languages
  • Support for on-premise message broker
  • Support for off-premise message broker
  • Support for bridging brokers (on-premise to off-premise)
  • Fast
  • Used by companies like COSM (was Pachube) and Github
  • Simple topic subscription model that is also powerful
  • I don’t want to write a message broker

For the most part I am moving toward having devices in the home that are relatively dumb and having services running on a home server that add the smarts behind the devices.  This will give me the flexibility to change the behavior of the system a lot quicker without the hassle of tearing apart a device to upgrade the software on it.  This means I needed to have my services available all the time.  Placing these services in the cloud for mission critical things would mean I am left with devices in the home that cannot function while my internet connectivity is down.  This was the biggest reason I moved to an off the self pub/sub infrastructure like MQTT.

Like most messaging protocols, MQTT works on the notion of a topic and a message.  The topic is just a unique way of addressing a message.  I struggled a lot and I probably will continue to struggle on what my topic structure for my home automation should look like.  One thing I wanted to do is try to make the topics readable so that troubleshooting message problems would be easier.  Hear are a few standards I am trying to settle on:

  • When a device or service changes state and wishes to notify interested parties the topic will end with /event 
  • When a device or service wants to know the status of another device or service the topic will end with /getstatus
  • When a device or service receives a topic /getstatus the response topic it generates will end with /status
  • When a device or service needs to set the state of another device or service the topic will end with /set

Here are a few examples of topics and messages for my irrigation system:

Description Topic Message
Zone 1 turned on irrigation/zone/event z1 On
Request the current schedule from the irrigation service. The service will respond to this request by publishing various status topics irrigation/schedule/getstatus  
Set the time that the irrigation service should start watering irrigation/schedule/starttime/set 09:00 AM
Status of the schedule start time in response to the getstatus request irrigation/schedule/starttime/status 09:00 AM
Set the days of the week that the irrigation system will water irrigation/schedule/days/set MON WED FRI
Status of the scheduled days of the week in response to the getstatus request irrigation/schedule/days/status MON WED FRI
Set the zones that the irrigation system will run and how long irrigation/schedule/zones/set z1 10 z2 8 z3 10
Status of the scheduled zones in response to the getstatus request irrigation/schedule/zones/status z1 10 z2 8 z3 10
Sets the zones to run on the irrigation device and how long irrigation/zones/run z1 10 z2 8 z3 10

MQTT does have a concept of publishing messages with a retain bit. This just tells the broker to hang onto the last message for a topic and when a new subscription arrives the client will receive the last message.  I could have used this concept instead of the /getstatus standard that I have show above.  I might change over to using the retain bit but for now the /getstatus works for me. I am also making my messages a little verbose as they tend to contain multiple values that could have been broken down into more granular topics.

Overall I really like how simple MQTT is and it is very easy to get a device like the Netduino to understand MQTT messages.  I am sure I will make modifications on how I actually define my topics and message body over time as I develop more and more devices and services that do useful stuf in my home.    

Presenting “Getting Started with Microsoft Robotics Developer Studio 4 and the Kinect”

by Mike Linnen 6. May 2012 14:16

I am excited about presenting on this topic for the Charlotte Alt.Net users group on May 8th in Charlotte. Head on over to the event posting and sign up to attend.

Here are the details about the talk:

The most recent release of Microsoft Robotics Developer Studio 4 (RDS4) has introduced two very exciting  concepts that make building robotic applications a reality to all developers: Kinect and Reference Platform Design specification.  The Kinect is the hot device that gives a new perspective on sensing your surroundings.  RDS 4 fully supports the Kinect and opens up all kinds of opportunities for awesome applications.  Do you want skeletal tracking in a robotics application, RDS 4 gives you that.  Do you want to perform obstacle avoidance with Kinect's depth sensor, RDS 4 gives you that. Do you want to simulate a Kinect in a virtual environment  to test out your high level code, RDS 4 gives you that.  The Reference Platform gives vendors a common design specification for building a working robot that includes sensors, motors and low level control. This allows for a developer that has little hardware experience to get up and running fast.  In this session I will introduced you to RDS 4 using the Kinect and an Eddie robot.

Eddie Robot http://www.parallax.com/eddie

Microsoft Robotics Developer Studio http://www.microsoft.com/robotics/

Lawn Sprinkler the Data Flow Part 3

by Mike Linnen 4. September 2011 16:57

This is part 3 of a multipart blog series that shows how commands that control my sprinkler flow through the infrastructure and reach the final destination of the Netduino Plus.  This video covers what components are used in the system to connect a Windows Phone 7 to the Netduino based Lawn Sprinkler as well as how a Weather Service can send data to the Netduino.  So connecting devices to devices and services to devices all using the Azure AppFabric Service Bus.

Here is the video:

Lawn Sprinkler the Demo Part 2

by Mike Linnen 21. August 2011 18:34

As mentioned in a previous post I am building a Home Automation project that consists of replacing my Lawn Sprinkler.  This is part 2 of the blog series but if you want to look at other posts related to this then here are some links to them as well:

Here is a video that demonstrates how the Lawn Sprinkler system works.

The code for the project is located on bitbucket at https://bitbucket.org/mlinnen/lawnsprinkler.

I will be doing a presentation on this project at the Charlotte Alt.Net meeting on August 24th 2011

The power point slides for the presentation is also posted in the docs folder of the source repository https://bitbucket.org/mlinnen/lawnsprinkler/src/f5cd6cda9501/Docs/.

Lawn Sprinkler the Introduction Part 1

by Mike Linnen 2. July 2011 09:00


The new craze for Home Automation is to use technology to Go Green.  One aspect of Going Green is about managing resources in a more efficient way.  I have seen a number of other hobbyists build projects that manage the amount of electricity or gas that they use within their home.  In this project I am going to manage the amount of water I use for watering my lawn.  In part 1 of this series I am going to cover the big picture of what I am attempting to do.

Since this is a multipart post I am including the links to the other parts here as well:


Of course I needed a few requirements to define the scope of what I am attempting to do.

  • Support for up to 4 zones
  • Be able to manually turn on 1 or more zones (max 4) and have them run for a period of time
  • Be able to schedule 1 or more zones (max 4) to come on daily at a specific time of the day multiple times a day.
  • Be able to schedule 1 or more zones (max 4) to come on every Mon, Wed and Friday at a specific time of the day multiple times a day.
  • Be able to schedule 1 or more zones (max 4) to come on every Tuesday and Thursday at a specific time of the day multiple times a day.
  • Be able to turn off the system so that the scheduled or manual zones will immediately turn off or not turn on at their scheduled time.
  • Be able to do any of the above requirements remotely.
  • Do not turn on the sprinkler if rain is in the forecast (Go Green)
  • Do not turn on the sprinkler if the ground is already moist enough (Go Green)
  • Be able to automatically set the clock when daylight savings time changes.

At first I was going to make the sprinkler system a completely stand alone device where I could setup the schedule by using a keypad and an LCD.  This would allow me to completely control the device without remotely connecting to it.  But since I wanted to control the device remotely anyway and the cost of hardware and development efforts would be higher for a stand alone device, I decided to abandon the “Stand Alone” capabilities.  I did want the ability to turn off the sprinkler system without remotely connecting to it and I also wanted a quick way to know if the device was off or not.  A push button switch can be used to turn the sprinkler immediately off.  A couple LEDs can be used to let you know what mode the sprinkler is in.

The Sprinkler

I am using a Netduino Plus as the microcontroller that operates my sprinkler heads.  I choose this device because it uses the .Net Micro framework and it also has an onboard Ethernet controller which makes connecting it to my network a real easy task.  You could very easily use another device to control the sprinklers as long as it could handle the HTTP messages and had enough I/O to interface to the rest of the needed hardware. 

This device is responsible for the following:

  • Monitor the schedule and turn on the sprinklers if it is time to do so
    • 4 Digital Outputs
    • Onboard clock to know when to run the scheduled time
  • Watch for HTTP JSON requests that originate from the Windows Phone
    • The onboard Etherent works well for this
  • Watch for HTTP JSON requests that originate from the weather service telling the sprinkler the chance of rain
    • The onboard Etherent works well for this
  • Watch for HTTP JSON requests that originate from the time service telling the sprinkler to change it’s onboard clock
    • The onboard Etherent works well for this
  • On power up ask the time service for the correct time
    • The onboard Etherent works well for this
  • Monitor the Off pushbutton and cycle the mode of the sprinkler through the 3 states: Off/Manual/Scheduled
    • 1 Digital Input
  • Yellow LED goes on when in the Manual state
    • 1 Digital Output
  • Green LED goes on when in the Schedule state
    • 1 Digital Output
  • Monitor the ground moisture (Note: I haven’t done much research on how these sensors work so this might change)
    • 1 Analog Input
  • Persist the Manual and Scheduled programs so that a power cycle wont these values

The sprinkler modes need a little more discussion.  When in the Off mode the sprinkler heads will not turn on but the board will be powered up and listen for any HTTP requests and monitor the push button.  When cycling to the Off mode from any other mode the sprinklers will turn off if they where on.  When cycled to the Manual mode from any other mode the sprinkler will immediately run the manual schedule turning on the appropriate zones for the appropriate length of time.  If no Manual schedule exists then the sprinkler does nothing. In Scheduled mode the sprinkler waits for the programmed day and time to turn on the appropriate zones for the appropriate length of time unless the ground is already wet or rain is in the forecast.

The Remote Control

The remote control is the only way to program the sprinkler since it doesn’t have any UI for this task.  There can be many different devices that serve as the remote control but I intend to use my Samsung Focus Windows Phone 7 for this purpose. 

The application on this device just needs to send HTTP Get and Post requests.  Depending on the type of request a JSON message might be required in the body of the request (i.e. sending data to the sprinkler).  Also depending on the type of request the response may contain JSON(i.e. returning data from the sprinkler).

I chose to use HTTP and JSON as the communication mechanism between the remote control and the sprinkler so that I could remain platform independent.       

Connecting the Remote to the Sprinkler

The Netduino sprinkler sits behind my home firewall.  If I want to talk to the sprinkler with a device that is not behind the firewall then things start to get a little painful.  I would basically have the following options:

  • Don’t expose the sprinkler to the outside world (kind of limiting).
  • The sprinkler microcontroller would have to poll some server on the internet for any new messages that it should process (lots of busy work for the controller).
  • Punch a hole in my firewall so I can get through it from the internet (can you please hack me).
  • Use Windows Azure Service Bus(no brainer).

The Service Bus allows me to make outbound connections to Windows Azure cloud infrastructure and it keeps that connection open so that any external device can make remote procedure calls to the endpoint behind the firewall. I have decided to use the v 1.0 release of service bus for now, but in the future I could see this changing where I would use more of a publish/subscribe messaging infrastructure (which is in a future release of service bus) rather than a remote procedure call.

To leverage the Service Bus you must have a Host that sits behind the firewall and makes the connection to the Azure cloud platform.  For the purpose of this post I am calling this service the Home Connector.  The responsibility of this service is to connect to the Service Bus as a host so that it can accept remote procedure calls from a client.  The client in this case I call the remote connector.

The Home Connector

The Home Connector is a windows service that runs on one of my windows machines behind my firewall.  When a Remote Procedure Call comes in it is converted to an HTTP Get or Post JSON request that is sent to the Netdunio sprinkler.  The response from the Netduino is then parsed and returned back to the RPC caller.  This routing of Service Bus messages to devices behind my firewall is built with the mindset that more than one Netduino microcontroller will be servicing RPC calls from a remote device over the internet.  So this architecture is not limited to just the Sprinkler System.  I intend to add more microcontrollers in the same manor and register them with the home connector so that they too can service RPC requests. 

The Remote Connector

I could have skipped this layer between the phone and the sprinkler.  Since the phone would not be able to use the Service Bus DLL’s directly I could have used the Service Bus WebHttpRelayBinding which would allow me to submit messages to the bus over a REST style api directly from the phone.  But I wanted another layer between the Phone and the Sprinkler so that I could cache some of the requests to prevent my sprinkler from getting bombarded with messages.  I needed a lightweight web framework that would make creating HTTP Get/Post JSON messages easy.

I choose to use the NancyFX framework because it seemed to fit the bill of being quick and easy to get up and running.  That sure was the case when I pulled it down and started building out the first HTTP Get handler.  I simply created an empty web site and used nugetto install NancyFX into this existing blank site.  After that I created a module class and defined my routes and handlers for the routes and I was running with my first Get request in about 15 minutes.  The NancyFX framework also handled processing my JSON messages with very little effort on my part.  All I really needed to do is have a model that represented the JSON message and performed a bind operation on it and the model ended up fully populated.  I haven’t tried to play around with caching the responses yet but I don’t think that will be too hard.

It is important to understand that this remote connector does not have to be on an Azure web role to work.  I could easily deploy this web site to another hosting provider that might be a little cheaper to use.


The Netduino, Service Bus and NanacyFX web framework all seemed to be pretty easy to get me going on connecting devices in my home to my phone.  At the time of this post I haven’t finished the sprinkler system but I got an end to end example of using the Windows Phone to control my Netduino behind my firewall without punching any holes in my router.  I spent most of my time working out the JSON parsing issues across multiple devices then actually getting the infrastructure in place.

This opens up a whole new world of possibilities for me of connecting multiple home devices to my phone and other services.  Before I go to a multiple device household I will most likely move away from the RPC calls and introduce a more publish/subscribe model of passing messages around.  That way I can decouple the message producers from the message consumers.  I will probably wait for the newer Azure Service Bus bits before I tackle that problem though. 

One thing that I started to think about while doing this project is how much smarts (code) should I be placing in the Netduino device.  Right now I have a considerable amount of code that performs all the scheduling functionality in the Netduino.  So once the Netduino receives its pre-programmed schedule it basically can run without any other communications from the outside world (as long as the power doesn’t cycle).  However the scheduling functionality that is built into my sprinkler code is kind of limiting.  If I wanted to add more features to the scheduling functionality it would require me to build a lot of the logic into the Netduino sprinkler code.  This also means I need to deploy more bits to my sprinkler device.  As you can imaging this could develop into a deployment nightmare if a lot of customers are using this product.  There are ways to solve that kind of deployment issues by automating the update process but another solution is to remove the scheduling smarts from the sprinkler device itself and place that logic into a cloud service.  Basically the sprinkler device would know nothing about a schedule and it would be told when it should turn on and how long the zones should run for.  This would eliminate a lot of code that is on the device and make it easier to add new features to the service.  Of course that means the sprinkler device has to be connected to the internet at all times in order to work but that’s doable.  Well I don’t intend to move in that direction yet but I think once I finish out the original design I will explore building out a Home Automation as a Service (HAAS) model.

Keep a watch on my blog for the future posts where I will be diving deeper into each layer of the system and showing some code.  Also I will be posting the source code to the project at some point for others to see.

About the author

Mike Linnen

Software Engineer specializing in Microsoft Technologies

Month List