The Netduino forums have been replaced by new forums at community.wildernesslabs.co.
This site has been preserved for archival purposes only
and the ability to make new accounts or posts has been turned off.
I have just bought my first house and am about to re-plumb and re-wire the whole thing (obivously not all myself but will have some professionals working with me and check what I do).
I want to go for a full home automation system and for me to write the software interface to control it and see if the netduino can manage as the hardware controller.
I have been reading around and there seems to be so many different protocals and products I simply don't know which way to go and so wondered if you can give me some advice.
I plan to start re-wiring in the next fortnight after I have stripped all the lath and plaster out of the basement, so could do with starting to order bits asap...
Perhaps if I say what I want to do you could suggest what products are available that I could work with netduino and can all be integrated together to make some form of .Net web application.
Like everything I am doing currently I want to aim for a SOA architecture, so my web app will talk to my netduino with some wep api's and messages will be sent back to my web app using json from the netduino.
In terms of control I want to think big picture and robust, so infrared control in my opinion is not the best, I would much prefer a communication protocal that has some form of handshake or response etc to gurantee messages are sent and received.
I want to zone the entire house for electrics and heating on a room by room basis, so controlling plug sockets, light switches (although I prefer to keep these to a minimum or have them as signal controllers only), radiator valves, burglar alarm, door contacts, pir sensors, locks, thermostats, audio, video, lighting.
So the idea is that on my application I can drill in to any room and control everything in that room and set automation of various tasks etc.
As I am dealing with mains electrics and want to control appliances and plug sockets, I want a solution that safe and professional, not my own relay enclosures.
I have seen some x10 din rail products and some lightwave sockets but I'm not sure on the way to go and again on the quality of these various protocals.
Are there any commercial control products that can be integrated together and interfaced with via your own software and most importantly that I can get the netduino talking with?
Start looking at X10, LightWave, ZWave products. Look into HomeSeer Automation Software. It's written in .NET and can be expanded. A PC will run Homeseer and host the web interface. You can also control homeseer from iPads, iPhones, iPods, Android devices. Since you can design your own drivers for custom devices for Homeseer, the Netduinos would be these custom devices (kind of worker bees) that handle some specific tasks.
I really want to get a netduino to communication via the serial modem of the Insteon devices. I think I saw somewhere some code for doing it, but I can't remember where.
Is there any preference, pros/cons between the various wireless signals zigbee, lightwaverf, z-wave, x10 etc?
I have a big old house (120 years old) so the walls are thick, cellings high and their are 4 levels including a basement.
My heating system and electrics are in the cellar (which is pretty dry), I'm not sure how to go about wiring everything, node zero or zoned?
Am I best having control signal cables going to devices (sockets, switches, lights, etc) to a room controller or node zero setup. Alternatively do I have the control (relay, dimmers, etc) in the node zero or room controller?
Would you recomend a centralised controllers (make, model?)?
Can you recommend any products in-line, din rail, wireless/wired products encompasing the control (switches, light fittings, sockets, etc)?
I will give some input on what I am planning on doing. The key word there is "planning" as I don't have a lot of this completed yet. First off I intend to have a network of Netdunios throughout my house doing various tasks. They will be connected via Ethernet and MQTT message protocol (http://mqtt.org). I have this working currently for my irrigation system and a garage door controller. I have my house hardwired with Ethernet to every room.
For controlling lights I have some x10 devices but I intend to go with the Insteon (http://www.insteon.net/) devices. They sell an LED bulb (http://www.insteon.net/bulb.html) that can be remotely controlled via the Insteon protocol.
For heating and cooling I intend to roll my own controller using a Netduino.
I also intend to build clock radio alarms using the Netduino for each room. This device will have temp, light, sound and motion sensors so that it can detect when someone is in the room. So for instance if a person walks into the bedroom the motion sensor would detect them and send off a MQTT message that a person was detected and some central controller would decide what action should be taken. Perhaps the light is turned on in the room if it is dark in the room already.
I also intend to roll my own external door lock, where the door can be locked or unlocked remotely. A keyless entry system to allow for me to enter my home. At night the doors all lock when I go to bed. When I arm my security system the doors all lock. When someone rings my door bell I get a text message and another netduino sounds the doorbell.
My main point here is that connecting everything through a publish/subscribe message system such as MQTT gives you the flexibility to have many devices react to the various messages that are published. Very powerful.
I have seen your sprinkler setup and it inspired me to look at azure, although it seems to cost the earth and my ultimate msdn subscription doesn't seem to give you much scope for playing with service bus (I maxed my month limit in a week!).
The link to the smart house was great and answers a lot of questions for me.
One thing I would ask is where would you suggest starting?
Network switches, rack and server setup or start on the power?
How do you plan to control your consumer units, I like the din rail stuff have you seen any hardware that could be of interest?
What is your opinion in terms of node zero or distributed intelligence? Obviously the later uses less cable.
Any ideas on the control ethernet, signal cable, signal over power cable (x10), wireless (z-wave, lightwaverf), I hear the high power stuff can interfere with x10?
I am on board with the mqtt messaging, I however am just learning this type of thing and even your sprinkler code seem a little over my head at the moment.
Is it possible to do the same as your sprinkler system using sql server 2012, am I right that this has service bus built in so I could host on my own server and do what you are doing with azure internally?
Is so can you point me towards some tutorials that would help me down a route similar to your sprinkler setup?
Thanks again and your srinkler setup is a real inspiration to me!
Andy
Ps I intend to use power over ethernet, it seems a brilliant idea!
You don't have to use Azure as the host of your message bus. You can certainly have a message bus broker such as MQTT that resides in your home on a machine. I have some simpler code that might be easier to consume conceptually that uses MQTT. Take a look at http://www.cloudhomesecurity.com as it is a basic home security system that uses a network of netduinos all connected thru MQTT. In the security system example I have the MQTT broker running in Azure but it could just as easily run an a machine in your home. The MQTT broker I am using is Mosquitto (http://mosquitto.org/). When you run the Mosquitto windows installer it can install the broker as a service, but you can also just use the console application to start the broker. Mosquitto is open source and is free for you to use.
Yes SQL Server does have a message broker but I haven't used it for anything so I cant really comment on how it would work. When evaluating a broker to be used with the Netduino you have to consider what the protocol is and how lightweight it is. If you are sending bulky XML or SOAP messages between the broker and the netduino's that can get challenging. I don't really know how bulky the SQL Server message is but you should look into that before going that route.
Since you are in the stages of remodeling your house the broker decision can wait. That's something that you can do after the remodeling is complete. I would start on the wiring (Ethernet and Coax) and make sure it terminates into a central location that has plenty of room for all the wiring. You will need room in this same location for a rack to hold your hardware (switches, power and servers). I picked a closet that extended under my steps to the second floor which allowed me to plan for a rack installation (which I haven't done yet). Make sure you get plenty of power installed in this location. If you are going to do Power Over Ethernet make sure you get the correct type of Ethernet cable routed to each room. I didn't do POE in my setup. I routed at least 2 coax cables to each room and in some rooms I routed 4 coax cables. I did this because I wanted to route multiple video sources into and out of the room. However I think this was a mistake on my part because I am now doing more video streaming over Ethernet instead of over coax. I actually wish I would have run more Ethernet cables to each room rather than doing the coax runs.
I have been a longtime user of X10 and I still have devices today that are X10 which as you pointed out is a signal over power lines. I have had issues with reliability of these devices at times. I have no idea if that is because of noise or what but I am leaning toward replacing the devices with Insteon commercial products. Insteon is a wireless mesh protocol which is supposed to be more reliable (I don't have any of these devices yet so I cant comment with experience here). I planed on buying the Insteon PowerLinc that should allow me to connect my computer over USB to the Insteon devices. My main concern here would be can I use my custom software to control the devices or would I have to use the Insteon compatible software. That should just depend on if the serial protocol is open or not which isn't something I have looked into yet. One other issue I have with my home is about every light switch box has 2 to 4 switches in it. None of these dumb switches are the Paddle type and all of the Insteon switches are the larger paddle type. So I am not sure if I will run into space constraints here. I haven't seen any of the din rail stuff you spoke of so send me a link.
Also have a look at how Jonathan Oxer decided to do his distributed control for lighting. I think he covers it on the first episode of Super House TV. He took the approach that Power Over Ethernet is routed to a panel in each room. A panel would be located in the room where your light switch used to be. He places an Arduino behind this panel. The switches on the panel are monitored by the arduino and when they change he sends an MQTT message to the broker in his home. A central computer then picks up this message and decides where to route the signal to for example turn on the light in the room. This is a very interesting approach and gives him a lot of flexibility in decoupling the switches from the desired actuator.
For distributed intelligence are you referring to not having a central computer that makes all the decisions? If that's the case if you have many Netduino's distributed through your home and each of them have there own logic and make decisions based on that logic then you have a distributed intelligence in a sense. I am a big fan of distributing the logic across devices. The only issue I see with that is if these devices are embedded in hard to reach places then you might have problems upgrading their software to fix bugs or add enhancements unless you have a way to remotely deploy the new software bits.
I have just had my sparks in today to get some estimates, obviously they aren't used to anything like this, just the standard twin and earth loop for lights on each floor with switches for the walls.
Node zero is basically routing everything back to one central point, so in my case I'm having it in the corner of my basement and every single ring or individual twin and earth, as well as any control (cat5, serial etc) goes back to a single point in my house. The sparks said this could use 4 times the amount of cable, which again brings back my question about dispersed intelligence.
In itself, dispersed intelligence has a few options in my opinion, firstly you could take a prewired house as an example with typical ring wiring. To control the lights in this instance you could do a couple of things, put a device where the fixture is or where the switch is. You can then decide if you control the device directly or whether you communicate back to a central controller which send the command out to the device controller.
For example, if I put a smart control signal switch panel (low power) where the switch was, this may have a relay in that replaces the switch. To control this the smart switch could either control some logic on the device or communicate to a central controller that controls the relay. If you completely disperse logic throughout the house, you either lose control centrally, or you need each device to be capable of taking control both directly (the smart switch) and from a central source (perhaps cat5 or wireless).
I like the dispersed route in terms of cable efficiency and in terms of my sparks it would be a standard install for them to which I can replace switches etc in my own time.
The node zero option has everything centralised and seperate, so your lights have their own twin and earth going back to the control board, your switches communicate back to the controller, then all the decisions are made in one place. This is where the din rail stuff comes in, on your smarthouse example I think the second video you can see in his consumer unit there are a number of din mountable relays. These then just have a logic out that go to his arduino (not perhaps to a certifiable standard though).
I quite like the din rail approach as it seems the most robust structure and din rail devices will all be rated.
Hi McInnes
Sounds like your going to be busy for a while.
I think the biggest hurdle is regulatory, and that is why the prices are ridiculous.
this is partly why I went Isolated Extra Low Voltage for my lights. Because in Aus a light doesn't need approval if it is SELV.
Once you are in the SELV domain the House wiring rules no longer apply to your wires.
Is it just lighting you want to control? Or specific items? Some Power outlets or all power outlets? individually?
With lights you probably only need to run the switch wire back to the node. I assume you will want a local light switch too wont you, you can probably run this as SELV to the micro.
With power you are going to have a stack of conductors going in the same direction, so they will be de-rated. You might consider a sub distributlon board or two.
You can get small contactors wired into your switch board, maybe you could wire up your own sub boards and put the contactors in your self. And get the sparky to connect to mains. Just run the coil wires out to a SELV box or section, where you can attach your driver circuits.
Too Easy. (... cough cough)
the other thing is light dimming, this can be a bitch with all the different sorts of dimmers (leading, trailing, universal, cfl) and the cfl ones dont work well anyway. But if you have LEDs it is a lot easier.
Also think about some fire detectors, I don't know what is best but if you're doing home electronics I think they are good. One in the lab too.
I am thinking of everything and me being me I want it to be as granular as possible.
So...
Lights, sockets, security alarm, cctv, fire alarm, door entry, heating (zoned for each room), the aquarium, outside lights, pond, irrigation, basement sump, audio, video.
That just about covers it, by the way not all of the above are installed yet and some not for a while e.g. pond, irrigation and basement sump.
So in terms of lighting (mains) are the contactors ok to use and be up to a standard?
I was thinking having a high power and a low power enclosure, with the contactor controller cables going through to the low power enclosure. Then as you summised the switches will be low power smart switches either running some low power signal cable or more likely cat5e down to the low power control enclosure.
I may go led in the future as you get so much more control, efficiency and as you say avoid the regs issue, but some bits will still have mains lights.
Are the din rail dimmers ok in the same way the contactors are? If I'm right a normal incandesant bulb uses leading edge dimming right?
Do you have any automation setup bar your leds?
Mike...
I was having a look at the mosquito mqtt stuff, is this all linux based and if so how hard is it to setup? How does it interface with .Net both micro and normal? and are there any microsoft equivelents?
I have had a detailed look at the mosquito mqtt project and have installed the windows exe, there were 2 I didn't use the http://cygwin.com version, is this correct? I read through the cygwin user notes and it seemed a lot more complex. With the other one I now have a service running on my laptop.
If I wanted to start playing with this how do I go about it? The .Net examples I found didn't compile so haven't really helped me start digging through the workings.
In your sprinkler system I remember you described you system as having a home controller in the cloud, a home controller on your home server and obviously your netduino code. Using your security example do you have the code for the home controller that pulls all this together, I may have over looked something but I only saw the netduino source and azure source?
I like the MQTT broker it seems to make sense from a structure perspective e.g. home1/lounge/tv etc etc.
I imagine that from your security example I can work out communication with the broker.
However I could do with a little guidance in terms of connection to the MQTT broker and how you centrally handle or interpret messages within your home server?
Also do you log data, not messages but the actual data for example times of events, security exceptions, sensor readings etc in a database for reporting, manipulation etc etc and how could this be done within the structure of the security system?
Finally (so many question sorry very interested) how could I do what azure does for your but within my home network? I intend to serve a web interface out from my home server (once I set it up).
I plan to install server 2012 and use sql server 2012 for data logging and provide an mvc front end.
PS I really like your use of SignalR, that is amazing!
So in terms of lighting (mains) are the contactors ok to use and be up to a standard?
An electrician would be allowed to wire contactors into a switch board or subboard, Many switchboards use Din rail mounting. I think this would be the main option for controlling your house wiring. They could run out the isolated switch wires to another box that you would interface too.
But for me I am quite happy with the non automatic light switch.
Are the din rail dimmers ok in the same way the contactors are? If I'm
right a normal incandesant bulb uses leading edge dimming right?
Anything can dim the incandescent bulbs, it is the compact flouros and the flouros that dont dim well, especially when its cold.
the only automation I have is temperature logging. Via netduino plus. It monitors my solar hot water and a couple of other places, I was going to get it to automatically turn on the gas booster but I have just been flicking the switch myself, that actually works pretty well so the project didn't need any more.
I once did make an automatic chicken feeder and watering system, with web cam feedback. So I could feed them when on holidays via the web but I couldn't get the mechanical side right so the pellets kept jamming. It looked a bit crazy while it was running and used to freak the chickens out. If you need the plans to this masterpiece I'll open source them. It was controlled by an openWRT wireless router.
I was having a look at the mosquito mqtt stuff, is this all linux based and if so how hard is it to setup? How does it interface with .Net both micro and normal? and are there any microsoft equivelents?
I have had a detailed look at the mosquito mqtt project and have installed the windows exe, there were 2 I didn't use the http://cygwin.com version, is this correct? I read through the cygwin user notes and it seemed a lot more complex. With the other one I now have a service running on my laptop.
I don't use the Cygwin version either. I also just run the EXE (not the service) on my laptop just because I don't want it running all the time on my development machine. I run the service on my media center PC (since it is on all the time anyway).
In your sprinkler system I remember you described you system as having a home controller in the cloud, a home controller on your home server and obviously your netduino code. Using your security example do you have the code for the home controller that pulls all this together, I may have over looked something but I only saw the netduino source and azure source?
I haven't really blogged (or posted my code) about my latest sprinkler setup, but it is setup with MQTT now and uses 2 brokers. One in the home (home broker) and one running on Azure (remote broker). The one in Azure is only used to connect my phone to my sprinkler brains. The brains (scheduler, time service, and weather analysis) of the sprinkler is also running in my home (the media center PC again) as a set of services. The netduino and the brains connect to the home broker. They simply pass messages between each other thru the home broker. The home broker is setup to forward messages to the remote broker. It is also setup to receive messages from the remote broker. My phone connects to the remote broker. All of this was done so that I would not need to poke a hole in my firewall to allow for traffic to come into my house from the internet. As you can imagine this makes the setup a little more complex.
In my cloud home security example I decided to remove the remote broker. I did this because of the complexity of having two brokers in the mix. It was just too much to go over in a presentation where I am introducing Netduino and Messaging. So the brains of the home security system are actually running in Azure in the Web Role. You can look at that code in CloudHomeSecuritysrcHomeSecurity.WebHubsMasterControlPanel.cs class (https://github.com/m...oudHomeSecurity). I discuss in my presentation the pros and cons of this approach and how you can move the brains into the home but I don't have a good example on how this is done. In order to do this with the home security example I would have to break apart the MasterControlPanel from the signalR hub and place it into it's own console EXE. That way I could run it in the home or in the cloud and it wouldn't care where it was. The only thing it would care about is which broker it should talk to.
If I wanted to start playing with this how do I go about it? The .Net examples I found didn't compile so haven't really helped me start digging through the workings.
However I could do with a little guidance in terms of connection to the MQTT broker and how you centrally handle or interpret messages within your home server?
Finally (so many question sorry very interested) how could I do what azure does for your but within my home network? I intend to serve a web interface out from my home server (once I set it up).
I have been meaning to put these concepts in a repo and in a blog post for a simpler example that shows 2 brokers and sending messages between console apps and netduino controllers. I really need to do that to make it easier to understand. But really the way I got started is by using the mosquitto console apps to subscribe (mosquitto_sub) and publish(mosquitto_pub) messages. Check out the batch file examples I have in CloudHomeSecuritysrclibmosquitto.
Also do you log data, not messages but the actual data for example times of events, security exceptions, sensor readings etc in a database for reporting, manipulation etc etc and how could this be done within the structure of the security system?
Right now I simply created a console app that listens to all messages on the bus and writes them to a file. I call it the Audit application. That way I have one central location that captures all messages. Anything I want logged is a message on the bus anyway so this is very convenient. I was going to later change this app to write the messages to a SQL Database.
If I wanted to start playing with this how do I go about it? The .Net examples I found didn't compile so haven't really helped me start digging through the workings.
I use the following .Net lib for my client connections to an MQTT broker:
Thanks I will take a look at that example to see how you are intefacing with the mqtt broker. I think I am decided on the protocal I am going for, and that is z-wave.
It provides 2 way communication and each device is addressable, plus with its mesh network everything is a lot more relaible. Its a shame the lightwaverf doesn't have the 2 way comms. as B&Q (a big diy store) in the UK have a range of siemens produts that are cheap and look good. In terms of z-wave have you found any .Net sdks? For example if I use the netduino as the control via serial thats fine but I need to handle the following:
Reading the network to discover new devices.
Registering the devices and perhaps retrieving info on what they are.
Mapping the device parameters to classes that can be assigned to them.
Send messages.
Listen for messages.
Handle errors.
I saw this or this, do you think it is any good for controlling z-wave product if I linked it up to the netduino? I wondering if its best to keep the netduino quite dumb, almost as a relay for sending / receiving messages via serial and then return them to my server for processing. I guess this could be the mqtt bit in terms of sending them back to the server for processing. Then the server bit is the home controller right? Cheers, Andy
Ok so a little time has passed but there have been some major advances in my home automation project...
Since my last posts I have played a little with mqtt in various flavours including mosquitto and rsmb...
I have also been on a Windows Azure and Web Services course...
And employed an electrician to start work at the end of september...
After reviewing many different technologies for home automation, as well as getting an estimate for the cost of my rewire I have made some key decisions...
Taking a lot of inspiration from super house I have decided to go for a wired approach, using cat5e to carry pretty much all signals and low voltage around my house. Not only is cat5e relative cheap ~£36 for 305m, it provides shielding, twisted pairs and can carry low voltages happily upto the POE standard of 48v, which covers me for just about every eventuality.
Another key decision is that I have taking a radial wiring approach, otherwise known by "node-zero" or "star" wiring. Basically this means having a central point where all wiring comes back to, or in my case 2 points where all wiring comes back to. As my house is over 4 floors (basement, ground floor, first floor and attic) I am having 2 node-zeros; one in the attic and the other in the basement. What this means is the maximum run is only between 2 floors.
Essentially the idea is that everything has its own circuit which goes back to a distribution board; located in either the attic or basement. The wiring side of things is a little different than a convensional house; other than a standard consumer board with some RCDs etc, there is very little that reprisents a typical wiring system. From the consumer board 2 circuits (an 80amp for the basement and ground floor and 65amp for the first floor and attic) will come out going to 2 distribution boards. Here this supply will be split up in to functional circuits such as light for each floor, sockets for each floor, boiler, shower, cooker, garden from the relvant distribution board.
Each circuit then splits down to granularise the control, so the circuits will all have an RCD, but each component (socket, light etc) will have in most case a relay. And this is where my next decision comes in to play, I have decided to go for a complete home brew approach rather than paying the extortionate prices charged by HAL supplies which invoke some benefits and serveral contstraints each. Not only does price play a key role, but the ability to interface and control things the way I want. One downside to my home brew route is that this technology is untested and has not gurantees or support.
To combat this issue I am building a fallback in; there is a requirement in the UK to have a controlable light in each room, and so I am going to keep controls on the walls that are "dumb" controls. Rather than carrying 240v these will however be cat5e carrying 24v, this will have the ability to override my control boards should any software or electronic issues arrise. Some benefits of my approach include using wired control, this removes issues related to wireless and other signals over power. Also using cat5 provides a nice shielded delivery method. Also having my intelligence centrallised (be it centralised to 2 locations) means that I have 2 points of failure, which may sound bad, but is certainly less invasive when distributed intelligence goes wrong and you have to start ripping up floor board or plaster. I will easily be able to replace and identify any broken hardware and my hardware is cheaper as less intelligence (netduinos) are needed. Also having the manual, logic free, electronic overides will provide a failsafe.
My control boards are going to be din mounted using something like Cambden Boss enclosures and will use a din mounted 24v supply and cat5e to connect the control boards to a centralise netduino module. See my other post for more details.
I have decided to go for mqtt as my message protocol as inspired by Mike. The Azure and web services course I went on last week gave me some great inspiration for having a service oriented architecture and using queues.
I have also built a low power server that runs flat out at around 36 Watts and is completely solid state with no fans or moving parts. This is running server 2012 datacentre core edition, which I am enjoying a steep learning curve on how to configure my first domain and Hyper-V environment. I plan to run several virtual servers on this to create my own Azure like distributed environment, all be it without the ability to automatically scale out across more VMs.
Over the coming months I plan to share my project exploits and actively welcome anyones input, ideas, opinions and support. Once I get my server up and running I plan to get mqtt running. I am still to find any detailed information on how to host mqtt and would appreciate any guidance on how to do this or configure the Hyper-V role on server 2012 datecentre core edition.
On the MQTT hosting question. Do you want to self host an MQTT broker within your server application or do you want an MQTT broker that already installs as a windows service. The Mosquitto MQTT windows installer will install the broker as a windows service. http://mosquitto.org/download/
I am looking forward to the updates you will be posting.
What would you recommend in terms of hosting MQTT? I have played on my laptop with RSMB and Mosquitto. I ideally want to have MQTT running on my home server so I can access it from any of my home automation devices as this will be the central point where all messages are handled.
