Recommendations for Easily Expandable and Programmable Home Automation.

[AdamCollins]

Hello all.

I've done a bit of lurking around here as well as some Googling here and there, but I have a few questions/thoughts that I'd like to get your opinion on, if that is not too much to ask. By all means, if I have missed something obvious on this site or elsewhere, please don't hesitate to tell me to go away and do some more reading.

I would like to gradually start controlling more and more things in my house with a PC. This is something that I will primarily be doing as hobby, because I think it's neat. So I'd like to start small, but in a way in which I can add more functionality as I have the time and money to do so. I am a programmer by trade, so I'd like to be able to write the software that will be controlling everything (this is the part that I find fun/neat). Also, I'm not at all comfortable with electrical work, so I'd prefer to use whatever requires the least amount of rewiring (I'm comfortable replacing light switches, but beyond that, I'd probably end up burning the house down).

My research so far has led me to believe that I should stay away from X10. I've got my eye on Insteon, but that is primarily due to the fact that I've already found some example code that I could build upon to start writing control software; I'm open to other suggestions, though.

Here is what I'm thinking of doing at this point:

I'd like to buy and install a couple Insteon light switches and a PLM. (Or is it the PLC? I'm not sure what the difference is exactly, or which one I'd need, or if paying for the SDK is required.) That way I could, for minimal effort/cost, get myself familiar with programming to the protocol Insteon uses. Once I become more comfortable with it, I could buy more switches/sensors/whatevers and expand the functionality of my software as I see fit. Does this sound reasonable, or is there something obvious that I am missing because I'm so new to all this?

Here are a couple of general questions I have:

• I have quite a few compact florescent bulbs in my house, so I am guessing that Insteon dimmer switches (or some other equivalent) are out of the question. Would just plain old Insteon on/off switches work for these? Similarly, a few places that I might want to control have florescent tube lighting. Would these have any special requirements?
• A few rooms in my house have two light switches for the same light, each on a different side of the room. Would installing one Insteon switch be enough here, or would I need to replace both switches?
• Just a random paranoid thought here, because I don't know much about electrical wiring: let's say I write my control software in a stupid manner, and a situation arises in which it sends some arbitrarily large number of on/off commands per second to a switch. What would happen? Would the switch just stop responding? Would it respond as best it could until it caught fire and burned the house down? Would it work like a champ and give me a sweet rave party?

As you can see, there's a lot I don't know, so any help/advice you can give me would be greatly appreciated.

 

[tmbrown97]

These are more generic answers; someone with specific model experience with Insteon can give better answers.

1. Usually the dimmer portion can be disabled for these types of loads; as long as the switch type uses a neutral and isn't power-stealing, then they're generally fine with CFL's. If you're worried about it, relay-type switches are even better, but then you'll have to replace them if you want to upgrade to dimmable LED in the future.
2. You'll need to use the special 3-way switch; most of these controllable switches have a special slave device you have to use in the second switch location. Otherwise you can use the wiring to send full-time power to the second switch and use virtual links to simulate 3-way function. Regardless, you can't leave the old 3-way switch in the mix.
3. The switch will only respond as fast as it can - it'll be limited by communication speed. If you do cause a failure, there's no guarantees how it'll fail - it mail fail on, it may fail off, or it may start smoking. I haven't heard of a single person having their house burn down yet though, so that's good!

From what I've read, each of these switch technologies has their own nuances. You could download something like Premise (if it supports insteon) and see how it talks to the switches... Might save you some steps.

This may just be me, but I never understand why all software developers want to roll their own code. Can you imagine how much collective developer time has gone into something like programming Premise, or the ISY or CQC or Elve? Wouldn't your time be better spent customizing the parts you want or extending where needed, but pulling from already developed code where possible? That way you only have to tinker when you want to make something better; but 98% of the work is already done. It'll take long enough customizing touchscreen interfaces... it's silly to start from scratch when you can get something like Elve for $99 then extend where necessary with the extensive API's available. This is big-picture experience that comes from running a large IT Department, including a couple software development teams, for many many years. Before starting any project you should evaluate buy vs. build and weigh out the pros and cons; and when estimating how long it'll take you to do something, multiply by 10 before weighing that in.

 

[naething]

I agree with not rolling your own code. In fact, I asked the same question on these forums about a year ago (should I roll my own system). I'm very glad I didn't.

I use a Micasaverde Vera 2, but this same advice applies equally to any descent automation system that is expandible with plugins. That is: take advantage of all the work that has gone into these systems and save your effort by only writing code that you really have to. A lot of smart people have spent a lot of time on most existing programs.

I've been slowly expaning the scope of my home automation project more devices, some of which I have to write plugins for and some I don't. If I'm lucky someone else has written a plugin for me. But even for the ones I need to write the code for myself I benefit from other people using it and improving it through feedback or patches. That only happens because other people are using Vera. No one else would be interested in running a custom home automation program I wrote from scratch. No offence, but I wouldn't be interested in running yours either. People want to use something with a nice existing user base and some guaratnee of future support.

I understand that you enjoy writing the code that controls everything. I do to! But once you get into home automation seriously the scope is too broad to do it by yourself. And I find it far more enjoyable knowing that other people are benefitting from my code as well.

 

[AdamCollins]

These are more generic answers; someone with specific model experience with Insteon can give better answers.

1. Usually the dimmer portion can be disabled for these types of loads; as long as the switch type uses a neutral and isn't power-stealing, then they're generally fine with CFL's. If you're worried about it, relay-type switches are even better, but then you'll have to replace them if you want to upgrade to dimmable LED in the future.
2. You'll need to use the special 3-way switch; most of these controllable switches have a special slave device you have to use in the second switch location. Otherwise you can use the wiring to send full-time power to the second switch and use virtual links to simulate 3-way function. Regardless, you can't leave the old 3-way switch in the mix.
3. The switch will only respond as fast as it can - it'll be limited by communication speed. If you do cause a failure, there's no guarantees how it'll fail - it mail fail on, it may fail off, or it may start smoking. I haven't heard of a single person having their house burn down yet though, so that's good!

From what I've read, each of these switch technologies has their own nuances.

Thank you! It's good to know that burning down houses isn't a thing home automation is known for.

I'm not sure I understand everything you've said about question #2, but that's fine; I don't think I'll be tackling any three-way switches starting off. Hopefully by the time I get to them I'll understand all of this a lot better.

You could download something like Premise (if it supports insteon) and see how it talks to the switches... Might save you some steps.


This may just be me, but I never understand why all software developers want to roll their own code. Can you imagine how much collective developer time has gone into something like programming Premise, or the ISY or CQC or Elve? Wouldn't your time be better spent customizing the parts you want or extending where needed, but pulling from already developed code where possible? That way you only have to tinker when you want to make something better; but 98% of the work is already done. It'll take long enough customizing touchscreen interfaces... it's silly to start from scratch when you can get something like Elve for $99 then extend where necessary with the extensive API's available. This is big-picture experience that comes from running a large IT Department, including a couple software development teams, for many many years. Before starting any project you should evaluate buy vs. build and weigh out the pros and cons; and when estimating how long it'll take you to do something, multiply by 10 before weighing that in.

Premise, CQC, Elve... these are things I haven't heard of yet. I will be looking into them, so thank you for pointing me in their direction.

And yes, I agree that using pre-existing software would be way more efficient. The thing is, my goal here is not entirely the end product of having an automated house. That's a big part of it, to be sure, but the majority of the reason I'm wanting to do this is because I want to learn how to do it. Learning a protocol, learning a device that uses that protocol, and using both to do something as neat as controlling things in my house, that's what I'm looking forward to doing. Even if it takes me 3 months to learn just enough to turn one light switch on and off at the click of a button on my computer, I'll enjoy every second of learning how to do that. I know it's not rational or efficient, but it sounds like fun to me.

I agree with not rolling your own code. In fact, I asked the same question on these forums about a year ago (should I roll my own system). I'm very glad I didn't.

I use a Micasaverde Vera 2, but this same advice applies equally to any descent automation system that is expandible with plugins. That is: take advantage of all the work that has gone into these systems and save your effort by only writing code that you really have to. A lot of smart people have spent a lot of time on most existing programs.

I've been slowly expaning the scope of my home automation project more devices, some of which I have to write plugins for and some I don't. If I'm lucky someone else has written a plugin for me. But even for the ones I need to write the code for myself I benefit from other people using it and improving it through feedback or patches. That only happens because other people are using Vera. No one else would be interested in running a custom home automation program I wrote from scratch. No offence, but I wouldn't be interested in running yours either. People want to use something with a nice existing user base and some guaratnee of future support.

I understand that you enjoy writing the code that controls everything. I do to! But once you get into home automation seriously the scope is too broad to do it by yourself. And I find it far more enjoyable knowing that other people are benefitting from my code as well.

Point well taken. The reasons I want to roll my own are:

1. I want to learn how to do it, just for the fun of learning something new. I have absolutely no doubt that whatever I produce in end will be fugly and won't work nearly as well as something that's already out there, but the process of learning as much as I can about it is what interests me most.
2. Customizability. How customizable are the existing programs? Here's what I mean by customizability (I'm turning my random BS machine up to 11 here): If it's the fifth Monday in July, the top Google search result for my home town mentions french fries, there are at least 8 different tweets with the hashtag #PinkElephantsOverOregon, the local barometric pressure is 30.08, and at least half the lights in my house are already on, then I want each light in the house to flip on and off in the Morse code sequence to spell out "Hallelujah Pink Elephants On Monday!" That is all very stupid, of course, but you get the idea; I want complete control and ability to do whatever I want. That said, I haven't looked into the Micasaverde Vera 2 and its plugin options (which I now plan to do soon), so I don't know if it would give me that kind of control. If so, and if it has a nice API, then I may give it a shot.

All silliness aside, thank you both for taking the time to respond to my questions. I really appreciate it.

 

[Dean Roddey]

There'll still be plenty of opportunities to customize and code. Any automation product will support a device driver slash plugin type of architecture to allow you to integrate devices into the system. Odds are you that you'll have one or more than aren't supported yet, and if not you can always arrange to if you want to. So there's device driver writing, which does involve understanding communications protocols and can be pretty challenging in and of itself for more complex devices.

And there are various other types of customization involved. So even starting with something like CQC you won't be short of interesting things to do. It can be taken as far as you want to take it pretty much.

 

[tmbrown97]

eh - just lost my typed response...

Anyway - I'm glad you're realistic about the project - just because I have a long history of working with and managing software developers, and that's one of the most important things to learn.

As far as your hypothetical, we can take Elve - you could write the Google and Twitter API's, use the existing wunderground weather feed to get the barometric pressure, then use a predefined action to flash your lights. So it's about 50/50 having to develop it all!

I'm not sure your programming language of choice, but I ran a .Net shop which I think is why I like Elve so much - it's 100% .net and well written; it's very extendable and intuitive... and the $99 into price can't be beat.

 

[JimS]

Others have good points about reusing and building on existing software. For most people that is the best way to go. But if you want to code for the learning experience, etc. then there are packages with varying degrees of openness. If you want to get deep into the code then misterhouse is an option. It is written in perl, runs on a PC, and has interfaces for just about everything. Warning - it has a steep learning curve. But the mailing list is pretty helpful and getting a basic system going isn't too difficult.

http://misterhouse.sourceforge.net/

 

[naething]

And yes, I agree that using pre-existing software would be way more efficient. The thing is, my goal here is not entirely the end product of having an automated house. That's a big part of it, to be sure, but the majority of the reason I'm wanting to do this is because I want to learn how to do it. Learning a protocol, learning a device that uses that protocol, and using both to do something as neat as controlling things in my house, that's what I'm looking forward to doing. Even if it takes me 3 months to learn just enough to turn one light switch on and off at the click of a button on my computer, I'll enjoy every second of learning how to do that. I know it's not rational or efficient, but it sounds like fun to me.

Well in that case, I say give it a shot! And despite what I said about not wanting to use your code, if you get something working well think about open sourcing it. Either way I would be interested in how things go for you, please post some progress reports here as you get further along.

I would really think about using an event-driven engine (there are lots available: Python Twisted, Perl AnyEvent, lebev for C, etc) as a basis for your work. You can look at EventGhost for windows to see how something was written with stacklass python with a very simple event-driven framework.

Misterhouse has a lot of community support and works well for a lot of people - but it is basically just a giant loop where you each device / interface is checked to see if there is a new action to take. There are a lot of better / more modern ways of tackling that now, so if you are writing your own from scratch don't use that as a guide on how to build one of these systems. If Premise gets open sourced (http://cocoontech.co...se-open-source/) I would check that code out too.

Either way I would grab any of the free or open source products you can (Misterhouse, Premise, Heyu, Zenah-Perl [1], EventGhost, Freedomotic) and see how they do things to get some ideas. Remember Picasso, "Good artists copy, great artists steal".

Oh, and I don't think three way switches are too hard. Just to be save you could pick up a book on wiring (something like Black & Deckers Complete Guide to Wiring for $15) and scan through it. Wiki has a good description of basic 3-way/4-way wiring: http://en.wikipedia....y_and_four-way. The tricky thing about home automation products is they don't use a typical 3-way or 4-way wiring circuit, and each manufacture does things a bit different. For instance, Intermatic z-wave switches use standard switches (not standard 3-way, standard 2-way) to add a 3-way zone. If you understand how normal circuits work, and then just spend a few minutes studying the little piece of paper that comes with your device, you can figure it out. Or, just post your existing circuit here and ask how to modify it and I'm sure someone will help.

[1] https://github.com/beanz/zenah-perl/

 

[wuench]

So Insteon specifically you'll not want to use dimmers for florescent loads, you should use the relay switches, also for LED, Ceiling Fans, etc. Also, the insteon protocol operates on the zero crossing so it sends at 60hz 60bps. If you swamp the system, you swamp the system, queuing is limited/non-existent. Direct commands have automatic reply/cleanup so therefore reliability, group/scene commands have cleanup until the next message is sent which typically doesn't complete so less reliability. There are two versions of Insteon, the older version there were CRC errors in the firmware so you couldn't send bulk messages, you were forced to send 1 byte at a time. So network discovery can take 10's of minutes. The newer firmware has fixed that as well as enable dual-band communciation and enhanced security. Scenes/groups are locally significant (so a group in 1 controller is not the same group in another controller) which is unlike any of the other automation protocols. The protocol is also missing very basic reliability features, it is basically X10 with CRC's for collision detection, reliability, switches will retransmit 3 times to try to cover for the lack of reliability. All the X10 noise and signal sucking issues still apply.

If you want to program Insteon yourself you should invest in the SDK as you can access their forum and get help with the quirks. The alternative would be to build upon something like the ISY that will handle the details for you. With Insteon you really only want one central controller, so if you need control from multiple devices (multiple PLMs) you are best to use a centralized controller like the ISY.

Having written my own Insteon software (PLC based, not PLM) as well as a driver for CQC to talk to the ISY, I can tell you from experience, you will be happier working with the ISY's API but it's up to you.

 

[123]

Adam,

If you're hell bent on writing an Insteon driver, consider writing one for an existing HA application (that lacks an Insteon driver). You're efforts will benefit the HA app's community and you can benefit from the community's existing body of work. It makes more sense to me than creating a standalone Insteon driver that, quite honestly, will have limited appeal to the general HA community.

Premise has an Insteon driver but it works with the PowerLinc Controller (PLC) as opposed to the newer PowerLinc Modem (PLM). A new Insteon PLM driver for Premise would earn you the undying gratitude of the Premise community (how cool is that!). If you know how to code in C++, C#, or VB.NET then you're good to go. Otherwise, you could attempt to write a driver using Premise's flavor of VBScript (the existing ZWave driver is written this way) as long as your driver model is single-threaded and uses no callbacks.

If that doesn't appeal to you, consider writing an Insteon driver for XaP or XPL. Several HA apps know how to communicate via these standards and would benefit from your hard work.

 

[electron]

And don't forget to check the link in my sig to see what other home automation software is out there, you can even filter by INSTEON support. There is so much software (and hardware) out there, rolling out your own solution only makes sense if you are looking for the experience (or researching fast-acting hair loss methods).

 

[firstone]

Premise has an Insteon driver but it works with the PowerLinc Controller (PLC) as opposed to the newer PowerLinc Modem (PLM). A new Insteon PLM driver for Premise would earn you the undying gratitude of the Premise community (how cool is that!). If you know how to code in C++, C#, or VB.NET then you're good to go.

Is it possible to use Java? If so, I can probably adapt my libraries to run as a Premise driver.

 

[AdamCollins]

There'll still be plenty of opportunities to customize and code. Any automation product will support a device driver slash plugin type of architecture to allow you to integrate devices into the system. Odds are you that you'll have one or more than aren't supported yet, and if not you can always arrange to if you want to. So there's device driver writing, which does involve understanding communications protocols and can be pretty challenging in and of itself for more complex devices.

And there are various other types of customization involved. So even starting with something like CQC you won't be short of interesting things to do. It can be taken as far as you want to take it pretty much.

Thank you for your response. Before I started this thread, I didn't really know about things like CQC, so I'm looking into those now. At this point, I kinda feel like I'm drowning in a sea of Home Automation information, but it is kinda sorta starting to gel in my mind.

 

[AdamCollins]

eh - just lost my typed response...

Anyway - I'm glad you're realistic about the project - just because I have a long history of working with and managing software developers, and that's one of the most important things to learn.

As far as your hypothetical, we can take Elve - you could write the Google and Twitter API's, use the existing wunderground weather feed to get the barometric pressure, then use a predefined action to flash your lights. So it's about 50/50 having to develop it all!

I'm not sure your programming language of choice, but I ran a .Net shop which I think is why I like Elve so much - it's 100% .net and well written; it's very extendable and intuitive... and the $99 into price can't be beat.

Yeah, I know plenty of developers who think that if they didn't write it themselves, it must be crap. I try not to be that way, but I'm not always successful.

Elve is starting to look more and more like something I'd like to have. My programming language of choice at the moment is C#, and while I'm not opposed to using another language, using something I'm familiar with would probably make things easier on myself.

So, if I am understanding you correctly, Elve's plugin architecture would allow me to write pretty much any logic I wanted into a plugin, and then that plugin would trigger actions in Elve. Is that right, or am I misunderstanding you?

Thanks again for your responses.

 

[AdamCollins]

Others have good points about reusing and building on existing software. For most people that is the best way to go. But if you want to code for the learning experience, etc. then there are packages with varying degrees of openness. If you want to get deep into the code then misterhouse is an option. It is written in perl, runs on a PC, and has interfaces for just about everything. Warning - it has a steep learning curve. But the mailing list is pretty helpful and getting a basic system going isn't too difficult.

http://misterhouse.sourceforge.net/

I don't know a lick of Perl, but I'm not opposed to trying to figure it out. Thank you for the suggestion.