Like a Scene from The Big Bang

Ron and his desk lampOk, I feel extremely nerdy. Did you see the episode of The Big Bang where they program an IoT device to turn on a lamp by sending a signal all around the world before it gets back to the lamp to turn it on? Well, I was suddenly struck by the similarity when I got Alexa to turn on the Z-Wave light bulb in my desk lamp last night:

  • “Alexa, turn on desk lamp”
  • The voice command is picked up by the Alexa Show device on my desk and sent to Amazon cloud.
  • Alexa Smart Home converts the audio command to a “turn on desk lamp” intent and routes it to my Alexa Smart Home skill running in an Amazon Lambda in the cloud.
  • My Smart Home skill converts the intent and forwards a “turn on desk lamp” command to Particle.io.
  • Particle.io publishes the command to my Photon IoT controllers, one of which is acting as a bridge from particle.io to MQTT.
  • The Photon bridge reformats the command and publishes it to MQTT.
  • The MQTT broker is running on a Mac Mini, which then forwards it to all other devices subscribing to MQTT.
  • One of these MQTT subscribers is the smartthings-mqtt-bridge app running on the Mac Mini. It then receives the MQTT “turn on desk lamp” command, and forwards it to the Smartthings web hook over the internet.
  • (Should I mention the Wifi to router to … Nah).
  • The Smartthings server receives the command via web hook, routes it through a Groovy program running in Smartthings, and forwards the command to my Smartthings hub over the internet (and router to wifi, etc).
  • My Smartthings hub then transmits the “turn on” command over Z-Wave radio to my desk lamp light bulb.
  • And the light turns on.

Woohoo!

Patriot 4: Swift Programming IoT Home Automation

I’ve turned a corner in my thinking about how best to automate my home. For years now I’ve been trying to integrate and generalize all of the various technologies (MQTT, SmartThings, Particle.io, Alexa, iOS, Raspberry Pi, etc) to provide a simplified interface for controlling everything. The tough part is really describing what I want the system to do; how it behaves. Using an any sort of text or table driven, simplified, abstract programming model always comes up short. By this I mean the sort of approach where I am allowed to specify things such as “if device A is on, and the time is after HH:MM PM, then set device B to XX”. This works great for doing fairly simple things, but doesn’t really handle the more complex types of scenarios I always run into, or if it does handle it, it does so in a manner that gets very complicated and unmaintainable very quickly. As a professional programmer, this situation is very familiar, and describes what happens with poorly written code. So I started thinking about how to solve this problem in a way that would be greatly extensible and very maintainable.

And I got to thinking about how Patriot automatically detects new devices. Well, sort of automatic. If I program a Photon to expose a device, and write the Photon code to implement that device, then Alexa can detect it and control it with voice. That’s very cool, but I’m getting tired of having to tell Alexa which lights to turn on and off every morning and night. I want my home automation system to know when to do that based on the plethora of sensors that I’ve installed.

So then the big ah-ha moment for me. Automatically detecting and installing devices doesn’t provide any automation until I write some sort of specific code to control it. This could be generalized code such as IFTTT, or bare bones custom code like Photon C++ or Python code on a Raspberry Pi.

One of the issues that frustrates me is the large number of different programming languages and platforms I’ve had to work with. So recently I decided to replace my Raspberry Pi with a used Mac Mini in order to reduce the number of different programming languages, environments, and platforms I have to remember and work with. The 2010 Mac Mini is quite powerful and flexible, and I found one on craigslist for $150. It’s now my media center as well as my MQTT broker and home automation controller. And I can program it in Swift, which is my daytime job programming language.

So if I have to write code no matter what, why not do it using the best tools, environment, and clean programming practices? To my mind that means Swift programming on the Mac Mini.

So that’s where I’m going with Patriot 4. I’m writing a Mac app in Swift using clean, TDD code. I’ll be sharing the Swift framework that is evolving out of this, but it’s going to require Swift programming skills in order to use it. I’m excited to see just how far this can go.

Alexa as Home Automation Integration Point

Over the past few months I’ve done a lot of experimenting with getting various home automation technologies to communicate with each other. For example, setup a Raspberry Pi to bridge between SmartThings and Home Assistant using MQTT. SmartThings in turned connected to my Harmony Hub. Then tied in my Patriot custom electronics devices via MQTT. What I ended up with worked, but seemed overly complex. I don’t even want to try to blog about how to reconstruct such a beast.

So that got me thinking about how to simplify things. I realized that the individual pieces are are fairly simple: SmartThings, Patriot, Harmony. Each of these provides their own Alexa smart home skill, and most of what I want to accomplish can be done entirely by Alexa. So it isn’t really necessary to try to bridge between each of those. Instead, I can let Alexa be the common interface to each of them.

So that’s what I’m trying now. I’ve removed the bridging pieces, and will be using Alexa to control everything. Things certainly are simpler now. I’m hoping that the recently announced support for Echo Buttons indicates a direction for Amazon that will allow other inputs to be used with Alexa. Amazon is holding to their “voice first” mantra, but we’ll see where this goes.

In the meantime, I’m investigating whether I can trigger an Alexa skill lambda handler directly from a Patriot device. I think it may be possible…

Making Sense of Z Devices (Z-Wave vs Zigbee)

As I stated in my last post, trying to understand and control Z-Wave and Zigbee devices sure is confusing.

Having looked at several different controllers and hubs, and used both Zigbee and Z-Wave lights, I’ve come to understand the major differences. But it is indeed confusing.

So here is my simplified, high level understanding and comparison of these two:

  • Zigbee devices are cheaper than Z-Wave.
    • For example, the cheapest light bulbs today are about $9 for Zigbee, and $17 for Z-Wave. That’s no big deal if you only need one or two. It is a big deal if you’re replacing a whole house full.
  • Zigbee is natively supported by the new Echo Plus; no extra hub required.
    • But if like me you already have Alexa, Dots, and Shows, you’ll still need a hub.
  • Z-Wave devices are certified.
    • What this means is that they should inter-operate more reliably, but that is probably why they’re more expensive.

Confusion factors:

  • Zigbee and Z-Wave are not interchangeable.
    • You cannot control a Zigbee light with a Z-Wave controller, and vs. a vs.
    • The SmartThings hub supports both, but Echo Plus supports only Zigbee, etc.
  • Light bulb advertisements on Amazon don’t always say which one is supported.
    • I was searching for Z-Wave light bulbs, and kept getting Zigbee bulbs that do not say “Zigbee”. It would be very easy to purchase the wrong type.

So caveat emptor.

Home Assistant to SmartThings and Back Again

How many programmers does it take to control a Z-wave light bulb?

I consider myself fairly technical, but I’ve got to say that trying to understand and control Z-Wave and Zigbee devices sure is confusing. And don’t even get me started with how complicated trying to program them has been. It all began when I decided that I needed to control a regular 120 vac light bulb in a desk lamp on my computer desk. As you can tell from all my earlier posts, I’ve done all my automation so far working with 12v DC, and prefer it that way. So it seemed like just using a Z-wave or Zigbee light bulb would be the easiest approach; no wiring required.

I did some early investigation, and saw that Home Assistant and SmartThings were a couple viable options. So I started putting together a Home Assistant system, while looking at ways for it to control either Zigbee or Z-Wave.

It just so happened that there was a Z-Wave sponsored contest starting, and applicants could receive a free Raspberry Pi + Sigma USB Z-Wave controller + choice of any Z-Wave device. So I applied, was accepted, and chose the Zipato RGBW2 light bulb. Off to a great start!

Home Assistant is an amazing open source project! I like it. I tried using it to control the light bulb via the Sigma controller, but could never quite get it working fully. Oh sure, I could turn the light on or off, but not control the RGB or 2 types of white light. Plus the darn Raspberry Pi 3 kept hanging overnight.

So, in typical programmer fashion, I decided I’d just cut out the middle man and write my own code to control the light bulb. Well, the scarcity of examples and lack of any tutorials made this a very frustrating effort. I never did get the Z-Wave bulb to respond to any of my own code. So when my maximum frustration threshold was exceeded, I decided to punt on writing my own code and give SmartThings a try.

Samsung has put together some pretty extensive, well written developer documentation. So I was initially very optimistic. I ordered a kit that included the hub plus a bunch of sensors, and quickly had the light bulb turning on and off in response to motion and doors opening. However, going from there to integrating with all my other automated lights and Alexa skills was again very challenging. I eventually was able to get everything integrated together by writing a SmartThings Device Handler and Service Manager that could together automatically detect and control my Patriot devices, but then what? In order to automate stuff, I had to write a bunch code, and organizing all this code quickly got out of hand.

In my googling about all this, I came across several posts that shared my experiences; SmartThings can control a lot of stuff, but using it to automate things required writing lots of code. Some of these posts described that Home Assistant was really good at solving that problem. I also read where the Home Assistant guy (folks?) are looking into using machine learning to establish automation rules. That’s the direction I’ve been trying to go. So maybe I’ll throw my hat into that ring.

So I’m back to working with Home Assistant. And I was delighted to find that in the several months since I looked at it before, installing Home Assistant has become simpler and easier.

There are options for connecting SmartThings sensors to Home Assistant using an MQTT broker, and I may take a look at that later. I’ll eventually want to be able to interface with Zigbee devices, and using the SmartThings hub is one option. But we’ll see.

Oh, and the Raspberry Pi hang? I finally figured out that it was caused by leaving an open SSH connection overnight. So now I exit SSH when I’m done using it, and haven’t had any more hangs.

Backup Internet for my Alexa and Particle.io Devices

I’m really enjoying using Alexa to control the lights in my home. One down side to this however is the requirement to be connected to the internet. Without an internet connection Alexa becomes a lazy idiot.

SmartThings, TPLink Nano, and iPhone

Since I live in an RV, I won’t always have a fast internet connection. So what I’ve done to deal with that is to have two separate hot spots, only one of which is ever active at any given time.

When my fast internet (was Roadrunner, will soon be Google fiber) is available and working, it transmits to all my devices.

When I’m on the road, or the fast internet breaks (like for the past 5 days!), I plug in a TPLink Nano Wireless N router. This little $35 box is very versatile, and can handle receiving and/or transmitting on Wifi and/or ethernet.

For example, as shown in the image, the SmartThings hub needs an ethernet connection. So the Nano is configured as a “Client”, to receive WiFi from the iPhone and convert it to an Ethernet connection for the hub.

Likewise, with the Nano connected to my normal WiFi router, I can configure the Nano to receive WiFi at whatever RV Park I happen to be staying, and distribute it to all my WiFi and IoT devices without having to reconfigure anything except the Nano.

SmartThings Control of Photon Devices Using Patriot

SmartThings

So having finally received my SmartThings hub and a few devices, I’ve spent the past week learning how to program it to interface with my existing Particle.io Photon controllers running Patriot. It turns out that SmartThings has a fairly nice architecture that made automatically discovering my existing devices fairly easy. I needed to write a Service Manager SmartApp and a child Device Handler. Altogether this was about 200 lines of Groovy code (basically Java). The service manage interacts with Particle.io to locate each Photon controller on my account, and then ask it what devices it supports. It then creates a child device for each using the name exposed by the Photon. Voila!

This is essentially the same approach used by the Patriot Alexa Smart Home controller that I published for Alexa.

So yesterday when I finally got the kinks worked out, I reinstalled the Patriot Service Manager SmartApp that I’d written, and it automatically discovered my two dozen Patriot lights and added them to SmartThings. Woohoo!

So now I just need to explore the Alexa implementation on SmartThings and see if my Alexa Smart Home skill is still needed. At this point it looks like it can be completely replaced by the SmartThings Alexa support.

Switching to Samsung SmartThings

After a frustrating few weeks of unsuccessfully trying to get Home Assistant working with my Patriot Particle.io devices and Z-Wave bulbs, I’ve decided to switch directions and won’t be investing any more time on Home Assistant. While I really appreciate all the hard work that folks have put into Home Assistant, it just isn’t a good fit me.

So I am switching gears and am investigating integrating Patriot with the Samsung SmartThings hub instead. This will give me both Z-Wave and Zigbee control, and some level of operation when disconnected from the internet.

I’ve read through much of the SmartThings developer documentation, which is very good. I like the architecture, and clear descriptions of how to integrate 3rd party hardware.

I ordered a SmartThings Monitoring Kit from Amazon that should be here tomorrow in time to have some of my Holiday time off to play with it. This kit includes the SmartThings hub, a couple door sensors (with vibration and temperature sensing), an a/c switched outlet, and a motion sensor.

In searching the developer forums I’ve located some posts by others that have been successful in integrating SmartThings with the Particle.io cloud, so I’m confident that in no time I’ll have it controlling the plethora of lights, fans, and awnings that I currently control using Particle.io Photons and Patriot.

Teaching Alexa What To Do

In the last post I described combining an Alexa custom skill with an Alexa smart home skill. Well, I got that working this past week and it works great. The combination really has some interesting potential.

Chihuahua and electronics
Just a cute little doggie watching me wire stuff together.

Smart home skills are great because they’re short and simple, and the supported device names can be extended dynamically through the “discovery” process.

Custom skills are great because they’re very flexible, and allow interactive dialog, unless something fits into a lot, isn’t extensible.

Putting the two together can provide the best of both. Here’s what a dialog with Alexa sounds like using both the Patriot smart home skill and the new Patriot custom skill.

Initially the skills don’t know about any activity called “computing”. So when I say:

“Alexa, turn on playing piano”, it responds with “Sorry, I didn’t find playing piano”.

That’s as expected. So let’s teach Alexa what we want it to do when we say “playing piano”. Note that I’ve made the invocation words “my lights”.

“Alexa, tell ‘my lights’ to turn on office when I say playing piano”

Then Alexa responds with:

“Ok, from now on when you say ‘playing piano’ I will set ‘office’ to 100 percent. This is a new activity, so you’ll need to tell me to ‘discover devices’ if you want to use the smart home command ‘start playing piano’.”

How cool is that?

So then saying “Alexa, discover devices” results in “Ok, found one new device ‘playing piano'”.

And then I can say “Alexa, start playing piano” and the office light comes on.

Using ASK CLI to Create a Custom Skill

When Amazon announced the ASK CLI a couple months ago, it created a simpler and more powerful way of creating and updating Alexa skills. We’re going to use the ask-cli to create an Alexa custom skill. In the next blog post we’ll extend that skill to interact with a Particle.io Photon using open source Patriot code.

Before the ASK CLI was available, I had to open multiple browser windows and edit data directly in the Amazon Alexa developer portal and AWS Lambda console. As a professional software developer, I’m accustomed to using powerful editors and source code management tools such as Git to track my changes. Being forced to enter data into a web browser page leaves a lot of room for mistakes. And tracking those changes with Git means having to cut/paste from a tracked local file to the browser, again leaving room for more mistakes.

The ask-cli goes beyond just allowing local files to be uploaded to an Alexa skill. It provides a start-to-finish set of commands to create, update, and publish skills.

So let’s see how the ask-cli can be used to create a new Alexa skill from the ground up.

Install and Initialize the ASK CLI

Refer to the Amazon documentation for instructions on installing and setting up the alexa skills kit command line interface (ask-cli). You’ll need to configure it with your Alexa developer account and an AWS account using the “ask init” command.

Create a New Skill

Now create a directory to contain your new skill, and run the “ask new -n <skillname> ” command. For example, I’m naming mine “Patriot”, so the command is “ask new -n Patriot”. This results in the following directory structure:

folder structure created by ask new

In one fell swoop we have created a basic “Hello World” Alexa custom skill. This includes the Alexa intent schema, utterances, and Lambda source and meta data. Pretty cool, eh?

Add Source to Git

If you use Git to track your source changes, now would be a good time to create a repo and add the files to it. This step is completely optional, but recommended.

Run the Skill

At this point, even without having changed anything, the new skill should work. Let’s upload it just to see:

ask deploy

If you’re accounts and ask-cli are setup correctly, then you should receive a series of messages indicating that the skill and lambda have been deployed correctly as shown here:

ask deploy
-------------------- Create Skill Project --------------------
Profile for the deployment: [default]
Skill Id: amzn1.ask.skill.your-new-unique-id...
Skill deployment finished.
Model deployment finished.
Lambda deployment finished.

Now if you check your Amazon Dev Alexa and AWS accounts, you should see that a new Alexa skill with the name you specified on the “ask new” command, and a Lambda¬† named “ask-custom-<name>-default” have been created. The default invocation word for the skill created by “ask new” is “hello world”.

By default, the new skill is not enabled for testing. Go to the test tab in the developer.amazon.com Alexa console console and enable it, and then you can test “hello world” on your Alexa device (Echo, Dot, EchoSim.io, etc).

Edit the Source Code

Ok, so now that you’ve seen the awesome power of a fully functioning death star, er I mean Alexa skill, we can commence to editing it to do something that we want beside telling us hello.

Now begins the iterative development process:

  1. Updating the source
  2. Deploying the skill
  3. Testing the skill
  4. Repeat

I strongly recommend that you make tiny changes each iteration, and use Git to check in each step of the way. That way you can back up a step if something breaks and you cannot figure out what.

There are 2 main source code files you need to work with. For simple skills, that’s all you need to modify:

  1. models/en-US.json (if you’re in the US, otherwise named for your language)
    contains the intents, slots, and utterances (now called samples)  that Alexa will respond to.
  2. lambda/custom/index.js
    contains the response to each intent.

By default your new skill will say “Hello” in response to launching the skill eg. “Alexa, open hello world”, or “Hello <name>” in response to “Alexa, tell hello world my name is <name>”. I recommend playing with the existing code, making small changes to the skill, redeploying it, and verifying that your changes act as expected.

Here are some things to try:

  1. Change the response to the SayHello intent from “Hello World!” to “Hello whatever your name is”. This should require just a change to line 25 of index.js.
  2. Change the help response. This is on line 43 of index.js.
  3. Add some additional samples to en-US.json for the user to say to invoke the two intents. For example, add “whats up,” between “hello”, and “say hello”,.
  4. Change the invocationName in en-US.json. For example, change “invocationName”:”hello world” to “invocationName”:”ahoy matey”. In addition, change the response in index.js to “Welcome aboard!”

If you don’t include quotes or commas where needed, ask will happily upload the broken code, and you won’t know until you test the skill. This is where a good Javascript editor comes in handy.

I’m not going to try to cover all the details of coding an Alexa skill here. There are lots of tutorials and blog posts in addition to Amazon’s documentation. I leave that as a homework assignment for you.

In the next article I’m going to show how to update this skill to send on and off commands to the LED on a particle.io Photon.