Speaking at the Austin IoT Meetup

Arduino and ChihuahuaLast night I gave a presentation at the Austin IoT Meetup group about my RV home automation work. It was well received, and a lot of great discussion ensued.

You can see a PDF of the presentation here.

 

 

Alexa as Home Automation Integration Point

Over the past few months I’ve done a lot of experimenting with getting various home automation technologies to communicate with each other. For example, setup a Raspberry Pi to bridge between SmartThings and Home Assistant using MQTT. SmartThings in turned connected to my Harmony Hub. Then tied in my Patriot custom electronics devices via MQTT. What I ended up with worked, but seemed overly complex. I don’t even want to try to blog about how to reconstruct such a beast.

So that got me thinking about how to simplify things. I realized that the individual pieces are are fairly simple: SmartThings, Patriot, Harmony. Each of these provides their own Alexa smart home skill, and most of what I want to accomplish can be done entirely by Alexa. So it isn’t really necessary to try to bridge between each of those. Instead, I can let Alexa be the common interface to each of them.

So that’s what I’m trying now. I’ve removed the bridging pieces, and will be using Alexa to control everything. Things certainly are simpler now. I’m hoping that the recently announced support for Echo Buttons indicates a direction for Amazon that will allow other inputs to be used with Alexa. Amazon is holding to their “voice first” mantra, but we’ll see where this goes.

In the meantime, I’m investigating whether I can trigger an Alexa skill lambda handler directly from a Patriot device. I think it may be possible…

Making Sense of Z Devices (Z-Wave vs Zigbee)

As I stated in my last post, trying to understand and control Z-Wave and Zigbee devices sure is confusing.

Having looked at several different controllers and hubs, and used both Zigbee and Z-Wave lights, I’ve come to understand the major differences. But it is indeed confusing.

So here is my simplified, high level understanding and comparison of these two:

  • Zigbee devices are cheaper than Z-Wave.
    • For example, the cheapest light bulbs today are about $9 for Zigbee, and $17 for Z-Wave. That’s no big deal if you only need one or two. It is a big deal if you’re replacing a whole house full.
  • Zigbee is natively supported by the new Echo Plus; no extra hub required.
    • But if like me you already have Alexa, Dots, and Shows, you’ll still need a hub.
  • Z-Wave devices are certified.
    • What this means is that they should inter-operate more reliably, but that is probably why they’re more expensive.

Confusion factors:

  • Zigbee and Z-Wave are not interchangeable.
    • You cannot control a Zigbee light with a Z-Wave controller, and vs. a vs.
    • The SmartThings hub supports both, but Echo Plus supports only Zigbee, etc.
  • Light bulb advertisements on Amazon don’t always say which one is supported.
    • I was searching for Z-Wave light bulbs, and kept getting Zigbee bulbs that do not say “Zigbee”. It would be very easy to purchase the wrong type.

So caveat emptor.

Home Assistant to SmartThings and Back Again

How many programmers does it take to control a Z-wave light bulb?

I consider myself fairly technical, but I’ve got to say that trying to understand and control Z-Wave and Zigbee devices sure is confusing. And don’t even get me started with how complicated trying to program them has been. It all began when I decided that I needed to control a regular 120 vac light bulb in a desk lamp on my computer desk. As you can tell from all my earlier posts, I’ve done all my automation so far working with 12v DC, and prefer it that way. So it seemed like just using a Z-wave or Zigbee light bulb would be the easiest approach; no wiring required.

I did some early investigation, and saw that Home Assistant and SmartThings were a couple viable options. So I started putting together a Home Assistant system, while looking at ways for it to control either Zigbee or Z-Wave.

It just so happened that there was a Z-Wave sponsored contest starting, and applicants could receive a free Raspberry Pi + Sigma USB Z-Wave controller + choice of any Z-Wave device. So I applied, was accepted, and chose the Zipato RGBW2 light bulb. Off to a great start!

Home Assistant is an amazing open source project! I like it. I tried using it to control the light bulb via the Sigma controller, but could never quite get it working fully. Oh sure, I could turn the light on or off, but not control the RGB or 2 types of white light. Plus the darn Raspberry Pi 3 kept hanging overnight.

So, in typical programmer fashion, I decided I’d just cut out the middle man and write my own code to control the light bulb. Well, the scarcity of examples and lack of any tutorials made this a very frustrating effort. I never did get the Z-Wave bulb to respond to any of my own code. So when my maximum frustration threshold was exceeded, I decided to punt on writing my own code and give SmartThings a try.

Samsung has put together some pretty extensive, well written developer documentation. So I was initially very optimistic. I ordered a kit that included the hub plus a bunch of sensors, and quickly had the light bulb turning on and off in response to motion and doors opening. However, going from there to integrating with all my other automated lights and Alexa skills was again very challenging. I eventually was able to get everything integrated together by writing a SmartThings Device Handler and Service Manager that could together automatically detect and control my Patriot devices, but then what? In order to automate stuff, I had to write a bunch code, and organizing all this code quickly got out of hand.

In my googling about all this, I came across several posts that shared my experiences; SmartThings can control a lot of stuff, but using it to automate things required writing lots of code. Some of these posts described that Home Assistant was really good at solving that problem. I also read where the Home Assistant guy (folks?) are looking into using machine learning to establish automation rules. That’s the direction I’ve been trying to go. So maybe I’ll throw my hat into that ring.

So I’m back to working with Home Assistant. And I was delighted to find that in the several months since I looked at it before, installing Home Assistant has become simpler and easier.

There are options for connecting SmartThings sensors to Home Assistant using an MQTT broker, and I may take a look at that later. I’ll eventually want to be able to interface with Zigbee devices, and using the SmartThings hub is one option. But we’ll see.

Oh, and the Raspberry Pi hang? I finally figured out that it was caused by leaving an open SSH connection overnight. So now I exit SSH when I’m done using it, and haven’t had any more hangs.

Backup Internet for my Alexa and Particle.io Devices

I’m really enjoying using Alexa to control the lights in my home. One down side to this however is the requirement to be connected to the internet. Without an internet connection Alexa becomes a lazy idiot.

SmartThings, TPLink Nano, and iPhone

Since I live in an RV, I won’t always have a fast internet connection. So what I’ve done to deal with that is to have two separate hot spots, only one of which is ever active at any given time.

When my fast internet (was Roadrunner, will soon be Google fiber) is available and working, it transmits to all my devices.

When I’m on the road, or the fast internet breaks (like for the past 5 days!), I plug in a TPLink Nano Wireless N router. This little $35 box is very versatile, and can handle receiving and/or transmitting on Wifi and/or ethernet.

For example, as shown in the image, the SmartThings hub needs an ethernet connection. So the Nano is configured as a “Client”, to receive WiFi from the iPhone and convert it to an Ethernet connection for the hub.

Likewise, with the Nano connected to my normal WiFi router, I can configure the Nano to receive WiFi at whatever RV Park I happen to be staying, and distribute it to all my WiFi and IoT devices without having to reconfigure anything except the Nano.

SmartThings Control of Photon Devices Using Patriot

SmartThings

So having finally received my SmartThings hub and a few devices, I’ve spent the past week learning how to program it to interface with my existing Particle.io Photon controllers running Patriot. It turns out that SmartThings has a fairly nice architecture that made automatically discovering my existing devices fairly easy. I needed to write a Service Manager SmartApp and a child Device Handler. Altogether this was about 200 lines of Groovy code (basically Java). The service manage interacts with Particle.io to locate each Photon controller on my account, and then ask it what devices it supports. It then creates a child device for each using the name exposed by the Photon. Voila!

This is essentially the same approach used by the Patriot Alexa Smart Home controller that I published for Alexa.

So yesterday when I finally got the kinks worked out, I reinstalled the Patriot Service Manager SmartApp that I’d written, and it automatically discovered my two dozen Patriot lights and added them to SmartThings. Woohoo!

So now I just need to explore the Alexa implementation on SmartThings and see if my Alexa Smart Home skill is still needed. At this point it looks like it can be completely replaced by the SmartThings Alexa support.

Switching to Samsung SmartThings

After a frustrating few weeks of unsuccessfully trying to get Home Assistant working with my Patriot Particle.io devices and Z-Wave bulbs, I’ve decided to switch directions and won’t be investing any more time on Home Assistant. While I really appreciate all the hard work that folks have put into Home Assistant, it just isn’t a good fit me.

So I am switching gears and am investigating integrating Patriot with the Samsung SmartThings hub instead. This will give me both Z-Wave and Zigbee control, and some level of operation when disconnected from the internet.

I’ve read through much of the SmartThings developer documentation, which is very good. I like the architecture, and clear descriptions of how to integrate 3rd party hardware.

I ordered a SmartThings Monitoring Kit from Amazon that should be here tomorrow in time to have some of my Holiday time off to play with it. This kit includes the SmartThings hub, a couple door sensors (with vibration and temperature sensing), an a/c switched outlet, and a motion sensor.

In searching the developer forums I’ve located some posts by others that have been successful in integrating SmartThings with the Particle.io cloud, so I’m confident that in no time I’ll have it controlling the plethora of lights, fans, and awnings that I currently control using Particle.io Photons and Patriot.

Teaching Alexa What To Do

In the last post I described combining an Alexa custom skill with an Alexa smart home skill. Well, I got that working this past week and it works great. The combination really has some interesting potential.

Chihuahua and electronics
Just a cute little doggie watching me wire stuff together.

Smart home skills are great because they’re short and simple, and the supported device names can be extended dynamically through the “discovery” process.

Custom skills are great because they’re very flexible, and allow interactive dialog, unless something fits into a lot, isn’t extensible.

Putting the two together can provide the best of both. Here’s what a dialog with Alexa sounds like using both the Patriot smart home skill and the new Patriot custom skill.

Initially the skills don’t know about any activity called “computing”. So when I say:

“Alexa, turn on playing piano”, it responds with “Sorry, I didn’t find playing piano”.

That’s as expected. So let’s teach Alexa what we want it to do when we say “playing piano”. Note that I’ve made the invocation words “my lights”.

“Alexa, tell ‘my lights’ to turn on office when I say playing piano”

Then Alexa responds with:

“Ok, from now on when you say ‘playing piano’ I will set ‘office’ to 100 percent. This is a new activity, so you’ll need to tell me to ‘discover devices’ if you want to use the smart home command ‘start playing piano’.”

How cool is that?

So then saying “Alexa, discover devices” results in “Ok, found one new device ‘playing piano'”.

And then I can say “Alexa, start playing piano” and the office light comes on.

Leveraging Two Types of Alexa Skills

I’ve been progressively expanding my RV’s automation system. At this point I have 14 automated lights, and I expect that number to at least double over the next year.

Voice control using Alexa is working fairly well. Currently, I use the Patriot Alexa smart home skill to control both activities (aka scenes) and individual devices. And I think this is becoming a problem.

As the number of devices grows, it gets harder to remember the exact name of each specific light, and harder for Alexa to accurately hear the correct device or activity name. For example, I have lights on the front, rear, and ramp awnings. I’d like to be able to control them individually, as well as being able to turn them all on or off with a single activity name. It’s easy for me to say “Alexa, turn on the awning light” as I’m heading out the rear door, when what I really mean is the “rear awning”. Sometimes Alexa responds with something like “I have several devices with that name. Which do you mean?”.

So I’ve been thinking about how to simplify the vocabulary that I use with Alexa. I really like that Alexa smart home skills provide a very simple and terse vocabulary (eg. “Alexa, turn on xyz”). So I’m currently allowing it to control both activities as well as individual devices. One of the things I’ve realized is that as the number of devices grows, I rarely want to control individual lights. So I use activity names most of the time. The large number of individual device names just provide opportunities for conflicts with activity names, and misinterpretations. It would be nice if I could clearly separate device names from activity names.

Also, I’d like to be able to dynamically create and edit activities. A smart home skill certainly won’t allow this. So I’ve come to the conclusion that I also need a custom skill to provide that additional functionality.

At this point a light comes on in my head, and I think “why not use a smart home skill for the commonly used things, and a custom skill for the rest?” This equates to using the smart home skill to only turn activities on and off, and use a custom skill to control individual devices, and add or remove them from activities.

So this is the direction I’m going now. I’m going to remove individual device control from the published Patriot hobbyist smart home skill. This doesn’t mean it can’t control an individual device. It just means that an activity will be needed to do that.

I’ve already started on the custom skill, and have published the beginnings of it on Github. I’ll be expanding both it and Patriot to allow defining new activities, as well as allowing activities to be edited or updated using voice also.

Extending an Alexa Skill to Control Patriot Devices

In the last post I showed how to create an Alexa skill using the Alexa Skill Kit Command Line (ask-cli). In this post I’m going to show how to extend that skill to control the blue LED on a Particle.io Photon.

Prerequisites

This post assumes:

  1. You are already familiar with Photon development using the Particle.io IDE of your choice
  2. You have a particle.io account already setup.
  3. You have completed the previous post to create a starter Alexa skill using the ASK CLI.

Continuing Where We Left Off

By the end of the last post we had a working Alexa skill, although it didn’t really do anything interesting. Let’s change that and add some code to make it talk with a Photon controller using the particle.io API.

Let’s start by adding a couple intents to turn on and off a light:

Alexa, turn on the blue light

Alexa, turn off the blue light

Easy, right? We’ll add the intent first without it doing anything except responding “Ok”, then add the calls the the particle.io after that is working.

Basic Changes

Let’s start by changing the invocationName on line 4 of en-US.json to “my photon”. Now we can say “Alexa, tell my photon to …”.

Adding a New Intent

Copied intent

Adding an intent will require defining it in the en-US.json file, and then creating an intent handler in index.js.

So open en-US.json in your favorite editor, and copy the entire HelloWorldIntent. After the copy, it should look as shown here.

Next edit the two copies to make them be our turn-on and turn-off intents.

Note: we could have used a single intent with a slot for the on/off state, but I’m trying to keep things very simple. Plus this allows you to use slang or fun expressions such as “light me up” or “gee, I feel blue”, etc.

So now we should have a TurnOnIntent and a TurnOffIntent defined. Let’s delete the previous MyNameIsIntent. So the final file should look like this:

Edited en-US.json file

So now we need to handle the two new intents, and remove the deleted intent from the Lambda source. So open the lambda/custom/index.js file and make the following changes (or you can cut/paste the entire thing from the example below):

  1. Delete the ‘SayHelloName’ handler on line 29 through 34.
  2. Edit the ‘SayHello’ function to say something appropriate such as “You can tell me to turn on or turn off the light”. Update the cardRenderer() text similarly.
  3. Copy the same or similar text to the help intent.
  4. Delete the HelloWorldIntent and MyNameIsIntent
  5. Copy the SayHello intent twice
  6. Change the first SayHello copy to ‘TurnOnIntent’, and change the text to something like ‘Ok, the light is now on”, and provide more descriptive text in cardRenderer. The later is the text to be displayed on Show and in the Alexa app.
  7. Similarly, change the second SayHello copy to ‘TurnOffIntent’, and change the response and card text.

The index.js handler’s section should look similar to this after these edits:

Updated index.js

Now deploy the changes by entering “ask deploy” again. Test the updated skill by saying “Alexa, tell my photon to turn light on”. Make sure that you previously enabled testing in the Alexa skill test tab.

Verify that all of the intents work as expected, including:

  • Just launching the skill: “Alexa, open my photon”
  • Asking for help: “Alexa, ask my photon for help”
  • Turning on light: “Alexa, tell my photon turn light on”
  • Turning light off: “Alexa, tell my photon turn off”

If all that is working as desired, let’s move on to sending on/off commands to the Photon via the particle.io and Patriot open source library.

Adding the Particle Javascript SDK to our Project

So now on to the fun part. Alexa can now handle commands to turn something on and off, so we need to forward those requests to the particle.io API.

We’re going to use the particle Javascript SDK. I had posted code to Github a couple years ago called alexaParticleBridge using raw HTML calls, but since then Particle published their SDK that makes this a lot simpler.

So first we need to fetch and add the particle-api-js package to our project.

  • Edit lambda/custom/package.json to add the following line to the dependencies section:particle api dependency
  • Within the lambda/custom directory, run “npm update” to fetch the particle sdk modules.

Now we need to add the Particle object to our code. Include the Particle object in index.js by adding the following line near the top below the line that includes the alexa-sdk:

Code to include particle sdk

Setting Up Alexa Account Linking

Before we can call the methods of the particle object to communicate with our device, we need to give Alexa the credentials needed when calling the Particle API. We could hardcode our Particle account username and password into the skill, but that would be unwise and is totally unnecessary. Both Alexa and Particle.io provide support for using OAuth2.

Without getting into the messy details, what OAuth2 does in this case is to allow the Alexa skill to call a webpage provided by Particle.io. That webpage prompts the user to enter their username and password, and if valid will forward an access token to the Alexa skill that it can use to access the API. The Alexa code never has to know or even see the user’s credentials. How cool is that?

To get Alexa Account Linking working, we don’t really need to understand how OAuth2 works. We just need to know where to get the information needed. I’m going to step us through doing that.

Note: currently it doesn’t appear that the ask-cli supports uploading Account Linking information, so we’re going to need to enter the information directly into the Alexa developer website for the skill.

So open up a browser window to you Alexa developer account, and browse to the skill’s Configuration tab. The Endpoint information should already be filled in. Below that is the scary looking Account Linking radio button. Select Yes and a bunch of cryptic entry fields will appear. Let’s fill then in one-by-one:

  • Authorization URL: https://api.particle.io/oauth/authorize
    This tells Alexa the URL to display to prompt the user for their account login. This is a Particle.io provided website.
  • Client ID: particle
    This is provided to the Particle.io authorization URL. We will generate this in the next step, but for now enter “particle”.
  • Domain List (optional):
    Leave blank. It isn’t needed.
  • Scope: none
    Click on the Add domain button and add the scope ‘none’. It isn’t currently used, but may cause problems if something isn’t entered here.
  • Redirect URLs
    These are filled in for you. We don’t use them, so you can ignore them.
  • Authorization Grant Type: Auth Code Grant
    Select Auth Code Grant
  • Access Token URI: https://api.particle.io/oauth/token
    This provides Alexa with the URI to use when requesting or refreshing an access token. It is provided by Particle.io.
  • Client Secret: particle
    We will generate this in the next step. For now enter “particle”.
  • Client Authentication Scheme: HTTP Basic
    Leave this the recommended default.
  • Permissions:
    Leave these all unchecked.

Save this page. We’ll come back and enter the actual Client ID and Client Secret later.

Make a copy of one of the Redirect URLs. We’ll need it to create the client ID and secret in the next step.

Generate an Oauth Client

We need 2 things to create an oauth client:

  • Your particle.io access token
  • The redirect URL from the previous step

To get your particle.io access token, log into your particle.io account and go to the IDE. Select Settings (the gear icon in the bottom left) and assuming that you are logged in, the Access Token will be displayed near the top left. Make a copy of it for use in a moment.

Now we need to use your access token along with the Redirect URL saved in the previous step to create a valid Oauth client. These instructions assume you are running on a Mac and using the Terminal, but you could use any program capable of issuing Curl commands.

  • Open a Terminal or Command Line window
  • Issue the following command, substituting your redirect url and access token:
    • curl https://api.particle.io/v1/clients -d name=<your-skill-name> -d type=web -d access_token=<your-access-token> -d redirect_uri=<your-redirect-url>.

The response should include an id and secret. Make a not of these.

Warning: This is the only place that the secret will be displayed. Save it carefully. If you lose it and need it again later you will have to repeat the above steps to recreate a new id and secret.

Now go back to the Alexa Configuration page and enter and save the new Client ID and Client Secret you just created.

Whew! That should do it. Save this information, and restart your skill. When your skill is enabled in the Alexa app, it will display the Particle.io login screen where you can enter your user id and password. From then on, Alexa will automatically fetch a Particle.io access token that your code can use to access the Particle.io API.

Using the Access Token

Now that we’ve setup account linking, Alexa will pass an access token to our code with each request. It will be passed in the event session information event.session.user.accesstoken. Let’s update our code to get and save it.

var token = this.event.session.user.accessToken;

Sending Requests to Particle.io

Now all that’s left is to use the Particle object created to to send requests to our Particle devices using the token obtained above.  So lets create a function to call the Particle API Publish function:

publish function

Now we can add calls to this method in our intent handlers, and Publish events to our Particle devices. Here’s the TurnOnIntent and TurnOffIntent will the calls to publish():

TurnOnIntent and TurnOffIntent code

Note that a promise is used, and that emit is only called after the promise resolves. Also, the arrow function is used to allow “this” to continue to point to the response object.

So at this point we have a working Alexa skill that Publishes events to our Particle devices. In this example we used the device name “blue”. This needs to match the name you gave a device or activity in your Patriot code. I’m not going to duplicate the information about using Patriot, since I’ve posted several articles about that already. And of course you can call other particle API methods in addition to publish.

So this example should give you enough information to make function calls, set or read variables, and so forth. The complete project code is on Github. I hope you have as much fun with it as I have.

Caveat: I’m very much an intermediate Javascript programmer at best. I’m working with and learning from some really good Javascript programmers, but I’ve got a long way to go. Please don’t use my code as an example of the correct or best way to do things in Javascript.