Colin Hickey has an interesting YouTube channel that features some of his cool home automation projects. What I like about his videos is that he doesn’t just show the end result: he goes through his setup step by step so that you can do the same thing yourself.
If you go back through his videos you’ll find plenty of fascinating information about his home-made PowerWall and battery management system.
His latest video should be interesting to anyone who has an iLife robot vacuum cleaner, so check it out and make sure you subscribe to his channel:
WiFi is a critical part of most home automation systems, but many people simply install the WiFi router that came with their Internet plan and forget about it. However, setting up a good home WiFi network that will handle dozens of IoT devices and still keep your home secure takes a bit more work.
The typical home network uses an all-in-one router to provide many different network services, including:
Router
Firewall
WiFi access point
Ethernet switch
Media streaming
VPN server
DNS resolution
DHCP server
Printer sharing
File serving
… and more
This usually results in a network topology that looks like this:
All-in-one WiFi routers try to do too much, so they end up being terrible at everything.
Well structured networks generally separate critical services into different devices that are optimised to do that job, and do it well. Access points don’t try to be file servers, and DHCP servers aren’t Ethernet switches.
Learning from the way that small corporate networks are structured is a great way to improve your home network and improve its reliability, performance, and security.
A better way to structure a home network is like this:
To convert the typical network into the improved network, we’re going to make changes in a series of steps.
For this tutorial I’m using parts from the UniFi range from Ubiquiti Networks, which is a range of networking products designed to be installed across small to medium corporate networks. It uses a management system that provides a single interface to control all your devices, even if they’re installed in physically separate locations.
Shopping list
The specific devices that I installed included:
1 x UniFi Security Gateway
2 x UniFi UAP-AC-Pro access points (probably should have got Nano HD though)
1 x Raspberry Pi 3 B+ (or you can use a UniFi Cloud Key)
1 x UniFi US-8-60W 8-port managed Ethernet switches (not shown in this episode: will feature in Part 2)
2 x UniFi US-24 24-port managed Ethernet switches (not shown in this episode: will feature in Part 2)
Step 1: Set up management system
Ubiquiti Networks provides management software called UniFi Network Controller. It has a web interface, so you run the UniFi Network Controller software on a computer somewhere on your network (or even outside your network) and then access it using a web browser.
The UniFi Network Controller can be run in three different ways:
On your local computer. You can download the software and run it on your Mac, Windows, or Linux computer. This is a great way to test it out and see how it works, but if your computer is turned off the management software will also stop running. If you have a computer that you leave on all the time, such as a home media server, you can run the UniFi Network Controller on it. Download it for your computer from Ubiquiti Networks.
On a UniFi Cloud Key. This is a tiny embedded computer a bit like a Raspberry Pi, which comes with the UniFi Network Controller software preinstalled. You just plug it into your network, open a web browser on your own computer, and log into it. The Cloud Key is a low-power device so you can leave it running continuously.
On a Raspberry Pi or other low power computer. This is what I’m doing. A board such as a Raspberry Pi B+ has plenty of power to run the UniFi Network Controller and still do other things as well, such as run PiHole or other services. Effectively this is just like a DIY version of the Cloud Key, but it’s cheaper and you can do other things with it.
After installation, make sure your Raspberry Pi is connected to wired Ethernet (not just to WiFi) and log in to the Raspberry Pi as the default “pi” user. The default password is “raspberry”.
Set a static IP address so that the Raspberry Pi will always boot up with the same address, to make it easier for you to log into it in future. Do this by editing the file /etc/dhcpcd.conf:
sudo nano /etc/dhcpcd.conf
Go to the bottom of the file, and add lines similar to this:
Those particular settings suit my network, which has the gateway running at 192.168.1.1 and also providing DNS resolution. You may need to change the values to suit your network.
Press Control-X to exit, then save the file.
Now run the “raspi-config” utility:
sudo raspi-config
Use the menu system to select “Change User Password”, and set a strong password.
Then go into the “Hostname” section, and change the hostname from “raspberrypi” to something more specific. I changed mine to “controller“.
Exit from raspi-config, and allow it to reboot.
A minute or so later your Raspberry Pi should come back up on the network. Now it will be at the IP address you specified above, it will have hostname you set, and the password will have changed.
Install the haveged daemon to provide a good source of entropy. This isn’t essential, but it will help the UniFi Network Controller boot up faster, because it will be able to generate cryptographic keys more quickly:
sudo apt install haveged -y
Install a version of Java that works nicely with the Controller:
sudo apt install openjdk-8-jre-headless -y
Add the Ubiquiti Networks software repository and GPG key:
Those lines above may have wrapped in the browser. It should be 2 lines, with the second line starting with “sudo”.
Now update the list of available packages, and install the UniFi Network Controller package:
sudo apt update sudo apt install unifi -y
In future you can update the Controller package just like you would with any other Raspbian package. If you regularly update your Raspberry Pi using the following commands, it will stay up to date along with all the other packages:
sudo apt update sudo apt dist-upgrade
Step 1d: Log in to Controller software
After the installation finishes and the Controller software has finished booting, you can log in using a web browser. Use your browser to open the IP address you configured for your Controller, but make sure you prefix it with “https” and use port 8443. With the example configuration I gave above, the URL is:
https://192.168.1.2:8443
Your browser will complain because it doesn’t trust the SSL certificate being used by the Controller. You can click through and load it anyway.
Step through the setup wizard and answer each question. There aren’t many questions and they’re fairly simple to work out. You can skip the WiFi configuration and do it later, as I show in the video.
Click around and explore the Controller interface if you like. There won’t be anything much happening in it because there aren’t any devices configured.
Step 1e: Plan your access point placement
The UniFi Network Controller has an interesting feature that lets you create a plan of the physical layout of your house, and place network devices on the plan. It can then use obstructions such as walls to do estimates of the signal strength in different parts of your house.
Go to the “MAP” menu item on the left, and you’ll see an example map. Edit the example or create a new map and upload a plan of your house, then use the Map Designer tool to draw lines where all your walls are located. Place access points, turn on coverage display in the “Layers” menu, and experiment with different locations for your APs.
Step 2: Replace all-in-one router with dedicated router
If your Internet connection uses a technology such as ADSL, or Cable, or optical fibre, you need to have it terminated in a way that gives you a simple Ethernet connection. A device that does this is called a Network Termination Unit, or NTU. The NTU is specific to the type of connection that your ISP provides.
In some cases, your ISP will have provided an NTU so that you can simply plug your new router straight into it. If that’s the case, you can skip ahead to installing your dedicated router. I’m going to use a UniFi Security Gateway (USG) as my new dedicated router for the rest of this example.
I have a Cable internet connection that comes in on coaxial cable, so it needs a device to convert the coaxial connection into an Ethernet connection. Instead of using an NTU, I used the original Telstra Gateway Max that was provided by my provider, and changed its configuration to put it into “bridge” mode.
Bridge mode is a simplified mode that turns off almost all the features of your existing router and turns it into a dumb adapter that just passes data through without trying to manage it. Effectively this is like taking a router and giving it a lobotomy to turn it into an NTU. This will also disable any WiFi functionality, so it won’t be useful as an access point anymore.
The specifics of how to do this depend entirely on your ISP setup and your existing router, so I can’t provide instructions here.
Once you have converted your ISP’s router into a bridge or replaced it with an NTU, it won’t provide any of the normal services that you need such as routing and DHCP. That will now by done by your dedicated router.
Disconnect everything from your existing router except the ISP connection.
Use an Ethernet cable to connect the “WAN” port of your new router to one of the “LAN” ports of the old router. The connection will then go:
Your ISP -> Your NTU / bridge -> New router [REPLACE WITH IMAGE]
Connect from the “LAN” port of your new router to an Ethernet switch, so that you have somewhere to connect wired devices to your network. This will include the Raspberry Pi that is now running the UniFi Network Controller software!
ADD IMAGE HERE
Turn on the new router, wait for it to start up, and then go back into the management interface on the Controller. If you’re using a UniFi compatible router such as the UniFi Security Gateway (USG) that I show in the video, the Controller should now see that you have a new device on the network. You can “adopt” the new device, which sets up a secure communication link between the Controller and the USG.
The rest of the setup of the gateway is done through the Controller interface. There’s no need to log in directly to the USG itself, because it receives its configuration from the Controller.
Step 3: Install dedicated access points
Ubiquity specialise in WiFi, so they have a huge range of access points available. In my case I stuck to the UniFi range because I wanted APs that could be managed using the UniFi Network Controller. I chose UAP-AC-Pro models because they seemed the best for my needs, but I’ve been told that I probably should have got the nanoHD model instead.
The particular APs that I got use PoE (Power-over-Ethernet) instead of plugging in a power supply directly. This means you can run a single Ethernet cable from your Ethernet switch to your AP, and the AP will take its power from the cable.
If you have a PoE Ethernet switch, there’s nothing else you need to do. Just plug in the cable, and the AP will receive power.
If you have a non-PoE switch, you can use a POE injector mounted near the Ethernet switch as an intermediary, to provide the power source. My APs came with PoE injectors included in the box, but some models of UniFi AP can be bought both with and without the injector in case you already have a PoE switch.
If you want a bit more background on Power-over-Ethernet, I did an introduction to it way back in episode #3! Check it out here:
Once your APs are plugged in and powered up, go back into the UniFi Network Controller interface to find them listed as new devices. You can then adopt them, just like you did with the Security Gateway.
However, don’t configure them directly unless you want to achieve something specific. The simplest thing is to leave them on default settings, which allows them to inherit the settings that you can apply to the entire site. In this context, “site” means your physical premises. The UniFi Network Controller is designed to scale up to installations with many devices across many physical sites, such as a group of remote branch offices, all managed through one central interface. For domestic use, you’ll probably only have one site.
To create a WiFi network that will be advertised through your fancy new access points, go to the “SETTINGS” menu item in the very bottom left of the Controller screen, then select “Wireless Networks“.
Click “CREATE NEW WIRELESS NETWORK“, and enter the details:
Set “Name/SSID” for the WiFi name that you want.
Leave the “Enabled” box ticked. Obviously!
Change the “Security” setting to “WPA Personal“. This is the common authentication type that most domestic WiFi networks use, with a single shared password for all devices.
Type the WiFi password that you want to use in the “Security Key” box.
Leave “Guest Policy” un-ticked, unless you’re specifically creating a guest network. Your first WiFi network will almost certainly be a normal network, not a guest network.
You don’t need to worry about the ADVANCED OPTIONS section for now. We’ll be coming back here in Part 2, when we look at more advanced things that you can do to improve the security of your network and provide compartmentalisation for troublesome devices.
Click the green SAVE button, and your new WiFi network will be created. This will also be automatically applied to all the access points that you have linked to your Controller, so it will become visible and you’ll be able to connect devices to the new network.
That’s all for now, but come back for Part 2 where I’ll show you how to apply more advanced concepts to really make your home network the best it can be.
Defeat package thieves with a combination of technology and steel!
Having packages stolen from your doorstep can be a big problem in a normal suburban neighbourhood, but when your front gate is at the end of a very long driveway in a remote area it’s even worse. Your packages could sit exposed by the side of the road for hours when you don’t even know they’ve been delivered.
SuperHouse forum member Guru_Of_Nothing was sick of package thieves taking his deliveries, so he’s started building a super-strong, high tech package vault:
The design is very clever because it covers various contingencies, including being able to open the box mechanically if there is a power failure, and handling multiple deliveries in a single day.
One of my pet peeves is devices that operate physically within my own house or network, but require an external cloud service to control. There are a big list of reasons why it’s a really bad idea to make your Internet connection a critical part of the control system for the devices around you, which is why a blog post by RevK made me want to stand up and cheer.
I’ve ranted about this on video before, but RevK’s post is definitely worth reading because he lays out the problems and also his plan to “de-cloud” his devices. This is a great idea, and I hope the term “de-clouding” starts to pop up more regularly.
Is it true that DIY electrical work can invalidate your home insurance policy? Can you go to prison for replacing the plug on a power board?
In Australia: yes. With the harshest restrictions in the world, Australian regulations don’t let you do anything unless you complete a 4 year apprenticeship, complete some certifications, and become a licensed electrician. It doesn’t matter that I’ve designed satellite payloads and that products I’ve designed have had more than a million units manufactured. Without completing a 4 year apprenticeship, there is no legal path for me to become certified to replace the plug on a power board.
Crazy.
And to top it off, clauses in your insurance policy that don’t seem to have anything to do with electrical safety can allow your insurer to refuse any claims that could be related to DIY electrical work.
If you’re into home automation, check out Rob’s YouTube channel, The Hook Up.
Make the Sonoff smart power controller even more awesome by installing the Tasmota open source firmware.
Tasmota adds many new features, and allows you to integrate Sonoffs into an existing home automation system without relying on external cloud services. It includes MQTT support, Belkin WeMo emulation, easy configuration using a web browser, and you don’t even need a compiler or IDE to install it.
Note: Since this video was released, the Tasmota project has changed its build process so that the binaries are now called “tasmota” instead of “sonoff”. I have edited the text instructions below to match, but screenshots and the video still refer to the old name.
To load new firmware onto a Sonoff, you need four pieces of hardware:
A compatible Sonoff model or equivalent (more than 50 devices are supported by Tasmota)
A USB-to-Serial converter that can run in 3.3V mode (don’t use a 5V converter!)
Jumper wires or a programming adapter to connect the USB-to-Serial converter to the Sonoff
A computer with WiFi. You can use a mobile phone for the WiFi steps if you prefer.
You can make the connections using jumper wires and the instructions in my previous video, or you can use my handy little programming adapter. Both methods do exactly the same thing: my programming adapter just makes it neat and easy.
Make sure the Sonoff is totally disconnected from any mains power.
Don’t proceed unless you have done that! Connecting your computer to a Sonoff while it is connected to mains is extremely dangerous.
Details of the electrical connection are shown in the previous episode, so check that out if you need more information.
Step 2: Download Tasmota
The Tasmota firmware and its documentation is available at github.com/arendst/Sonoff-Tasmota, including both firmware and pre-compiled binary releases. If you want to compile the code yourself that’s fine, but you don’t need to if you just want the latest version. The binary releases are at:
There is a big list of binaries for each release, which can be confusing if you don’t know which one you need. The binaries fall into several categories which are outlined on the releases page.
For most devices, you should choose the binary called “tasmota.bin” which includes all the features required for the majority of compatible hardware.
The binaries with 2-letter country codes appended, such as “tasmota-IT.bin”, have the exact same functionality as the standard “tasmota.bin” but with different languages for the user interface. If you want to run Tasmota in a language other than English, select the appropriate “tasmota-XX.bin” file.
Whichever version you select, download it to your computer.
I prefer to rename the binary file so that it includes the release version, such as “tasmota-8_4_0.bin”. That way if I come back to it later, I can see what version I downloaded.
Step 3: Download and install esptool
Esptool is a utility that can be used to read and write the flash memory on devices using the ESP8266 and ESP8285 microcontrollers, including all current Sonoff models. It’s written in Python so you will need a working Python environment on your computer.
Follow the instructions on that page to install it on your computer.
Step 4: Find serial device
Your USB-to-Serial converter connects to your computer by USB, and appears to the computer as a virtual serial port. The exact location of the virtual serial port will vary depending on the type of converter that you have, what operating system you use, and which physical USB port you plug the converter into.
If you have the Arduino IDE installed, a good way to find the location of the virtual serial port is to start the IDE, look at the ports list in Tools -> Port, then plug in the converter and check the ports list again to see if a new port has appeared.
On my Mac, the port appeared at /dev/tty.usbmodem14201, so I’ll use that in the following examples.
Step 5: Put Sonoff into bootloader mode
Disconnect power from the Sonoff, hold the GPIO0 pin low, and reconnect power. Then release GPIO0. This process is explained in detail in the previous video.
This will put the Sonoff into a mode where it waits for new firmware to be loaded.
OPTIONAL Step 5b: Back up original firmware
If you want to make a backup of the original firmware so you can restore it back to factory-original later, jump to the addendum near the bottom and then return here. Otherwise, carry on!
Step 6: Upload the Tasmota firmware
Open a terminal, and go into the directory where you have the Tasmota binary. Use esptool to push it to the Sonoff using the serial port location that you found earlier:
The “write_flash” command tells esptool to send new data to the flash memory
The “-fs” parameter is the flash size. Put this parameter in if you know it. If you don’t, you can omit this parameter and esptool will attempt to auto-detect the flash size for you
The “-fm” parameter is the flash mode to use
The “0x0” parameter tells esptool to start writing into flash from address 0
The final argument is the binary image to load
Uploading the binary takes about 30 seconds.
While esptool is doing the upload it will report useful information about the hardware in your Sonoff, including the MAC address. This can be extremely useful if you want to put a DHCP static lease into your router fix your Sonoff to a specific IP address. You can copy and paste the MAC address out of the terminal and save it in a document or spreadsheet for future reference if you want to.
Step 7: Connect to Sonoff via WiFi
Disconnect the Sonoff power power, then reconnect it. When the Tasmota firmware starts up and doesn’t find any existing configuration, it goes into a setup mode and creates its own WiFi network. The network will have a name similar to “Tasmota-6392”, with the 4 digits based on the last part of the unique MAC address of the Sonoff hardware.
On your computer or smartphone, go into WiFi settings and look for a network with a name similar to this. Make a note of the network name, because you will need it in a moment!
Connect to the Sonoff’s WiFi network without any username or password. Your computer should then automatically open a captive portal, with a WiFi configuration screen for the Sonoff.
Enter the WiFi settings for your normal network into the Sonoff. You can enter them directly, or you can click the “Show wifi networks” link to have the Sonoff scan for available networks so that you can select the right one and then enter the password manually.
Enter the details, and click “Save”.
The Sonoff will then reboot and attempt to connect to your normal WiFi network.
This will cause your computer to drop off the temporary WiFi network that was created by the Sonoff, and reconnect to your normal network.
Step 8: Connect to Tasmota configuration interface
Open a web browser, and enter the address of the Sonoff using the network name that you noted in Step 7 with “.local” added to the end. For example, the address could be something like
tasmota-6392.local
This will open a configuration interface where you can view information about the device, change its settings, backup and restore the configuration, install firmware updates, and control the output.
Many of the configuration options are explained in the video. Explore the menus and set up the Sonoff to suit your home automation system.
Addendum: Reconfiguring from scratch
If your home WiFi changes or you take the Sonoff to another location, it will keep trying to connect to the network that you originally configured and you won’t be able to control it.
If this happens, you don’t need to re-flash Tasmota. Instead, you can put it into WiFi recovery mode.
Tasmota can use the button on the Sonoff to put it into special modes. If you quick-press the button 4 times, it will restart the setup process and create its own WiFi network again just like in step 7. Then you can connect to its WiFi, and give it new details so it can connect to your main WiFi network again. Your other settings will be retained so you don’t need to set them again.
If you press and hold the button for more than 40 seconds, Tasmota will clear all its settings, reboot, create a WiFi network, and begin the setup process from scratch.
Addendum: Backing up and restoring original EWeLink firmware
If you want to keep the option of returning the Sonoff to factory-original condition, you can download the original firmware out of its flash memory and save it for later use. Obviously this has to be done BEFORE you flash Tasmota onto it!
Do everything up to Step 5, but DON’T continue with Step 6. Instead, use esptool to determine the flash size of the Sonoff. Using your own serial port address, run a command like this:
esptool.py -p /dev/tty.usbmodem142101 flash_id
This will give a result that looks something like this:
jon@Jonathans-MacBook-Pro:~/Downloads$ esptool.py -p /dev/tty.usbmodem142101 flash_id
esptool.py v2.6-beta1
Serial port /dev/tty.usbmodem142101
Connecting….
Detecting chip type… ESP8266
Chip is ESP8266EX
Features: WiFi
MAC: dc:4f:22:2b:c8:63
Uploading stub…
Running stub…
Stub running…
Manufacturer: e0
Device: 4014
Detected flash size: 1MB
Hard resetting via RTS pin…
You can see the result “Detected flash size: 1MB” near the end.
The Sonoff will have reset back into normal mode, so put it back into bootloader mode in the usual way, then use a command like this to download the existing flash memory and save it in a local file:
Pay attention to the value “0x100000” above. That example will read the flash starting at address 0, and continuing on to the 1MB point. If you have a 2MB flash, you’d use “0x200000”, for example.
On my laptop, it looks like this:
jon@Jonathans-MacBook-Pro:~/Downloads$ esptool.py -p /dev/tty.usbmodem142101 read_flash 0x0 0x100000 sonoff-ewelink.bin esptool.py v2.6-beta1 Serial port /dev/tty.usbmodem142101 Connecting…. Detecting chip type… ESP8266 Chip is ESP8266EX Features: WiFi MAC: dc:4f:22:2b:c8:63 Uploading stub… Running stub… Stub running… 1048576 (100 %) 1048576 (100 %) Read 1048576 bytes at 0x0 in 94.0 seconds (89.2 kbit/s)… Hard resetting via RTS pin…
After it’s finished, you will have the original flash memory saved on your computer in a file called “sonoff-ewelink.bin”.
Then continue with installing Tasmota.
Later, if you change your mind and want to put the original firmware back in place, you can follow the normal steps for installing Tasmota but instead use the file you saved. The command will look similar to this, but with your serial port address substituted:
I spent last week in Christchurch, New Zealand, for the always amazing linux.conf.au conference. It’s my annual pilgrimage: I’ve attended every single LCA since 2002, and this year I presented a talk about my work using DIY home automation projects to help a couple of old friends who have Duchenne Muscular Dystrophy and rely on electric wheelchairs for mobility.
This was a very difficult talk to give because it tells a story that is both tragic and uplifting. The projects I’ve worked on with Nick and Chris Fryer have been some of the most rewarding things I’ve ever done, but along with that came the terrible grief of Nick’s death last year.
Many people have the preconception that home automation is “expensive toys for rich lazy people”, but this talk shows that it’s far more than that.
If you personally benefit from the use of home automation, or have friends or family members who do, I’d love to hear about it. Please comment here or post in the forum, but of course please be respectful of privacy and don’t reveal personal information about anyone else unless they are OK with it.
