IoT inspired smart home application using HTC Vive with Unity that allows users to control smart home devices through virtual reality.

Overview

The application is a mixed reality experience that allows users to connect and control their smart home in fun and unique ways. The application uses a TCP client-server connection between Unity and multiple ESP8266 wifi microcontrollers. The ESP8266 modules control various home devices such as a light, fan, and speakers. Music is controlled with a laptop running a Processing script which connects to the Unity server.

Development

The specific brand of ESP8266 I decided to use is the IZOKEE D1 Mini which is almost identical to the Wemos D1 Mini. These boards are a smaller version of the regular Wemos board so it has less pins but it was still more than enough for what I needed. After downloading the ESP8266 library, I was able to use the Arduino IDE to write and upload code to the boards. I referred to the pinout diagram below to figure out how to initialize the proper pins in Arduino.

I was able to control an LED strip through the ESP8266 using the FastLED library. The ESP, with everything connected, can be seen below. I used a relay attached to an extension cable to control the fan, but any on/off device plugged into the cable is also able to be controlled. To allow for music playback in the scene, I wrote a Processing script on my laptop which connects to Unity as a client and can play back selected files.

I wanted to create fun ways of interacting with the devices so I opted to use a bow and arrow mechanic to turn on the devices. Additionally, I added UIs to the controllers that allow users to choose the color of the light, turn off devices, and play music. I used basic cubes in the scene as virtual switches which are activated by the arrows. A basic interaction diagram can be seen below.

Interactions