Smart control and security using Augmented Reality

/Smart control and security using Augmented Reality
Total score : 4.5 (9 / 2)
x 0


Ever imagined turning on light using your smartphone? Now you can , with the help of Augmented Reality(AR).Point your smartphone at the target image, say the light bulb in your home. The AR app in the smartphone will display control buttons once it detects a target image. Click on the buttons to control your home devices. We integrated Unity an Augmented Reality app with Wizwiki W7500 to make this possible. The app sends data to firebase , we have a python program to fetch this data from firebase and publish it to MQTT. Then Wizwiki can subscribe the data and accordingly control the devices. Another application would be to authenticate users. Users enter credential and this is stored in firebase. Python program fetches it and publishes it in mqtt. The w7500 subscribing data from mqtt  and accordingly blinks led(if it is a valid user) or buzzing buzzer(when an invalid user enters or wrong credentials is entered) . Here the target image is the door with push written.

ARM Embedded
Project participants

Project summary

Home automation has been done through many technologies . Here this projects explores augmented reality to control any appliance. An advantage to augmented reality is that it creates virtual switches to any target image of appliances. the project shows user authentication using Unity(Access Control)

Project Content

Augmented Reality application : Augmented reality is an interactive experience of a real-world environment whereby the objects that reside in the real-world are “augmented” by computer-generated perceptual information, sometimes across multiple sensory modalities, including visualauditoryhapticsomatosensory, and olfactory

AR interacts with real time appliances virtually. Unity is used to make the app for Augmented reality. Currently our AR app has 2 features.  The first one being controlling light, fan , door locks. Secondly to authorise user, the AR app asks for user credentials and checks with it’s database in firebase if it is matching the user is authenticated . 

The basic architecture of this project is shown in the figure given below.

Firebase: Firebase is a mobile and web application development platform developed by Firebase, Inc. in 2011, then acquired by Google in 2014. Firebase provides a realtime database and backend as a service. The service provides application developers an API that allows application data to be synchronized across clients and stored on Firebase’s cloud. The data sent from unity d is stored in firebase. 

Python program : fetches this database from firebase and publishes in MQTT:

WIZ w7500: Subscribes the data from MQTT and control the physical devices. This is how you virtually control physical devices.

Unity App:  Open the app and point the camera towards the target image (the device that you want to control) and buttons would appear . Below are the steps to install Unity 3D,Vuforia,Android SDK , Firebase SDK.

UNITY 3D And Vuforia Setup

  • Download Unity3D for Windows and install it
  • Go to
  • Register and login
  • Download the Vuforia unity extension package from Downloads
  • Go to develop and in license manager click on get development key 

  • Next in target manager click on add database
  • Add target images, videos or sound clips for your project and then download the database.

  • Open Unity 3D and create a new project
  • Go to Assets àImport Package à Custom Packages
  • Import Vuforia extension packages and the database you created on Vuforia
  • Go to Game object àVuforia and add AR Camera and Image Target
  • Delete the Main Camera
  • Go to File àBuild SettingàSelect platformà click on Player Settings

  • In player settings go to XR Setting and click on Vuforia Augmented reality
  • For Android Application Other settings need to be modified as follows:
  • If you want to include firebase then download Unity firebase package and import the package in Unity
  • Also change the Api Compatibility Level in Other settings to .NET 2.0
  • To Add object to your target image, go to Assets and click on import new assets and select the particular 3D object.
  • We can also download 3D obj from Assets Store and import it in Unity.
  • Click on assets folder à Vuforia à Prefabs to include features like cloud, text reorganization, virtual button and other objects which were imported in unity etc

    • Create a Firebase project in the Firebase console, if you don’t already have one.
    • Click Add project and enter a project name to create a new project.
    • Open Unity3D and create a project
    • Download the firebase setup SDK file for Unity from
    • For android configuration ,download google-services.json
    • Select the Assets > Import Package > Custom Package menu item.
    • Import the FirebaseDatabase.unitypackage package from the Firebase Unity SDK, downloaded previously.
    • The following code should be added at the start of your application to check for and optionally update Google Play services to the version required by the Firebase Unity SDK before calling any other methods in the SDK.

    Firebase.FirebaseApp.CheckAndFixDependenciesAsync().ContinueWith(task => {
      var dependencyStatus = task.Result;
      if (dependencyStatus == Firebase.DependencyStatus.Available) {
        // Set a flag here indiciating that Firebase is ready to use by your
        // application.
      } else {
          “Could not resolve all Firebase dependencies: {0}”, dependencyStatus));
        // Firebase Unity SDK is not safe to use here.

    • Select the File > Build Settings menu option.
    • Select Android in the Platform list.
    • Click Switch Platform to select Android as the target platform.
    • Wait for the spinner (compiling) icon in the bottom right corner of the Unity status bar to stop.
    • Click Build and Run.
    • To set users DatabaseRef.Child(“users”).Child(userId).Child(“username”).SetValueAsync(name);
    • To simultaneously write to specific children of a node without overwriting other child nodes, use the UpdateChildrenAsync()
    • The simplest way to delete data is to call RemoveValue() on a reference to the location of that data.

      • Using Cloud Messaging you can release new content and send push notifications to players who have completed your available content.
      • Remote Config allows on the fly adjustment of gameplay settings like enemy density, power-up frequency, or anything else you want to control.
      • Realtime Database stores user-generated content like custom maps safely in the cloud.
      • Cloud Storage delivers downloadable content to your players using our secure service.
      • Cloud Functions lets you run backend code in response to events triggered by Firebase features and client requests.
        1. Interface Google Firebase with Augmented Reality using Unity

        The purpose is to make an Augmented Reality home application which can send and receive data from firebase inorder to provide easy interaction with the devices in the room.

        Follow the steps below to interface firebase into Unity:-

        • Open firebase console , if you don’t have an account then sign up and login . Else directly login and add a new project.
        • Assign name to your project and click on create project.
        • Go to docs àclick on Get Started with Unity àDownload SDK.
        • The SDK contains all the setup for firebase.
        • Open Unity and import the package you want to use from the firebase SDK file.
        • Go to firebase console and click on Add firebase to your Android app.
        • Register your app with the same name as assigned to the Bundle Identifier in Unity, which is given in the form :
        • Download the firebase project .json file and import it to Unity.
        • Refer for more projects and scripts
        • For Email Authentication in firebase the following script was used
    • Below is the screenshot of the app when it sees a target image
  • The screenshot of Firebase  database when user enters wrong password is given below.
  • The screen shot of the serial monitor cool term displaying the data it subscribed from mqtt is given below.
    • The data  is fetched from firebase and then published in MQTT using python program(server) . The code’s github’s link is provided below.
    • The mbed code used for subscribing data from mqtt  and accordingly blinking led(if it is a valid user) or buzzing buzzer(when an invalid user enters or wrong credentials is entered) is published in Github.
    • The target image in this project is a door with PUSH written on it. Similarly the target image can be a bulb and instead of blinking led we can turn ON and OFF the light. 
1 420 1

One Comment

  1. minnu September 23, 2018 at 12:03 am - Reply video link for the project


Leave a Reply