noun A technology that superimposes a computer-generated image on a user's view of the real world, thus providing a composite view.
Some examples of augmented reality apps include:
- Pokemon GO
- Ikea - Ikea released an app where you can experiment with different pieces of furniture in your home to see if it goes. You can place multiple objects and capture how a room looks with these in.
- Google translate - if you hold your phone over a word or sentence then it will translate it in place.
Augmented reality (AR) enhances a users experience in the 'real world', whether that's through the things that we see, hear or feel.
There are different types of AR:
- Marker based AR
- Projection based AR
- Markerless AR
- Superimposition based AR
After attending The Full Stack Fest in Barcelona and being inspired by Shagufta Gurmukhdas's talk - Web Based Reality, we were inspired to have a play.
Whilst in Barcelona with my three fellow developeres, we sat on the balcony with a few Estrella's and played with A-frame. We challenged one of the developers to a micro-hack, to build out a 3D model in 10 minutes. This just proved how quick and easy it was to use.
A- frame is a WebVR framework that Mozilla released just over a year ago. A frame is an open source web framework with open standards and an open culture, it's made it easy for building out WebVR apps.
This is all the code that was needed to create the scene and add a few bits and it only took a few minutes.
Setting up was really easy to do.
- Add the scene
- Add the marker
- Add the camera
- Add the different entities - sphere / cylinder etc
https://gist.github.com/c2b87ece5fc6ff9e4109480f64396e72
The a HTML-based markup language has three.js under-the-hood, enabling developers to create 3D scenes using only HTML making it accessible and effortless for almost any developer. The quickstart framework allows developers to create fantastic, cross platform, cross device for AR / VR experiences on the Web for anyone. Three.js can be hard for developers to setup, there are a lot of gates to get through when building for AR / VR, a lot of bulky software with complex set ups to just get set up to start development, this is before thinking about multiple devices. Developers have to setup camera, intialise the scene, set up the lights, declare and pass canvas, import WebVR polyfil, install VREFFECT, instatiate renderer, create render loop, preload assets, figure out reponsiveness, deal with metatags on mobile.AR / VR apps can often be unefficient and inconvenient for the end user, often they need to download apps or other things to make them work. This isn't the case with A-frame.
A-frame cuts out all of the complex set up and within about just 15 minutes it's possible to add entities to a scene, style them and animate them to move around the scene. It's perfect for a developer whom wants to concentrate on having an easy to use playground and get results quickly.
The out of the box functionality that A-frame offers includes camera, geometry, material, light, and sound. It comes ready for stereo viewing in VR headsets and with WASD / arrow movement and camera panning / rotation support. We aren't utilising those attributes of A-frame in this particular project but it's an example of how easy it is to use and set up. All developers have to do is write some HTML in the scene, you can scale add positon and rotate each of the entities on the scene. To enhance this you can use regular javascript to create animations.
Naturally, due to the simplicity A-frame, it does mean that developers will eventually hit a wall with what they can achieve, saying this there are some quite complex prjects out there and it doesn't take away how valuable A-Frame is. A-frame has even more potential and has already proven itself to be extendable one example is Jerome Etienne's library ar.js - I will go into more detail about this shortly. A-frame has demonstrated itself to be hugely popular (and growing), with a great , thriving community.
A-frame has a built in inspector which you can enable by:
https://gist.github.com/ea99e29675d4e164856096d2edf01076
This inspector is extremely useful, when first using A-Frame the positioning, rotation and radius seemed a little tricky and confusing, very sensitive. It was proving a little tricky to try and achieve the desired results through changing the values in the code.
https://gist.github.com/49c8ec71560ea4aa00f8556c8baef6a7
Opening up inspector in the browser makes it a lot easier and way more efficient to move entities around. You can physically move each of the entities around, when doing so you can see the values for the X, Y and Z for the position and rotation. When each of the entities are in the correct place, one can copy the numerical values and put them in the code. One can also change the scale
IMAGE HERE.
As I briefly mentioned earlier @jerome_etienne (the author of Learning Three.js Blog), wrote a library - ar.js in order to get AR running 100% into the web browser, meaning no app needs to be installed and it's supported across all platforms and devices, also it can run up to 60 frames per second (fps).
It's a great library and made it really easy for us to be able to achieve what we wanted to do.
There was a slight problem with being able to use custom markers, and we had to change a small amount of code to get that working.
// Link to issue
// TO DO
We can use JavaScript and DOM API's to control the entities within A-Frame as we would in normal web development, this is one of the many benefits of it being a HTML framework. This means we can use all of the popular libraries such as:
- d3
- Vue.js
- React
- Redux
- jQuery
- Angular
We had two developer days for innovation. This meant developers had two days off of client work to work on something new and innovative. In order to get the best out of the two days we thought it would be most effective to come up with a theme to constrain ourselves to a particular technology. Off of the back of Full Stack Fest we decided to focus on AR and VR - inparticular A-Frame. After brainstorming a number of ideas, we limited them and narrowed them down, and all chose what we wanted to work on and started to build out each of the projects.
One of the ideas that excited us the most was to visualise the real time energy usage of different appliances through AR / VR.
We thought it would be cool if you could point your phone's camera at an appliance and see the amount of energy it was currently using overlayed using augmented reality.
-simple version showed bare min for MVP quick prototype
We sketched out some initial ideas of how we could display useful information using charts
we needed a way to measure the energy and soon realised that the best way to do this was using smartplugs
We are using node.js and a number of packages to the data from the smartplug. There were certain packages that we used to get the backend up and running.
We used express to create a simple web application.
we discovered wemo-client, a JavaScript client library for controlling and subscribing to Wemo Devices.
We already had a couple of Wemo smart plugs in the office, so we used this library to build a simple Node.js server that discovered devices on a local network, retrieved power consumption data and listened for events. We then added Socket.io to our server, which allowed us to broadcast these events over a websocket - such as changes in power consumption or the status of smart plug (e.g. whether it was on or off, whether the device was taking more or less energy).
We got a couple more Wemo smart plugs and used them on different devices.
We have to use node to run the backend. In order to see if any devices are connected we set logs to output in the terminal giving us the Mac Address of the device connected, we can also see what port is being used to connect.
- Screen
- Microwaave
- Kettle
- Lamp
I will go through some of the code in order to help understand how we created the finished product. I will explain how you can do this using the WeMo smart plugs as we did, but if you don't have the smart plugs then you can just fake the data.
This part is all just set up
https://gist.github.com/c5f0f5d86f976676944166d45a74a308
We sketched out some initial ideas of how we could display useful information using charts and discovered wemo-client,
We had to get the information from the smart plugs to the - to a frontend built using A-Frame.
https://gist.github.com/21468515ec52505ddb198b06dd662ed6
Now that we have the data coming through we can start to do some cool stuff in the frontend which I will go through with you in my next blog post.