Can mixed reality give humans the edge over automation?

/0 Comments/in , /by

There’s a trend that’s been around since the dawn of the industrial revolution – if you can build a machine that automates the process of a worker (or workers), it will be more efficient in almost every way – cheaper, faster, and more accurate. At first, this trend wasn’t all that worrisome – while some jobs were made obsolete by automation, so many new fields and industries were created that workers had more job opportunities than ever before.

Like any new facet of technology, however, automation slowly (and then not so slowly) began advancing – taking a lot of jobs with it. But workers found a new avenue of growth through the rise of the service industry, and while manufacturing positions were pushed to the wayside, the economy found a way to capitalize on the trend of automation.

This automation, in the same manner as during the industrial revolution, opened up entirely new industries with millions of jobs to fill; the stand-out of the 2010’s being social media. The best part was, these jobs seemed future proof; the future was digital, and social media was a new form of digital communication by humans, for humans.

And then along came artificial intelligence.

If you had asked someone in the 90’s, 00’s, or even very early 10’s to imagine A.I., they’d probably have described a large supercomputer, housed in something akin to the datacenters integral to Silicon Valley and government digital operations around the world, calculating the great philosophical questions that have plagued humanity since the dawn of time; What is the meaning of life? What is the true nature of the universe? Where do we fit in amongst all of this?

Well, it turns out that while A.I. isn’t to the point where it can answer with “42” yet, it is really good at doing tasks that were largely unexpected just a few years ago – like diagnosing medical conditions, driving cars, and even writing prose. Deep learning neural networks are just plain better at processing vast quantities of data when compared to humans, and more than likely, if there’s a task that is processed through a digital medium, A.I. will, with time, be better at that task than humans.

Social media algorithms are already wreaking havoc with elections across the world by influencing millions of users – and with much more efficiency and effectiveness than any marketing team could ever hope to achieve themselves.

If humans were, for instance, Batman, A.I. would be Bane – just as the dark knight merely adopted the darkness, humans adopted the method of processing data via a digital interface – A.I. was born into, and is better adapted to this digital environment. We’re like the first creatures to move from the sea to land; just as our lumbering ancestors struggled against gravity, we interact with digital mediums with very low input/output when compared to A.I. – most of the time, our input is our two thumbs, and our output is limited to the processing power of one brain.

A.I., on the other hand, inputs and processes information orders of magnitude faster than we can – it was designed to work with 1’s and 0’s, while we work within a whole host of sensory and emotional data. Plain and simple, given enough time, A.I. will replace almost every job that works within a medium provided by a computer.

Take, for instance, the Space-X December 2018 Falcon 9 landing failure. The re-usable rocket’s first stage had a malfunction during re-entry – the hydraulics that control the grid fins which stabilize the rocket stalled. The flight computer, which assesses the situation during re-entry, detected the malfunction, and went into a “safe” landing mode – meaning it aimed for the ocean, and not the pad. The flight computer is smart; if an ocean landing is impossible, it is programmed to avoid buildings and people. But what was truly astounding was the flight computer using algorithms to learn – in real time – a method to stop the seemingly out-of-control spin by using the rocket engines themselves.

This hadn’t been pre-programmed; the rocket took in data, processed it, and then improvised. Fully automated, without any input from ground control. Let that sink in for a second – the first stage of a re-usable rocket malfunctioned upon re-entry, and improvised a safe landing using methods of control it was not programmed to, in order to slow (and ultimately regain control of) its descent into the ocean – by itself.

If this were 2014, that would be the plot of a sci-fi short – not a widely covered news story.

Luckily for us slow-input humans, rocket landings aren’t an everyday occurrence (yet). There’s still jobs to fill, work to be done. There are still plenty of things humans do better than A.I. and robots. We still have time to develop ways to keep up (and frankly, in certain fields, catch-up) with A.I., robotics, and the ever-looming presence of automation.

Augmented Reality (AR) and Mixed Reality (MxR), as Microsoft’s HoloLens 2 exemplifies, might be our method for keeping up with the advancement of automation in the near future.

Our edge against automation

As Boston Dynamic’s Spotmini has shown us, robots, just like velociraptors, can now open doors – or moonwalk, if you prefer. Atlas can backflip, turn 180 degrees in the air, and even run. But we’re still a lot better at this kind of stuff than robots. We’re much more adaptable – while a robot surgeon has higher dexterity than a human surgeon, it’s not going to build a car anytime soon. But with enough training, a human could become proficient at both surgery and auto repair.

But that training takes time, which takes money. If a manufacturer can spend $100,000 on a machine that takes the place of a human worker, they will – and a large part of that is due to the robot requiring zero on-the-job training.

Where automation is limited (for now, at least) is the number of functions a single robot can perform. Tesla’s Gigafactory, which was built with the idea of one day being fully automated, exemplifies this with its cautionary tales of too much automation too soon.

There’s no doubt humans are more adaptable than computers – we came to be the dominate species on this globe for a reason – so what would happen if we could learn almost as fast as A.I. algorithms?

First, when universal A.I. comes about, this is all null and void – there’s no way we could keep up with an intelligence like that – but for now, MxR might be what gives humans the edge over automation.

We’ve covered the enterprise applications of AR before – but with the release of the HoloLens 2, the power of MxR is becoming rapidly apparent. There have already been a lot of cool methods for bringing MxR into manufacturing, but due to the wearable, hands-free nature of the HoloLens, MxR is now a truly viable option.

First of all, you wear HoloLens on your head, and when it’s not in use, you can flip it up, akin to a welder’s mask. Microsoft put a lot of time and effort to make sure it was comfortable, allowing workers to wear it for extended periods of time.

The actual area on which images are projected (called the field of view) is larger than the HoloLens 1, which was released in 2015, made possible by mirrors called MEMS that project visual data at 120 FPS. There have been many other improvements since then as well – a big one being the reduced weight of the HoloLens 2 – and, unlike larger AR rigs, it can provide workers with in-the-field, real-time MxR experiences. Other workers can see what the HoloLens wearer can see via cameras, and highlight objects in real-time. Eye tracking helps to focus in on objects the wearer is looking at, and learns to predict what users will find interesting. Cnet’s Ian Sherr and Scott Stein described this predictive feature as feeling like “practical magic” and that is was almost like having your mind read. These eye-tracking cameras can even read the wearer’s emotions, and can detect who is wearing them – allowing shared headsets to switch from one user to the next without spending valuable time setting up personal options. The wearer can even manipulate virtual objects with their hands.

Ian Sherr and Scott Stein, who are not auto mechanics, were put in front of an ATV in need of repair, as a demo of the HoloLens at Microsoft’s headquarters. They were able to fix it in a short period of time with real-time instructions provided by HoloLens 2.

These features, and their implications, could possibly have a resounding impact on the structure of the manufacturing and production industries. With MxR, workers don’t need training – they only need to be able to follow real-time, visual instructions. They don’t need to have any knowledge prior to completing the task. They have, essentially, access to Matrix-like superpowers of downloading information directly to their brain – without the giant hole in the back of their skull.

And when things fall apart – as they inevitable do – the adaptable humans will still be there to improvise in ways automation physically couldn’t. This might be the crux of what sets MxR enhanced workers apart from automated robots – a worker with MxR enhancements is efficient and precise, but still has the ability to think, and act, outside of the box.

The AR and MxR industry was already worth $6 billion in 2018, and is expected to grow to $200 billion by 2025. That’s a huge leap – and with it, advancement and innovation. As A.I. takes over the digital realm, maybe we’ll regress back to a manufacturing based economy, albeit in a more futuristic way.

AR – It’s time to start thinking about it

/0 Comments/in , /by

There seem to be new technologies and trends emerging every day as the internet of things grows to envelop and shape the way we interact with each other, our jobs, and our hobbies. Today, one of these hot-topic-pieces-of-tech is augmented reality (AR).

With the release of Apple’s ARKit in June 2017, and Android’s ARCore in March 2018, mobile developers now have the tools to natively develop AR apps for smartphones, wearables, and even cars. If you’re a business that isn’t in the industry of user-facing software and products like Google or Snapchat, it might be easy to overlook the value of implementing AR into your daily procedures, or difficult to determine just how AR would work for you and your company.

Before we get into how your company can implement AR into its business model, let’s go over why AR (despite being a relatively new and untested technology) is probably the most exciting field of innovation for consumers and business alike.

In October of 2001, it was difficult to imagine that the iPod would completely revolutionize the technological landscape. We’d seen the evolution from records to cassettes and 8-tracks, to CDs and Walkmans – but all that had changed was the size and mobility of the music player – listeners still interacted with their devices using play, skip, stop, etc. It wasn’t until the iPod came to the market that a device changed the way listeners interacted with their music – suddenly, you didn’t need a backpack to carry your CDs, and track titles were visually displayed in searchable lists. The planning required by an individual to take music on the go was replaced with the ease of picking up a device with all of your favorite songs on it, and simply putting it in your pocket. This led to a drastic shift in the way people thought of their interactions with their music players – you listen to a Walkman, but you use an iPod.

We all know what came next – the largest shake-up the music industry had seen since the switch from analogue to digital recording – and like any significant change to a market, there were growing pains aplenty as artists and producers alike adapted to the changing climate.

It’s a trend that has continued rapidly ever since, as advancements in technology revolutionize every industry imaginable. In 2007, when the iPhone was first introduced, people stopped talking on their phones, and started using them. A year later, Android phones were on the market, and soon enough, smartphones evolved from a trend-setter status-symbol to ubiquitous multi-tools used to enhance almost every aspect of our daily lives. It took only six years to go from a device that holds 1000 songs to smartphones that put Captain Kirk’s communicator to shame.

Well, it’s 2001 all over again; except this time, it’s not music – it’s augmented reality.

Training and quality control

Imagine hiring a new HVAC technician. For that new hire to be successful, they must be trained, which means another technician is either pulled from their current job, or given the extra responsibility of teaching while also working. This is the way skilled trade knowledge has been passed down from expert to apprentice since the dawn of time, but it lacks the efficiency AR provides.

With an AR enhanced training program, that new technician can learn the ins and outs of HVAC systems on their own, without receiving instructions from an experienced employee – greatly increasing productivity and reducing the total cost of the hiring process. New employees are not only trained on their own – they are also trained more effectively, as AR ensures each training session disseminates the correct information and processes every time.

AR allows that new technician to see every layer and part of the HVAC system – from individual screws to the entire ventilation structure – and how each piece fits together. Your trainee can experience every possible type of mechanical failure for the system, and learn the appropriate remedy at their own pace.

Imagine being able to provide real, on-the-job experience without ever risking inefficiency or error at an actual job site; with AR, a technician can become an expert without ever going into the field.

When it comes to manufacturing, AR is the ultimate quality control assistant. Manufacturing jobs such as those on automobile production lines require high levels of accuracy, utilize a staggering number of parts, and rely on step by step processes that can easily be forgotten or implemented in the wrong order – especially during a long shift.

AR can, in real-time, highlight the next step in the assembly process, show where and how the parts are implemented, and check for accuracy after the step has been completed.

To see how this works, check out this amazing AR system by SCS Concept Group, VPG+.

Efficiency

Industries that utilize automated equipment, such as manufacturing or mining operations, can easily be enhanced with AR diagnostic systems. A fun project we did for Luckstone (one of the largest family-owned and operated producers of crushed stone, sand, and gravel in the U.S.) was an AR app that enhanced the productivity of quarry operations by providing the foreman with an interface that allowed on-site remote data analysis in real-time. The user points their mobile device at an operating machine, and a tag is displayed on the device’s screen – that tag can be expanded, providing real-time data on pounds of stone moved, operating efficiency, and any other data point relevant to the operations of the machine.

This saves foremen the wasted time of either walking across the quarry to the specific machine, or leaving the job site to analyze data in their office. This system was adapted to their business development team’s needs, and we developed an app that gave sales representatives the ability to access cloud data on past, current, and prospective clients based on geo location.

The sales representative points their phone towards a business, and in real time, information on past interactions with that customer is displayed on their device, giving them the ability to research potential clients, follow up on leads, and ensure they don’t mistakingly visit the same business twice, all in real time.

AR may be new, but it may just be the next big thing

It was only in 2017 when Apple released ARKit, after all; but AR is already making waves in the mobile industry. In 2018, Snapchat, which heavily invests in AR technology, posted a net worth of $3.2 billion. With leaps in advancement with wearable technology, and the integration of smart technologies in cars, it wouldn’t be surprising for AR to become commonplace in our daily lives in the coming years. It might even come to be expected by users – in 2006, if someone had told Jack Dorsey that Twitter would help decide the 2016 presidential election, he’d probably have laughed – but a social media startup became so integral to the way we communicate our ideas that it did just that.

We’re right at the cusp before the AR industry begins a rapid period of growth, so now is the time to start planning just how to implement AR into your own business’ needs.