Augmented Reality: The New Knowledge Management

If you’ve been around a while as I have, you may remember the excitement about knowledge management (affectionately known as KM). One frequently-cited story from those days—mostly in the 1990s—came from BP, which had invested heavily in KM. The story goes that a drilling rig in the North Sea had stopped working back in 1995, and nobody on the drilling ship could repair it. But BP had invested in a “Virtual Teamwork” program that allowed the offshore drillers to quickly find an expert in an online system, and then enabled him to view the machinery from afar via a videoconference and recommend a successful fix. This was, as you might imagine, much faster and cheaper than the usual process of searching manually for an expert and then flying him (I think they were mostly men at the time) out to the rig.

Now, of course, anyone who Zooms for a living all day would be less than impressed by this achievement. But at the time it was considered quite exciting. Knowledge management may have faded somewhat as a management imperative, but we still need to get expertise to locations where it is scarce. And finding experts, on Zoom or not, can still be challenging. Fortunately there is a new approach to capturing and disseminating knowledge about physical products. It’s called augmented reality (AR).

Taqtile and the HoloLens

You may be most familiar with AR and virtual reality (VR) in the context of gaming. Many of the earliest adopters of AR and VR on Facebook’s Oculus or Microsoft’s HoloLens headsets were gamers. Pokemon Go probably drove more purchases of the headsets than any other program. But Microsoft has always emphasized the business value of the HoloLens, and one of the most obvious AR applications is visually guided equipment installation, repair, and maintenance.

Taqtile, a Seattle-based AR developer, was one of seven original members of Microsoft’s HoloLens Agency Readiness Program, which began about nine months prior to the initial launch of the HoloLens in 2016. Taqtile has focused on creating software for augmented reality ever since. Taqtile got connected with teams at Microsoft who were interested in applications for front-line industrial workers in big companies. Dirck Schou, the CEO and co-founder, commented:

There are 2.5 billion non-farm front-line workers who use their hands on the job. There is no computer appropriate for them. Their work consists of looking at a problem in the field or the factory floor, deciding what needs to be done, and then following a sequence of steps to fix the problem—a checklist. There is no Microsoft Office for this type of frontline worker, and we’d like to fill that gap.

Taqtile had a compelling vision for using the Hololens for digital transformation for industrial frontline workers. The goal was to democratize expertise and make “everyone an expert.” Ever since the vision was created, they’ve been implementing and scaling it, with an eye toward how it changes work and learning at the companies using it while preserving worker safety. They still do a lot of work with the HoloLens—now HoloLens 2—and it’s, as Schou explains, “a ubiquitous level of digital understanding and action embedded in safety glasses.” But they also work with other AR devices like Magic Leap, iPad, and Android mobile phones.

Taqtile’s content platform is called Manifest. It’s an enterprise platform for knowledge capture and reuse for industrial workers—a tool for structuring the “checklist” items for a particular task. It’s unlike anything we saw in the KM era. Manifest procedures contain instructions, photos, videos, pointers, and the like. If that’s not enough, it can also contact experts in real time—as with the BP Virtual Teamwork system.

Unlike the slow “knowledge engineering” process that characterized expert systems, Manifest makes it easy for subject matter experts to capture their knowledge about a particular industrial task. A field-based expert might initially capture the knowledge, and then an engineer might refine it, add more images, or update it. Field-based operators can then access knowledge from Manifest on any public or private network as well as completely offline. Its biggest customer category is the defense industry. For example, Manifest delivers maintenance and repair expertise from Navy veterans in Pearl Harbor and Washington state shipyards to sailors in Guam. Military users can follow step-by-step procedures overlaid upon defense apparatus to correct equipment failures, ensure configuration control, increase operational availability, and improve readiness.

Taqtile and Manifest at PBC Linear

But Taqtile also has many private sector customers, one of which is PBC Linear, a large machine bearing manufacturer based in South Beloit, IL. They have one of Northern Illinois’ largest machine shops, with about 120 machinists. But machinists and tool and die makers are becoming increasingly difficult to hire. In the “old days,” PBC could hire apprentices—typically from high school vocational programs—and they would train them for four years before they became journeymen toolmakers or machinists. For the last decade, however, high schools in the area haven’t offered vocational programs, and “it’s left a large talent gap for us,” according to Tim LeCrone, a manufacturing manager who has worked at PBC for 28 years. The company’s turnover rate was also high, and there was no good source of new talent. LeCrone said he didn’t have the time to train individual workers.

PBC executives realized they needed a new approach to training their people, so Beau Wileman, who manages the “Factory of the Future” program at the company, began to explore AR-based training at the suggestion of PBC’s owner. Wileman and some colleagues had a quick conversation with Taqtile and decided to give it a try. Wileman said that Taqtile reminded him of the video games he’d played in the past, and he thought younger workers would find that appealing. In April 2020 they brought in some Taqtile people to create some templates, and shortly thereafter they began to produce content in Manifest themselves.

Tim LeCrone, despite being a self-confessed “old school” manager, is enthusiastic about Taqtile. He’s said, “I can duplicate myself over and over.” The knowledge capture process allows Tim to go through the training process one time; for a typical task with 30 or 40 steps, it takes him about six hours. After the instructions are laid out in Manifest and another manager approves it, it’s capable enough to train a new employee. In 6 or 7 months PBC Linear has created about 70 new templates with 30-40 procedural steps each, and they are working well for training. He said, “There are lots of steps involved in what a machinist has to do to make a part on their machine. It’s intimidating for a new employee to remember tasks and the order of operations—Taqtile takes a lot of stress away from the user.”

Sam Aluko, a recent engineering graduate at PBC Linear, is one of the users of the system. As an engineer, he will eventually create templates himself, but thus far he’s been a user—“between an operator and an engineer,” as he put it. His focus thus far with Taqtile is using it to understand new machines that PBC has acquired. For example, the company recently bought a “thermoforming” machine. Thermoforming is a small-volume manufacturing process in which a plastic sheet is heated until it becomes pliable, then pressed to a mold to create a specific shape, and trimmed to create the final product. PBC is using the products as containers for their material handling of parts that are transferred throughout the machine shop. Aluko said he didn’t know anything about the machine or how to use it, but Taqtile let him go through the steps and form containers. If he doesn’t initially get a step, he can go back and look at it. He learns at his own pace, which, he says, is fast. Aluko is also using Taqtile to understand how to use a new collaborative robot that the company bought.

Benefits and Progress Thus Far

Tim LeCrone said that Taqtile and Manifest have helped them improve their quality. Seasoned machinists can show a new employee all the steps and shortcuts they need to machine a quality product—without four years of apprenticeship. PBC operators can take photos of parts and then compare them to the finished part in the Manifest instructions; visual cues are easily captured and applied in the system.

Just as important as the quality improvements is that Taqtile has largely helped them solve their talent problem. They have a lot of local competitors competing for their workforce, and because they have Manifest they have a competitive advantage for people. New operators and machinists find it sexy to work with AR, and they stay in the job for longer. PBC encourages its operators and engineers to look at lots of Manifest templates and learn about the entire shop’s operations.

Beau Wileman said that PBC is also beginning to use Taqtile and Manifest to help support their customers. Along with the products they supply they are now sometimes also supplying an AR headset, which the company can buy or rent—and Taqtile-based training. Not only do they learn how to use the equipment, but they start creating AR training modules themselves.

Dirck Schou, the CEO of Taqtile, said that the vendor is also beginning to have conversations with manufacturers who are using AR modules as a service tool. Some are already shipping a HoloLens and Manifest templates with a complicated piece of machinery.

Using AR to diagnose and maintain machinery is much easier than the BP solution in the early days of knowledge management. Augmented reality is a new way to package important content that can help industrial companies get up and running faster. Whether it will become the Microsoft Office of frontline workers is still unclear, but it is undoubtedly a useful product.