Computer Vision - Exploration

A picture is worth more than a thousand words. Computer Vision (CV) is the field of Artificial Intelligence (AI) that enables computers to analyze the real world. Using Deep Learning models, machines can accurately identify and classify objects from within digital images and take action based on what they see driving an increased level of adoption of vision-guided robotics. Learn how to achieve end-to-end CV and AR solutions for your organization by analyzing machinery sensor data and images in edge, AR and VR environments with TIBCO.

This Exploration showcase how to handle a Hololens 2 as a IoT Device inside Project Air™ and EdgeX Foundry, plus use the Holographic device for an Computer Vision and Deep Learning use case.

^{Computer Vision Scan on Hololens 2}

Note: Hololens shows holographic objects always ~2cm to high in Video Recordings,
they appear in Hololens itself on the right position!

Business use cases

there are many use cases where a human can get supported by computer vision e.g.

• Detect suspicious data on physical labels
• Find defects on physical objects
• Identify incorrect object placements
• and more …

When a suspicious scenario get detected, actions in this context can directly get executed by this human e.g.

• Open instruction guide
• Raise case / ticket
• Show additional statistics
• Contact helpdesk, or an expert

to mention only some!

Architecture

High-level Architecture Blueprint

Project ART - CV App

Handle taking Images, via voice command and send them via MQTT to EdgeX

Project Air - EdgeX

Project Air is fully based on EdgeX. The Project Air UI allows to configure Computer Vision Models and Edge Analytics on the fly using the IoT Pipeline, The Pipeline allows to deploy CV Models as well. Finally the validated Data get send back to Hololens, again via MQTT.

TIBCO Team Studio

Used for all preprocessing, and for ML Model / deep learning model training.

More Details about this Exploration

check out this YouTube Video

Conclusion

How do you deploy and inference a computer vision model at the edge?

• Tip: Use IoT TIBCO Project AIR™

How do you use Augmented Reality to visualize machine learning results?

• Tip: Use AR TIBCO Project ART™

How do you train a computer vision model and do analytics at the edge?

• Tip: Use TIBCO Team Studio™

Can you do it all at the same time?

• SURE!

Target Devices

Tested with Microsoft Hololens V2.