Intro 

Image Annotation and Data Labelling are a pre-requisite for ML algorithms to learn. ML Algorithms and Computer Vision Models must receive training from annotators to identify and interpret the relevance of objects in an appropriate manner.

Image Annotation and Data Labelling – Use cases, Types, and Benefits by Digit7 endeavors to provide an informative discourse about the state of image annotation and data labeling today. 

For example, an Autonomous Vehicle cannot (on its own) recognize the gestures made by a traffic sergeant, nor can it interpret the meaning of traffic signals.

Various annotators are leveraged to add metadata to objects so ML Algorithms and Computer Vision Models can be trained – based on this training, AVs can make split second decisions that can mean the difference between crashing and a smooth ride.

You might have encountered it on any kind of media file, be it a movie, music, or YouTube video.

Metadata, in layman’s terms, is data about data. 

• In our example, music and video files might contain meta information like movie name/track name, duration, bit rate, video quality, length, and a lot more supporting information.
• The purpose of metadata is to provide you with a quick snapshot of the file you are using, so you have the critical data accessible, in an easy-to-understand, streamlined, way.
• Without metadata, we would be in the dark, about the relevant information about what we are seeing or listening.
• Labelling of data has a very similar function in Image Annotation and Data Labelling.
• The Computer Vision Model or ML Algorithm would be unable to interpret and provide meaning to objects in images, defeating the very purpose of training.

Annotators, in essence, confer meaning upon objects in imagery. 

Use Cases 

Autonomous Vehicles 

AVs are computer-vision based deployments and additionally leverage ML algorithms to function in an appropriate way. They are very reliant on annotators for the insight into the training that they are receiving, without that, they do not understand the context and nuance associated with the detected objects in the imagery. 

Without a human agent annotating the metadata onto imagery, an autonomous vehicle can’t distinguish between the gestures made by a traffic cop, a changing traffic signal, and a lane exit symbol. 

Image Annotation for AVs need the following aspects down pat: 

1. A very high degree of accuracy – There is minimal room for human error. A fully manual annotation process is bound to be rife with errors.
   1. Automation steps in here; it is a basic feature of TagSquare.
   2. Automation circumvents human error to a great degree.
   3. Accurate metadata tagging and auto-tagging are central to smooth AVs.

2. Comprehensive knowledge of annotation – Which annotation is apt in a scenario?
   1. Judgment calls on the scenario, of which annotation to use needs knowledge of annotation.
   2. Image Classification, Object Detection, Semantic Segmentation, and Instance Segmentation are variants.
   3. Instance segmentation, while complex, might be apt; it classifies instances of objects.
   4. These might potentially be more dynamic.

3. Time/Monetary Resources – Image annotation, when done manually, is tedious, monotonous, and time-intensive.
   1. Organizations might prefer that their staff be assigned dynamic tasks.
   2. Annotation is outsourced; cost is a factor. Outsourcing is expensive.
   3. Partners with high degree of expertise charge good money.
   4. Even in-house, it would be expensive, as monotony for staff must be remunerated.

Automation is the obvious solution. Reiterating what we’ve said in the ‘Innovation Spotlight’ Section in prior blogs, TagSquare carries out 80% automated annotation with just 20% manual annotation input. 

Image Annotation for AR/VR 

Semantic Comprehension 

• Current-day AR does not profess an accurate semantic comprehension of the environment 
• What this implies is that there is a lack of seamless integration with the physical world. 
• Semantic understanding stitches the physical and digital world, weaving them for a fluid experience. 

What this means, in layman’s terms, is that present-day AR tech has a few blind spots about the physical world, thereby being a hindrance to 100% authentic augmentation. 

What does Semantic Understanding Facilitate? 

• Sematic comprehension of a user’s immediate surroundings enhances experiences.  
• For ex., not obstructing user’s faces when they are using AR devices, during AR experiences.  
• Meshing is problematic for present AR tech. Meshes for flat surfaces like the floor/coffee tables tend to be uneven. 

Semantic data could communicate with meshing algorithms to smooth out the meshes. For example, current day tech has problems like an object augmented onto the physical world appearing “glitchy.” 

Object Tracking :

Object tracking superimposes, or projects an augmented overlay, of digital objects onto environments in the physical world. Augmented overlays are incredibly useful in an array of applications, and professions. 

• Household Supplies / Consumer Electronics – Supplies like FMCG goods can be scanned with an AR device for overlays of nutrition, expiry, and even recipes. 

Scan that package of black rice with AR for some delicious recipes. 

• An air conditioner mechanic can scan the innards of an AC and diagnose the condition of components, and identify which part is malfunctioning. 

This would cut down on labor hours to a great extent. 

• Toys – Children can learn from AR by looking at a skeleton (With AR-enabled headsets.) They can learn about the bones in the body and what function each bone serves. 

They could use AR on edibles like chocolates, fast foods, veggies, and fruits, learn about nutrients in food and about what is good and bad for them. 

• Monuments / Statues – Another educational application for AR. A visitor to a museum or cultural centre could point their AR glasses at a statue/painting. 

They’d see overlays about the culture, its age, who the sculptor and patron were, and if it is of someone significant from a particular era. 

• Industrial Objects – This is one of the mission-critical applications for advanced AR-based object tracking. 

Electricians looking at broken industrial fitments could point their phone at the errant unit and get overlays about hotspots needing their attention, leading to swift resolution. 

Facial Recognition:

• You might remember this tech from your smartphone.
• Facial recognition is being combined with AR for breakthrough innovation.

Apple’s ARKit 

• Apple’s ARKit features advanced face tracking features.
• It is a versatile piece of tech with multiple use cases.

Samsung’s Bixby Vision 

• Bixby Vision truly ‘augments’ your physical surroundings.
• It can provide context to several objects with just your smartphone camera.

The Metaverse – AR or VR driven? – Potential Disruptive Market Use Case 

• With the buzz surrounding the metaverse, it is fascinating to think about the extent of the utility that will be extracted from AR and VR technologies.  
• VR has one major problem to tackle before it can experience mainstream acceptance – Comfort and Pricing. Top-of-the-line VR, like Valve Index, is still expensive. 
• On top of that, high-end gear is always out-of-stock, and not very easy to find. 

We’ve tried VR on Steam and while the games are stunning, some of us just wanted to barf within fifteen minutes of usage. 

• We’ve tried our best, be it Oculus or Valve Index, and motion sickness and discomfort with the headset was a very glaring issue.  
• We tried ADR1FT, a game about an astronaut stranded in space, and while it was stunning, we could not play for more than 15 minutes. 
• At this point, we cannot say with confidence that VR is cut out for long-term usage, regardless of the massive increase in graphical fidelity. 

The VR Apparatus is clunky and unwieldy and uncomfortable for long hours of use. Apps like VRChat are undeniable popular, but for a Metaverse like, longer duration experience? We’re not so sure. 

• The metaverse, being a world within a world, will not prioritize visual fidelity, but rather, comfort, accessibility, and reach.  

• Unless rapid evolution occurs in the VR space and headsets become intuitive to wear, inexpensive and comfortable, AR will have the edge. 
• The metaverse, without question, has been developed for extended hours of usage, and motion sickness is just unacceptable. 

The Metaverse has Matrix and Johnny Quest vibes, with its own economy, communities, and maybe one day, even work will be carried out within the Metaverse. Ergonomics matter. 

• Motion sickness can be felt during long travels when you cannot literally perceive the movement that is occurring with your body.  

• In VR deployments, you are standing, or seated, in a stationary position – your hands glued to a controller.  
• That’s one strike toward immersion right there. Visual fidelity aside, you are immediately receiving feedback that you are operating in a virtual environment. 

This is what is referred to as cognitive dissonance. Much like when you’re on a bus with shuttered windows, but are still experiencing objective movement, your brain is confused – due to your body’s stillness causing motion sickness. 

AR is a lot more intuitive, seamless, and handsfree way to experience the metaverse, no clunky headsets, no controllers, what you see is just pure, next-gen tech superimposed onto the real world, in a way akin to magic. 

Conclusion 

This brings us to the conclusion of the first part of this multi-part blog series. The next parts will round out the use cases, speak about the differing types of annotation (with a focus on Digit7’s DigitSquare feature set), and enumerate the advantages conferred by image annotation and data labeling onto businesses. (From the DigitSquare and overall market perspective.)  

You can expect the next part to be heavy on the stats, now that we have the technical primer out of the way. 

Stay tuned for the upcoming parts. We hope to finish this in a total of three parts, but it might flow over into a four-part article.