Video analytics

Today , there are billions of  of video cameras in our homes,  phones , ATMs ,  baby monitors , laptops , smart watches , traffic monitoring , IOT devices , bots , you name it. The underlying purpose of most of them  is to capture media streams and optimize the content for further processing.

Stages of video Analytics-

1.Data Acquisition

Data gathered from multiple camera sourced need to be streamed in an aggregated manner to data processing unit in an analytics engine  or for archiving . It may be subjected to overall monitoring such as sunlight hours etc  or detailed low level object identification  such as facial recognition of passengers

2.Transformation to Data Sets

The assimilated data is grouped to operable entities. The identification and classification is done by adding attributes to recognizable shapes ,movements , edges and patterns  .

3.Calculate deviation or compliance

A trained model recognizes normal behavior and differentiation from the same is calculated .


Video content Analytics in surveillance 

Considering the use case for monitoring and surveillance cameras  , there is a growing need for realtime video analytics  , to ” detect and determine the temporal and spatial events “. Consider Surveillance cam recordings as forensic evidence or just monitoring incidents and reporting the specific crucial segments of video , both these usecases involve filtering a vast amount of recorded or steaming media to filter out the exact parts that the authorities are looking for. This involves custom identification and recognition of objects in frames .

There is growing research into extracting real time events of interest , minimizing the search time and maximizing accuracy from computer vision .

Consider following use-cases :

  1. Surveillance cam in solar farms or home based setups to predict sun light hours and forecast energy generation value  . Described in greater details here .
  2. Traffic monitoring cameras :
  • Automatic license / number plate recognition – surveillance cams for traffic need to record vehicle plate number to identify and tag the vehicles as they pass by  .
  • Car  dashboard cams for investigative purposes  post accidents and insurance claims
  • Motion tracking – Mapping the vehicle movement to detect any wrong turns , overtakes , parking etc
  • Scan for QR codes and passes at toll gates.
  • Identifying over-speeding vehicles

3. Security and Law enforcement

  • Trigger alarms or lockdowns  on suspicious activity or intrusion into safe facility
  • Virtual fencing and perimeter breach – Map facial identification from known suspects
  • Detection of left items and acceleration of emergency response

Communication based video analytics 

Unified enterprise communication , conferences , meeting , online webcasts , webinars , social messengers , online project demos are extensively using video analytics for building intuitive use cases and boost innovation around their platform . Few examples of vast number of usecases are

  1. Sentiment Analysis : Capturing emotions by mapping key words to ascertain whether the meeting went , happy , positive , productive or  sad , complaining , negative
  2. Augmented Reality for overlaying information such as interactive manual or a image . Areas for current usage include , e-learning and customer support .
  3. Dynamic masking for privacy

Autonomous Robot purpose Video analytics

Self driving drones , cars and even bots extensive use the feed from wide angle / fish eye lens cameras to create a 3D model of their movement in given space of 3 dimensional coordinates system.

Key technologies includes :

  1. Ego-motion estimation – mapping a 3D space with captured camera feed
  2. Deep Learning ( part of AI) from continuous feed from video cameras to find route around obstacles
  3. Remote monitoring for an unmanned vehicle
  4. Sterile monitoring for a unreachable or hazardous area example war-zone , outer  territorial objects as moon , mars , satellites

Bio-metrics based Video analytics 

Often video feed is now used for advanced search, redaction and facial recognition , which leads to features such as

  • unlocking laptop or phone
  • performing click with blink of eyes
  • creating concentration maps of webpage based on where eyes focused

Read more about role of webrtc in bio- metrics here

Video analytics in Industrial and Retail applications 

Application of video analytics in Industrial landscape are manyfold . On one hand it can be for intelligence and  information gathering such as worker foot count . Machine left unattended etc while on the other hand by using specific image optimization techniques can also audit automation testing of engines , machine parts , rotation counts etc .

  1. Flame and Smoke Detection – images from video streams are analysed for color chrominance, flickering ratio, shape, pattern and moving direction to ascertain a fire hazard.
  2. Collect demographics of the area with people counting
  3. Ensure quality control and procedural compliance
  4. Identify tail-gateing or loitering


List of few companies focusing on Video Analytics :

  1. Avigilon -
  2. 3VR –
  3. Intelli-vision –
  4. IPsotek –
  5. aimetis –


Edge Analytics

Performing data analytics at application level on the edge whole system architecture instead of at the core or data warehouse level. The advantage to the computation at fringes of network instead of a centralized system are faster response times and standalone off grid functionality support .

The humongous data collected over by IOT devices m machinery  , sensors  , servers , firewalls , routers , switches gateways and all other types of components are increasingly getting analysed and acted upon at the edge ,  independently with machine learning instead of data centers and network operation centers . With the help of feedback loops and deep learning one could add data drive intelligence to how operations are performed at critical machines arts such as autonomous bots or industrial setups.

Error Recovery and streaming

To control the incoming data stream , we divide and classify the content into packets , add custom classification header and stream them down to the server. In the event of data congestion of back pressure , some non significant packets are either dropped or added to a new dead queue .  The system  is thus made stable for high availability and BCP / fail-over recovery .