OTT ( Over the Top ) Communication applications

Market trends are not in favour of Telecom Service /providers with increasing use of OTT ( Over The Top ) applications like WhatsApp, Facebook messenger, Google hangouts, skype, Viber, etc. OTT applications are often blamed to take a stake in voice traffic revenue by using IP calls where the telco could’ve charged based on its rate plan of call seconds. This especially intensifies for long-distance or international calls where customers can use OTT providers instead of expensive telco rate plans.

What is an OTT ?

An Over The Top ( OTT ) application is one which provides communication services over Internet . Therefore these bypass the communication billing system setup by a Telecom Operator , resulting in no gain or loss of revenue to Telecom Operator who is providing the Internet service to user in first place .

Hence we see that OTT are major source of concern for Telecom Operators whose traditional and obviously expensive ( when compared to OTTs free service ) billing models are facing disruption .


Telecom Regulatory bodies around the world

The telecom regulatory authorities in some of the countries are for example listed as :

  • Afghanistan Telecom Regulatory Authority (ATRA) – Afganistan
  • Australian Communications and Media Authority (ACMA) – Australia
  • Bangladesh Telecommunication Regulatory Commission (BTRC) – Bnagaladesh
  • Canadian Radio-television and Telecommunications Commission (CRTC) – Canada
  • Ministry of Information Industry (MII) – China
  • Autorité de Régulation des Communications Électroniques et des Postes (ARCEP) – France
  • Bundesnetzagentur (BNA) – Germany
  • Telecom Regulatory Authority of India (TRAI) – India
  • Ministry for Communications and Informatization of the Russian Federation (Minsvyaz) – Russia
  • Infocomm Development Authority of Singapore (IDA) – Singapore
  • Independent Communications Authority of South Africa (ICASA) – south Africa
  • Federal Communications Commission (FCC) , National Association of Regulatory Utility Commissioners (regulators of individual states) (NARUC) , CTIA – The Wireless Association (CTIA) – USA

Such telecom regulatory bodies get to decide whether to enforce differential price to end consumers for using OTT so that telecom service providers can benefit or keep the Internet fair and open by passing Net Neutrality Laws and Bills and amendments .

What is Net Neaurality ?

The fundamental principle of Net Neurality is that Telecom Operators should not block , slow down or charge consumers extra for using other services as their means of communication. This states that it is wrong to charge users above the regular data rates for using VOIP apps and other internet based communication services.

The following counteries have adopted principles of Net Neutrality by passing bills or making law .

  • Chile – Chile’s General Law of Telecommunications, “No [ISP] can block, interfere with, discriminate, hinder, nor restrict the right of any Internet user of using, send, receive, or offer any content, application, or legitimate service through the Internet, as well as any activity or legitimate use conducted through the Internet.”
  • Brazil – ” Internet Bill of Rights ” makes equal access to internet mandatory in Brazil .
  • Netherlands – Even European Union has adopted Netherlands’ Net Neutrality amendment which reads “traffic should be treated equally, without discrimination, restriction or interference, independent of the sender, receiver, type, content, device, service or application.”
  • USA – Citizens make ‘We the People’ platform to ‘Restore Net Neutrality By Directing the Federal Communications Commission (FCC) to Classify Internet Providers as ‘Common Carriers‘. Therefore not allowing them to either throttle speed by paid prioritization , discriminate in pricing or block any broadband access to legal content .  Above facts are from this tech.firstpost.com article.

 

Inspite of the fact that I Support Net Neutrality with all my heart , as a telecom engineer I understand the cost investment made by Telecom operators in providing am efficient communication network to its subscribers ( Access , Network and Application layers ). Therefor I do have my sympathies with the Telcos and to level out the wide ranging conflict between Telcos and  ISP ( Internet Service Providers ) , I pen down the following points which reflect the Telecom Operators Problems and also highlight the solutions that can be adopted to counteract the OTT threat .

Depleting revenue for Telco

  1. Messaging – OTT messaging cost operators $13.9 billion, or 9% of message revenue in 2013
  2. Voice – Voice services under threat from VOIP services like Skype, Viber
  3. OTT apps – Voice & Message apps have been the operator’s biggest headache. Its time Operator should launch its own OTT Services
  4. Data Traffic – The utilization is yet to reach its peak. Will face challenges from  WiFi access
  5. Critical Pain areas – Erosion of Operator’s revenue from voice and (especially) messaging

Telco’s OTT Application

At this stage, a telecom Service provider / Operator must enter the apps market and bring forth a Messenger which is more powerful, interactive and awesome than an OTT application. Fortunately, the Operator can always couple this application with his background telecom infrastructure to provide the edge in performance and functionalities.

Road block while developing a OTT application for a Telecom Service Provider :

  • Investment in Data Network is not being utilized due to lack of service
  • Reuse of Existing business Logic and extending the service reach across devices and networks is tough
  • Operator already has full fledged network Infrastructure in Place
  • Desire for minimum CAPEX while investing in new technologies
  • compete with OTT players and open new revenue streams is a challenge

Next we find the way of solving the problems and integrating them together to form a Solution .

OTT Application for Telecom Service provider

  • Introduce new services to benefit from investment on Data Plans and Bandwidth
  • Expose REST API to enable 3trd party Integration with existing network Infrastructure
  • Partner with individual OTT players to make new services  that do not compete on core competencies like billing etc
  • Use protocols like SIP that reduce CAPEX and have goto market more quickly
  • Go for enriched service that lead to better user experience

This write-up outlines the process of creating an OTT application for a Telecom Service Provider. Components for the application include cloud Address Book, Video Chatting, Location share, Contact synchronization ,REST-based thin client , OS and device agnostic etc shown in the figure below:

telco's OTT app
telco’s OTT app

The Application  is designed to close knit with Operator’s own infrastructure hence the crucial entities like Network Address Book , Location Service are synced and fetched from Backend Network .

OTT application Feature Overview

Smart Address Book

  • Automatic: Get contacts from Gmail, Facebook
  • Fast search by first, last name, frequently
  •   dialed number
  • Roadmap: View calendar events
  • Personal: Get image from Gmail and display in   contacts list

Geo Location

  • Share own location during chatting
  • Get map for calculating the distance between two chat users
  • Roadmap : Trigger device (say Switch on/off AC before reaching home) from a threshold distance away from home   location

Messaging

  • Ad-hoc Chat
  • Session Based Chat
  • Voice Input for texting
  • Presence information of contacts
  • RoadMap: Legacy message integration

Telephony

  • Voice call to mobile
  • Voice call to PSTN
  • Video call to other @imAll user
  • Share images during voice call to other

Device agnostic

  • Compatible with IOS, windows
  • Can run as native app on ipad
  • Can run as browser client on windows
  • RoadMap: native app for android, windows phone,blackberry10

Roadmap

features of Unified Communications ( UC)
features of Unified Communications ( UC)
  • To upgrade the application and provide enganced and enrich service support the I propose the following roadmap.
  • From plain vanilla voice and video calling ( supported by every other OTT application ) our application should progress towards  legacy telecom support whihc included PSTN , GSM , ISDN etc . This requires backbone of telecom network and a good setup for media codec conversion to suit various legacy media codecs .

Road Map  from Traditional to New age services 

  1. Voice and video calling
  2. Legacy services support like MMS and SMS
  3. Integration with 3rd party Vendors
  4. Give new enriched services like Multilingual support , file transfer , screen-sharing etc
  5. give facility to integrated web plugins for web calling

To keep the interest of customers it is essential that the application be supported on other popular OTT services like skype  , Gtalk . for exmaple a caller should be able to make call from Skype  / Gtalk to our application .Multilingual capabilities, support for larger protocol spectrum will just act like icing on the cake .

How does it benefit the Operator??

  1.  Saves on development cost and time
  2.  Device Agnostic OTT Applications
  3. Simplified Service deployment
  4. Saves licensing cost per client
  5. Reuses existing Messaging and   Address Book service logic.
  6. Open New Revenue Streams for operator
  7. No separate SIP stack required for the client
  8.  Faster Time to Market

Update : At the time of writing this post I did not anticipate the wave of change that bring focus on subjects like “net neutrality” , ” Save the internet” and “free internet”. However now most of the telcos providers have either joined the bandwagon by prividing SIP trunk endpoinst for cloud teelphony providers ( eg twilio, Google Calls) or have made their own IP call application for B2B customers.


Service Creation Environment (SCE ) for SIP Applications

I hoped of making a SIP application Development environment a year back and worked towards it earnestly . Sadly I wasn’t able to complete the job yet I have decided to share a few things about it here .

Aim :

Develop  a SCE ( Service Creation Environment ) to addresses all aspects of lifecycle of a Service, right from creation/development, orchestration, execution/delivery, Assurance and Migration/Upgrade of services.

Similar market products :

  • Open/cloud Rhino
  • Mobicents and Telestax

Limitations of open source/other market products:

  • Free versions of the Service Creation Environments do not offer High Availability.
  • High Cost of Deployment grade versions.

Solution Description

I propose a in-house Java based Service Creation Environment “SLC SCE”. The SLC SCE will enable creation of JAINSLEE based SIP  services. It can be used to develop and deploy carrier-grade applications that use SS7 and IMS based protocols such as INAP, CAP, Diameter and SIP as well as IT / Web protocols such as HTTP and XML.

Benefits:

  • Service Agility
  • Significantly Lower price points
  • Open Standards eliminate Legacy SCP Lock-in

Timeline

  • Java-based service creation environment (SCE) – 1.5 Months
  • Graphical User Interface (GUI) and schematic representations to help in the design, maintenance and support of applications – 1.5 months
  • SIP Resource Adapter – 1 month

Architecture

Service Creation Environment (SCE) for SIP Applications
Service Creation Environment (SCE) for SIP Applications

In essence it encompasses the idea of developing the following

  1. SIP stack
  2. Javascript API’s
  3. Java Libraries for calling SIP stack
  4. Eclipse plugin to work with the SIP application development process
  5. Visual Interface to view the logic of application and possible errors / flaws
  6. SDKs (  Service Development Kit) , which are development Environment themselves

Extra Effort required to put in to make the venture successful

  1. Demo applications for basic SIP logic like Call screening , call rerouting .
  2. tutorial to create , deploy and run application from scratch . Aimed at all sections ie web developer , telecom engineer , full stack developer etc .
  3. Some opensource implementation on public repositories like Github , Google code , SourceForge
  4. Perform active problem solving on Stackoverflow , CodeRanch , Google groups and  other forums .

—————————————————————

Call Continuity from Mobile GSM/LTE network to VoIP/WebRTC

WebRTC is an evolving technology which promises simplified communication platform and stack for developers and hassslefree experience for users. It has the potential to provides in-context, call routed to the best personnel in service calls. Real time mapping of caller’s IP , locations and source metadata can be used for IVR eliminated. Such a complete collaboration tool is possible through WebRTC which is easy set-up, requires no installation no pugins and no download. Extremely secure, WebRTC can interoperate with existing VoIP, video conferencing and even PSTN. The only concern is the Integration with legacy PSTN and teleco environment.

In the present age of IP telephony when telecom convergence is the big thing all around the world, need of the hours is to enable fixed and mobile Service Providers ( SP )  to monetize the subscriber’s phone by extending it to new web based services. SPs can offer a WebRTC Communicator endpoint that uses the same phone number as the subscriber’s fixed or mobile phone. Advanced features enable calls to be transferred between fixed-line, mobile and WebRTC endpoints.

Position of WebRTC on Network protocol stack.

GSM is incompatible with WebRTC media stream due to legacy codecs, even if the WebRTC UA was to support these codecs the signalling translation will be a dffucult feat. Signalling is used for subscriber mobility, subscriber registration, call establishment, etc. Mobile Application Part (MAP), Base Station System Application part (BSSAP), Direct Transfer Application part (DTAP), ISDN User Part (ISUP) are some of the protocols making up GSM. In my opinion Some of the ways to integrate WebRTC to GSM backened could be

  • Develop GSM-To-IP Interworking Component and integrate it with GSM network components (like BTS ).
  • Integrate solution with H.323 based VoIP (Voice Over IP) components like Gatekeeper, Gateways/PBXs, to provide a complete voice/data network solution

Using telco service provider’s SIP trunk , if available, is the easiest way to conect to such backened communication systems.

  • A interface – connection between MSC and BSC;
  • Abis interface – connection between BSC and BTS;
  • D interface – connection between MSC and HLR;
  • Um interface – radio connection between MS and BTS.

GPRS/UMTS Mobile Network can be compatible to WebRTC via Data based communication on GPRS gateway.

LTE Network using Evolves Packet Core can communicate with WebRTC using realtime transcoding and SIP (Session initiation protocol) endpoints conneted to core IMS. AnICE server provides the reflexive IP addresses that the WebRTC implementation needs; the signaling gateway converts the WebRTC webapp’s communication into SIP/IMS signaling and the media relay converts the WebRTC media framing into the telco conformant representation.

Interworking between a WebRTC enabled browser and IMS based Telco Backened : A session is established so that the web app sends an initial INVITE, including an SDP offer for the “outgoing” stream, to the gateway. The signalling gateway will reserve the resources from the media relay in both directions. Consequently, the signalling gateway will send an SDP answer to the initial INVITE and create an SDP offer of its own. This SDP offer is carried in a SIP UPDATE. Once the media between the web app and media relay is set, the session will progress towards IMS and will be handled like any other session. At this point, the media relay has mapped two unidirectional “web app streams” into one bidirectional “IMS stream” and will forward all media between the two. The mapping is done for both audio and video streams, meaning that we are able to support both audio and video calls between WebRTC and Telco clients and conferences.

WebRTC bypasses many limitation of earlier p2p (Peer-to-peer media) streaming frameworks like NATS. It opens avenues for innovative cross-platform use cases such as Healthcare, service technicians on call, Retail and financial communications, phone payments and insurance claims. Other applications such as Unified communication and collaboration are applicable for sales, CRM, remote education etc.

Transfer mobile callto WebRTC session
Transfer mobile callto WebRTC session

SPs can offer 3rd Party WebRTC endpoints to access the user’s phone number and subscription . E.g. enable web applications such as Facebook, Amazon or Netflix to allow their users to make/receive calls or messages directly from the web applications

Revenue Streams :

  • monthly fee for access to WebRTC endpoints and for receiving calls from by 3rd Party WebRTC endpoints
  • One time upgrade fees for Accessing the Web service integration with telecom network like a plan upgrade

Brownie points

  • No software is required to be downloaded on the subscriber’s computer, tablet or mobile phone
  • No desktop support required for the service provider

Plans For Consumer Customers:

  • Subscribers can use the WebRTC endpoints on their computers, tablets or mobile phones as a fixed-line device at home, as a desktop solution when away from home and to avoid international tolls when traveling
  • Subscribers can connect their web services (e.g. Websites , Facebook, Amazon, Netflix) to their fixed or mobile services subscriptions using their SP-provided phone number

Plans For SP Enterprise Customers:

  • Enterprises can deploy a WebRTC endpoint for their employees that provides a single corporate communications endpoint that can be connected to any of the corporation’s UC/PBX and Call Recording systems
  • Employees can use the WebRTC endpoint as their office phone at work, home or when traveling
  • Connects to all leading UC/PBX and Recording platforms simultaneously
  • Enterprises can deploy a single WebRTC endpoint across all their UC/PBX and Recording platforms – current and future
  • Easy for IT departments to deploy – no software is required to be downloaded to employees’ computers, tablets or mobile phones
  • Enables corporate policies and features from the WebRTC endpoint including
  • Displaying the corporate identity
  • Routing calls via corporate networks
  • Tracking and Recording calls and messages

Harmonization of services between generations of telecommunication core layers


A communication system can be made up of many components which are individually undergoing evolution such as access layer generations, and core layer upgrades. Harmonized and uniform open standard-based service delivery platforms over legacy Proprietary codebase is the preferred choice for most service providers to save the investment in their infrastructure and programming while keeping up with the shift in technology. I shall be editing this post to discuss more on the process of Service Harmonization.This saves the Telecom Service Provider the trouble of rewriting call logic with every telecom generation evolution ie IN to SIP to Web based WebRTC phones.

Landscape shift for Telecommunication Service providers includes Transmission layer which is ATM/Frame relays moving towards IP/MPLS. Access Layer hardware specific to POTS / PSTN / ISDN upgrading towards NGN and VOIP.  Packet Switched Next gen Soft Switches based on SIP.

Telecommunication service Harmonization

The Service Harmonization Layer does the job of holding all new and legacy services while providing uniform interface to interact with access network regardless of the back-end Call program logic. It involves consolidation for the service layers across IMS and legacy mobile network and Orchestration to extend the capability of underlying platform to support multiple IN variants. Diagrammatic depiction of scope of Service Harmonization.

Gateways based Harmonization

Service Broker based Harmonization

As CSPs evolve their networks for LTE, the resulting networks present tremendous challenges in voice services and application delivery. Realizing this opportunity, the telecom software industry has come forward with a purpose-built network element: the Service Broker, a solution specifically designed to overcome network architecture challenges and ensure voice service delivery from any network domain to any other network domain. Service Brokers are placed between the application layer and the control layer.

A service broker is a service abstraction layer between the network and application layer in a telecom environment. SB( Service Broker ) enables us to make use of existing applications and services from Intelligent Network’s SCP ( Service Control Point ), IMS’s Application Server as well as other sources in a harmonized manner

Legacy switches vs Softswitches

Legacy switches are circuit-switched, monolithic, propertiary and expensive while Softswitch is packet-switched and open interfaced. They are scalable and vendor-independent which enables easy convergence. Softswitches forms the basis for a service harmonization engine as they increase the granularity and power processing distribution of the Network

Service Delivery Layer in Legacy vs Harmonized Services

Legacy Service Layer has a function-centric architecture having multiple domain-specific session types such as Mobile calls, IPTV and broadband. Harmonized service delivery layer has Open APIs and is essentially Data-centric. This leads to fast and agile development and deployment of convergent services specifically IMS system providing the framework for underline network agnosticism across fixed and mobile.


WebRTC communication over Web Services

This post is about communication from any application to WebRTC using Web Services.

HTTP and XML is the basis for Web services

Advertisement Engine with WebRTC

WSDL
  • WSDL stands for Web Services Description Language
  • It specifies the location of the service and the operations (or methods) the service exposes.
  •  XML-based language for describing Web services.

SOAP
  • SOAP stands for Simple Object Access Protocol
  • SOAP is an XML based protocol for accessing Web Services.
  • SOAP is based on XML

UDDI
  • UDDI stands for Universal Description, Discovery and Integration
  • UDDI is a directory service where companies can search for Web services.
  • UDDI is described in WSDL
  • UDDI communicates via SOAP

RDF
  • RDF stands for Resource Description Framework
  • RDF is a framework for describing resources on the web
  • RDF is written in XML
uses :Web services can offer application-components like: currency conversion, weather reports, or even language translation as services.
…………..

OfficeSIP

This post describes the installation , setup and configuration of Office SIP server to provide a registrar to our SIP based WebRTC application .

1. Get office SIP from this website

officesip0

2. Downloaded office sip executable on windows looks like this . double click on it to start installation .

officesip1

3.Installation screen appears as follows

officesip2

4. Terms and conditions

officesip3

5. Destination folder for installation

officesip4

6. Installing

officesip5

7. Validating install

officesip6

8. Installation completed

officesip7

9. Launch icon on dekstop

officesip8

10 . Office SIP web login screen

officesip9

11. create account ( also known as realm or domain ) for users to register in their SIP and webrtc sip agents

officesip10

12. Fill in account details for every account added to officeSIP

officesip11

13. add users that will be registering to officeSIP server

officesip13

14.Office SIP settings file can be modified for changing server parameters like ip and port

officesip14_001

15 . Office SIP can now be used by sip agents like softphones , webRTC pages , etc to register with OfficeSIP server


BEA Weblogic SIP server

Bea server is a old SIP servlet container ie application server which is used to embed control logic in a program.

1. Install Bea Weblogic

2. Follow the Installation steps

Make domain

3. Goto the installation directory . Usually C:/bea/user_projects/mydomain/ .

click on startweblogic.cmd in windows. In case the system is linux run startweblogic.sh script

4. Open Web console on url : http://127.0.0.1:7001/console. Enter username password

default username password weblogic , weblogic .

It can also be customized for example my username and password are altanai , tcs@1234

5.  Make Converged SIP Servlet Application in any editor such as notepad , edit+ etc .

The project structure looks like

Call screening
src
build
src
web
build.xml

The SIP servlet are put side directory structure of src. For example : sample application for Call screening :

package com.altanai.voice;

import java.io.IOException;
import javax.servlet.*;
import javax.servlet.sip.*;
import javax.servlet.sip.Proxy;
import org.apache.commons.logging.Log;
import org.apache.commons.logging.LogFactory;
import java.util.*;

public class CallScreening extends SipServlet{

private static SipFactory factory;
private static SipApplicationSession sas;
private static Proxy proxy;

public void init(ServletConfig config) throws ServletException{
	System.out.println("Call screening SIP servlet initiated");
	super.init(config);
}

protected void doInvite(SipServletRequest req) throws java.lang.IllegalArgumentException,java.lang.IllegalStateException,javax.servlet.ServletException,java.io.IOException{
	System.out.println("Received an Invite Request");
	if(req.getFrom().toString().indexOf("alice")!=-1){
		req.createResponse(406).send();
		System.out.println("User is blocked");
	} else {
		req.createResponse(200).send();
		System.out.println("User is not blocked");
	}
}
}

6. Build it with ant . For this go inside the application folder and run ant. Output will either be “failed to build “ or “build successfully” .

The ant  command generates the war file from SIP servlet Web application .

7. Incase of successful build . Add the application to Weblogic web console install section and activate it.

I will demonstrate this process in step by step manner . First  click on “ Lock and Edit “ Button on the left panel . Then goto Install button in the centre area and browser to the location of application war or sar we have build through ant ,

8. We can delete an application in exactly the same way . click on “ Lock and Edit “ Button on the left panel . Then goto the delete button after selecting the radio button alongside the application we want to delete.

8. For enhanced application building we can also refer to sample provided along with bea weblogic . file:///C:/bea/sipserver30/samples/sipserver/examples/src/index.html

It is supported on jdk1.5 hence the system’s environment variables must match. Otherwise in later stages deploying applications throw class version error.


WebRTC Media Streams and Quality metrics


Media Stream Tracks in WebRTC

The MediaStreamTrack interface typically represents a stream of data of audio or video and a MediaStream may contain zero or more MediaStreamTrack objects.

The objects RTCRtpSender and RTCRtpReceiver can be used by the application to get more fine grained control over the transmission and reception of MediaStreamTracks.

Media Flow in VoIP system
Media Flow in WebRTC Call

Video Streams

Video Capture insync with hardware’s capabilities

WebRTC compatible browsers are required to support Whie-balance , light level , autofocus from video source

Video Capture Resolution

Minimum WebRTC video attributes unless specified in SDP ( Session Description protocl ) is minimum 20 FPS and resolution 320 x 240 pixels. 

Also supports mid stream resilution changes such as in screen source fromdesktop sharinig .

SDP attributes for resolution, frame rate, and bitrate

SDP allows for codec-independent indication of preferred video resolutions using a=imageattr to indicate the maximum resolution that is acceptable. 

Sender must send limiting the encoded resolution to the indicated maximum size, as the receiver may not be capable of handling higher resolutions.

Dynamic FPS control based on actual hardware encoding

video source capture to adjust frame rate accroding to low bandwidth , poor light conditions and harware supported rate rather than force a higher FPS .

Stream Orientation

support generating the R0 and R1 bits of the Coordination of Video Orientation (CVO) mechanism and sharing with peer.

Audio Streams

Audio Level

Audio level for speech transmission to avoid users having to manually adjust the playback and to facilitate mixing in conferencing applications.

Normalization is considering frequencies above 300 Hz, regardless of the sampling rate used. Can be adapted to avoid clipping, either by lowering the gain to a level below -19 dBm0 or through the use of a compressor.

GAIN calculation

  • If the endpoint has control over the entire audio-capture path like a regular phone
    the gain should be adjusted in such a way that an average speaker would have a level of 2600 (-19 dBm0) for active speech.
  • If the endpoint does not have control over the entire audio capture like software endpoint
    then the endpoint SHOULD use automatic gain control (AGC) to dynamically adjust the level to 2600 (-19 dBm0) +/- 6 dB.
  • For music- or desktop-sharing applications, the level SHOULD NOT be automatically adjusted, and the endpoint SHOULD allow the user to set the gain manually.

Acoustic Echo Cancellation (AEC)

Endpoints allow echo control mechanisms

SDP signaling and negotiation for media plane

Media plane adaptation is done at the SBC for network carried media, it should be done for all network hosted media services which face peer-to-peer media.

The high-level architecture elements of WebRTC media streams consists of

  • Encryption, RTP Multiplexing, Support for ICE
  • Audio – Interworking of differing WebRTC and codec sets
  • Video – Use of VP8, Support for H.264
  • Data – Support of MSRP ( RCS standard for messaging over DataChannel API)

Media Source

RTCVideoSource_4 (media-source)

timestamp	03/01/2022, 23:07:05
trackIdentifier	1bcab53d-1eca-41d1-a96a-00f1458c9b1b
kind	video
width	640
height	480
frames	7556
framesPerSecond	30

RTCAudioSource_3 (media-source)

timestamp	        03/01/2022, 23:06:26
trackIdentifier	        12cb979c-b40f-4de7-8b50-be6f4425e0b2
kind	                audio
audioLevel	        0.020599993896298106
totalAudioEnergy	1.8476431267450812
[Audio_Level_in_RMS]	0.02541394245734895
totalSamplesDuration	213.66999999995065
echoReturnLoss	        -0.11197675950825214
echoReturnLossEnhancement 8.111690521240234

Peer-to-Peer Media Stream

Direct connection to media servers and media gateways.

Use common codec set wherever possible to eliminate transcoding —Use regionalized transcoding where common codec not available Real-time video transcoding is expensive and performance impacting.

On-going standards/device/network work needs to be done to expand common codec set. WebRTC codec standards have not been finalized yet. WebRTC target is to support royalty free codecs within its standards.

MediaWebRTClegacy
AudioG.711, OpusG.711, AMR, AMR-WB (G.722.2)
Audio – ExtendedG.729a[b], G.726
VideoVP8H.264/AVC

Supporting common codecs between VoLTE devices and WebRTC endpoints requires one or more of the following:

  1. Support of WebRTC codecs on 3GPP/GSMA
  2. Support of 3GPP/GSMA codecs on WebRTC
  3. WebRTC browser support of codecs native to the device

RTP streams and RTCP stats Outbound Video

Chrome Browser on ubuntu OS

RTCOutboundRTPVideoStream_3305924664 (outbound-rtp)

timestamp	03/01/2022, 22:23:32
ssrc	3305924664
kind	video
trackId	RTCMediaStreamTrack_sender_4
transportId	RTCTransport_0_1
codecId	RTCCodec_1_Outbound_96
[codec]	VP8 (96)
mediaType	video
mediaSourceId	RTCVideoSource_4
packetsSent	171360
[packetsSent/s]	204.02266754223697
retransmittedPacketsSent	620
[retransmittedPacketsSent/s]	0
bytesSent	177210957
[bytesSent_in_bits/s]	1680050.6587655507
headerBytesSent	4218672
[headerBytesSent_in_bits/s]	39812.423281967494
retransmittedBytesSent	668008
[retransmittedBytesSent_in_bits/s]	0
framesEncoded	22003
[framesEncoded/s]	30.00333346209367
keyFramesEncoded	14
totalEncodeTime	418.017
[totalEncodeTime/framesEncoded_in_ms]	9.533333333333378
totalEncodedBytesTarget	0
[totalEncodedBytesTarget_in_bits/s]	0
framesSent	22003
[framesSent/s]	30.00333346209367
hugeFramesSent	1
totalPacketSendDelay	29963.73
[totalPacketSendDelay/packetsSent_in_ms]	31.62745098039772
qualityLimitationReason	none
qualityLimitationDurations	{bandwidth:0,cpu:174895,none:717684,other:0}
qualityLimitationResolutionChanges	0
encoderImplementation	libvpx
firCount	0
pliCount	2
nackCount	161
remoteId	RTCRemoteInboundRtpVideoStream_3305924664
frameWidth	640
frameHeight	480
framesPerSecond	30
qpSum	151000
[qpSum/framesEncoded]	9.3

RTCP statistics RTCRemoteInboundRtpVideoStream_3305924664 (remote-inbound-rtp)

timestamp	03/01/2022, 22:25:29
ssrc	984864038
kind	audio
transportId	RTCTransport_0_1
codecId	RTCCodec_0_Outbound_111
jitter	0.026854166666666665
packetsLost	19
localId	RTCOutboundRTPAudioStream_984864038
roundTripTime	0.048
fractionLost	0
totalRoundTripTime	8.932
roundTripTimeMeasurements	201

Frames

Initially

After considerable time( 10 minutes in my case ) the quality of the media stream adjust to network conditions and variations ( peaks and dips) flat out.

after some time
after some time has passed
after some time

Packets

Initially
After some time

Bytes Send And Received

Initially
After some time has passes

Headers

Initially
after some time has passes

Outbound Audio from Ubuntu Chrome Browser

RTCOutboundRTPAudioStream_984864038 (outbound-rtp)

timestamp 03 / 01 / 2022, 22: 13: 26
ssrc 984864038
kind audio
trackId RTCMediaStreamTrack_sender_3
transportId RTCTransport_0_1
codecId RTCCodec_0_Outbound_111
    [codec] opus(111, minptime = 10; useinbandfec = 1)
mediaType audio
mediaSourceId RTCAudioSource_3
packetsSent 14292
    [packetsSent / s] 50.003051944088384
retransmittedPacketsSent 0
    [retransmittedPacketsSent / s] 0
bytesSent 1151754
    [bytesSent_in_bits / s] 32449.980589635597
headerBytesSent 400176
    [headerBytesSent_in_bits / s] 11200.683635475798
retransmittedBytesSent 0
    [retransmittedBytesSent_in_bits / s] 0
nackCount 0
remoteId RTCRemoteInboundRtpAudioStream_984864038

RTCP statistics RTCRemoteInboundRtpAudioStream_984864038 (remote-inbound-rtp)

timestamp	03/01/2022, 22:17:05
ssrc	984864038
kind	audio
transportId	RTCTransport_0_1
codecId	RTCCodec_0_Outbound_111
jitter	0.002
packetsLost	3
localId	RTCOutboundRTPAudioStream_984864038
roundTripTime	0.023
fractionLost	0
totalRoundTripTime	4.344
roundTripTimeMeasurements 98	

Inbound Video from Android Webrtc Browser

RTCInboundRTPVideoStream_3384287918 (inbound-rtp)

timestamp 03 / 01 / 2022, 22: 55: 35
ssrc 3384287918
kind video
trackId RTCMediaStreamTrack_receiver_4
transportId RTCTransport_0_1
mediaType video
jitter 0.027
packetsLost 78
packetsReceived 79545
    [packetsReceived / s] 0
bytesReceived 77156700
    [bytesReceived_in_bits / s] 0
headerBytesReceived 1978716
    [headerBytesReceived_in_bits / s] 0
jitterBufferDelay 2284.024
    [jitterBufferDelay / jitterBufferEmittedCount_in_ms] 0
jitterBufferEmittedCount 13100
framesReceived 13101
    [framesReceived / s] 0[framesReceived - framesDecoded] 0
framesDecoded 13101
    [framesDecoded / s] 0
keyFramesDecoded 1
    [keyFramesDecoded / s] 0
framesDropped 0
totalDecodeTime 94.229
    [totalDecodeTime / framesDecoded_in_ms] 0
totalInterFrameDelay 442.0259999999831
    [totalInterFrameDelay / framesDecoded_in_ms] 0
totalSquaredInterFrameDelay 20.370232000000772
    [interFrameDelayStDev_in_ms] 0
decoderImplementation libvpx
firCount 0
pliCount 2
nackCount 51
codecId RTCCodec_1_Inbound_96
    [codec] VP8(96)
lastPacketReceivedTimestamp 1641276962171
    [lastPacketReceivedTimestamp] 03 / 01 / 2022, 22: 16: 02
frameWidth 480
frameHeight 640
framesPerSecond 4
qpSum 97949
    [qpSum / framesDecoded] 0
estimatedPlayoutTimestamp 3850268134980
    [estimatedPlayoutTimestamp] 03 / 01 / 2092, 22: 55: 42

missing

Inbound Audio from Android Webrtc Browser

RTCInboundRTPAudioStream_579305270 (inbound-rtp)

timestamp 03 / 01 / 2022, 22: 50: 14
ssrc 579305270
kind audio
trackId RTCMediaStreamTrack_receiver_3
transportId RTCTransport_0_1
mediaType audio
jitter 0.003
packetsLost 208
packetsDiscarded 0
packetsReceived 124469
    [packetsReceived / s] 50.03320990953163
fecPacketsReceived 0
fecPacketsDiscarded 0
bytesReceived 4433321
    [bytesReceived_in_bits / s] 14209.431614306981
headerBytesReceived 3485132
    [headerBytesReceived_in_bits / s] 11207.439019735084
jitterBufferDelay 17887008
    [jitterBufferDelay / jitterBufferEmittedCount_in_ms] 113.79999999996896
jitterBufferEmittedCount 119485440
totalSamplesReceived 118645920
    [totalSamplesReceived / s] 48031.88151315036
concealedSamples 689415
    [concealedSamples / s] 0[concealedSamples / totalSamplesReceived] 0
silentConcealedSamples 338882
    [silentConcealedSamples / s] 0
concealmentEvents 230
insertedSamplesForDeceleration 33841
    [insertedSamplesForDeceleration / s] 0
removedSamplesForAcceleration 1562246
    [removedSamplesForAcceleration / s] 0
totalAudioEnergy 4.458078675648182
    [Audio_Level_in_RMS] 0
totalSamplesDuration 2472.2900000075438
codecId RTCCodec_0_Inbound_111
    [codec] opus(111, minptime = 10; useinbandfec = 1)
lastPacketReceivedTimestamp 1641279014658
    [lastPacketReceivedTimestamp] 03 / 01 / 2022, 22: 50: 14
audioLevel 0
remoteId RTCRemoteOutboundRTPAudioStream_579305270
estimatedPlayoutTimestamp 3850267813642
    [estimatedPlayoutTimestamp]

RTCP statistics RTCRemoteOutboundRTPAudioStream_579305270 (remote-outbound-rtp)

timestamp 03 / 01 / 2022, 22: 48: 47
ssrc 579305270
kind audio
transportId RTCTransport_0_1
codecId RTCCodec_0_Inbound_111
packetsSent 120306
bytesSent 4285534
localId RTCInboundRTPAudioStream_579305270
remoteTimestamp 1641278927459
    [remoteTimestamp] 03 / 01 / 2022, 22: 48: 47
reportsSent 480
roundTripTimeMeasurements 0
totalRoundTripTime 0

Comparision of Media stream QoS metrics between laptop browser and mobile browser

chrome browser on laptopmobile chorme browser
higher frame received ( 30)lower frame received (20)
lower jitter (0.002) and packet loss (3)higher jitter (0.003) and packet loss (208)

Bundled Streams

Same port used for all emdia stream. Fir exmaple port 9 is used for audio video as well as their RTCP feedbacks in snippet below.

a=group:BUNDLE 0 1
a=extmap-allow-mixed
a=msid-semantic: WMS kAGMqdVh7lL70CVUVZQblgjPYsuhOAiGY3ii
m=audio 9 UDP/TLS/RTP/SAVPF 111 63 103 104 9 0 8 106 105 13 110 112 113 126 (33 more lines)
a=rtcp:9 IN IP4 0.0.0.0
a=sendrecv
a=msid:kAGMqdVh7lL70CVUVZQblgjPYsuhOAiGY3ii ed96e925-4425-467b-a099-8fb2e0c67b88
a=rtcp-mux
...
m=video 9 UDP/TLS/RTP/SAVPF 96 97 98 99 100 101 102 122 127 121 125 107 108 109 35 36 120 119 124 (100 more lines)
a=rtcp:9 IN IP4 0.0.0.0
a=sendrecv
a=msid:kAGMqdVh7lL70CVUVZQblgjPYsuhOAiGY3ii f45d3002-2866-44ce-a807-ac59a4f6708c
a=rtcp-mux

Same CSRC : To run multiple streams of media over a single RTP stream, a common SSRC is used.

a=ssrc:1683985800 cname:OYWXQ35YL2Hh+eUX
a=ssrc:1384669066 cname:OYWXQ35YL2Hh+eUX

Peer to Peer Data Transfer

Data Channel API of Webrtc allows bidirectional communication of arbitrary data between peers. It uses the same API as WebSockets and has very low latency.

  • (+) DataChannel is p2p and is also ened to end encrypted leader to higher privacy
  • (+) build in security due to p2p transfer
  • (+) high throughput than text transfer via a messaging server
  • (+) lower latency as p2p transfer takes shortest route

SCTP is the protocol that opens connectiosn for peer to peer data channel support in WebRTC. It can be configured for reliability and ordered delivery. It provides flow and congestion control to the data messages.

Data Channel Metrics

timestamp  03/01/2022, 23:13:13
label	   sctp
protocol	
dataChannelIdentifier	1
state	    open
messagesSent	42
[messagesSent/s]	0
bytesSent   1962750
[bytesSent_in_bits/s]	0
messagesReceived	31
[messagesReceived/s]	0
bytesReceived	4712
[bytesReceived_in_bits/s]	0
After sharing 2 files of 1.5 Mb each

Bitrate

Webrtc Changes bitrate , resolution and framerate dynamically to accomodate the network conditions, policy constraints or user equipment capability. Higher the bitrate, higher the media quality.

Birate of Audio Codecs

Lossey formats
– iLBC (narrow band )13.33, 15.20 kbit/s
– iSAC ( wideband) 10–52 kbit/s
– GSM-EFR 12.2 kbit/s
– AAC 8–529 kbit/s (stereo)
– AMR-WB (G.722.2) 6.60, 8.85, 12.65, 14.25, 15.85, 18.25, 19.85, 23.05, 23.85 kbit/s
– Opus – 6–510 kbit/s(-) higher bitrate consumes more bandwidth
(-) can cause congestion on network route

ITU-T formats
– G711 64kbps
– G.711.1 ( MDCT, A-law, μ-law) 64, 80, 96 kbit/s
– G.722 64 kbit/s (comprises 48, 56 or 64 kbit/s audio
and 16, 8 or 0 kbit/s auxiliary data)

Lossless formats (such as Dolby trueHD, MPEG-4 ALS)
consume much larger bitrates.

Bitrate of Video

QVGA 200-500 kbps
VGA 400 – 800 kbps
720p+ > 800 kbps
4K( 60fps) > 20 mbps

Packet Loss

Packet loss can cause choppy audio and distorted, blurry or frozen video.

Audio Packet loss

Video Packet loss

Jitter

Jitter is the packet delay variation in an otherwise predictable normal rate of delay. This could indicate route changes, growing congestion etc.

Audio

Jitter fo Audio

Video

Jitter for Video

Round Trip Time

High RTT is indicative of network congestion and causes delays.

Audio RTT

Video RTT

Cummulative Analysis of packet lost , RTT measurement and total RTT on an internetwork scenarios with peerfelxive and relay ICE candidates

Deeper Analysis of fraction lost , jitter and RTT

The chart shows how jitter follows RTT

References :

Read more on SDP and its attributes