Category Archives: WebRTC SaaS

Session Border controller for WebRTC

Session Border Controllers ( SBC )  assist in controlling the signaling and usually also the media streams involved in calls and sessions.

They are often part of a VOIP network on the border where there are 2 peer networks of service providers such as backbone network and access network of corporate communication system which is behind firewall.

A more complex example is that of a large corporation where different departments have security needs for each location and perhaps for each kind of data. In this case, filtering routers or other network elements are used to control the flow of data streams. It is the job of a session border controller to assist policy administrators in managing the flow of session data across these borders. – wikipedia

SBC act like a SIP-aware firewall with proxy/B2BUA.

What is B2BUA?

A Back to back user agent ( B2BUA ) is a proxy-like server that splits a SIP transaction in two pieces:

  • on the side facing User Agent Client (UAC), it acts as server;
  • on the side facing User Agent Server (UAS) it acts as a client.

B2BUAs keep state information about active dialog. Read more here .

Remote Access

SBC mostly have public url address  for teleworkers and a internal IP for enterprise/ inner LAN . This enables users connected to enterprise LAN ( who do not have public address ) to make a call to user outside of their network. During this process SBC takes care of following while relaying packets .

  1. Security
  2. Connectivity
  3. Qos
  4. Regulatory
  5. Media Services
  6. Statistics and billing information

Topology hiding

SBC hides and anonymize secure information like IP ports before forwarding message to outside world . This helps protect the internal node of Operators such as PSTN gateways or SIP proxies from revealing outside.

Explaining the functions of SBC in detail

1. Security

SBCs are often used by corporations along with firewalls and intrusion prevention systems (IPS) to enable VoIP calls to and from a protected enterprise network. VoIP service providers use SBCs to allow the use of VoIP protocols from private networks with Internet connections using NAT, and also to implement strong security measures that are necessary to maintain a high quality of service. The security features includes :

  • Prevent malicious attacks on network such as DOS, DDos.
  • Intrusion detection
  • cryptographic authentication
  • Identity/URL based access control
  • Blacklisting bad endpoints
  • Malformed packet protection
  • Encryption of signaling (via TLS and IPSec) and media (SRTP)
  • Stateful signalling and Validation
  • Toll Fraud – detect who is intending to use the telecom services without paying up

2. Connectivity

As SBC offers IP-to-IP network boundary, it recives SIP request from users like REGISTER , INVITE  and routes them towards destination, making their IP. During this process it performs various operations like

  • NAT traversal
  • IPv4 to IPv6 inter-working
  • VPN connectivity
  • SIP normalization via SIP message and header manipulation
  • Multi vendor protocol normalization

Further Routing features includes  :
Least Cost Routing based on MoS ( Mean Opinion Score ) : Choosing a path based on MoS is better than chooisng any random path . 

Protocol translations between SIP, SIP-I, H.323.

In essence SBC achieve interoperability, overcoming some of the problems that firewalls and network address translators (NATs) present for VoIP calls.

Automatic Rerouting

connectivity loss from UA for whole branch is detected by timeouts . But they can also be detected by audio trough SIP OPTIONS by SBC .  In such connectivity loss , SBC decides rerouting or sending back 504 to caller .

SBC 2 (1)

4. QoS
To introduce performance optimization and business rules in call management QoS is very important . This includes the following :

  • Traffic policing
  • Resource allocation
  • Rate limiting
  • Call Admission Control (CAC)
  • ToS/DSCP bit setting
  • Recording and Audit of messages , voice calls , files
  • System and event logging

5. Regulatory

Govt policies ( such as ambulance , police ) and/ or enterprise policies may require some calls to be holding priority over others . This can also be configured under SBC as emergency calls and prioritization.
Some instances may require communication provider to comply with lawful bodies and provide session information or content , this is also called as Lawful interception (LI) . This enables security officials to collect specific information rather than examining all the traffic that passes through a particular router. This is also part of SBC.
6. Media services

Many of the new generation of SBCs also provide built-in digital signal processors (DSPs) to enable them to offer border-based media control and services such as- DTMF relay , Media transcoding , Tones and announcements etc.

WebRTC enabled SBC’s also provide conversion between DTLS-SRTP, to and from RTCP/RTP. Also transcoding for Opus into G7xx codecs
and ability to relay VP8/VP9 and H.264 codecs.

7. Statistics and billing information

SBC have an interface with and OSS/BSS systems for billing process , as almost all traffic that pass through the edge of the network passes via SBC. For this reason it is also used to gather Statistics and usage-based information like bandwidth, memory and CPU.  PCAP traces of both signaling and media information of specific sessions .

New feature rich SBCs also have built-in digital signal processors (DSPs). Thus able to provide more control over session’s media/voice . They also add services like Relay and Interworking, Media Transcoding, Tones and Announcements, DTMF etc.

Session Border Controller (SBC)

Session Border Controller for WebRTC , SIP , PSTN , IP PBX and Skype for business .

Diagram Component Description

Gateways provide compression or decompression, control signaling, call routing, and packetizing.

PSTN Gateway : Converts analog to VOIP and vice versa . Only audio no support for rich multimedia .

VOIP Gateway : A VoIP Gateway acts like a translator converting digital telecom lines to VoIP . VOIP gateway often also include voice and fax. They also have interfaces to Soft switches and network management systems.

WebRTC Gateway : They help in providing NAT with ICE-lite and STUN connectivity for peers behind policies and Firewall .

SIP trunking : Enterprises save on significant operation cost by switching to IP /SIP trunking in place of TDM (Time Division Multiplexing). Read more on SIP trunk and VPN  here. 

SIP Server : A Telecom application server ( SIP Server ) is useful for building VAS ( Value Added Services ) and other fine grained policies on real time services . Read more on SIP Servers here . 

VOIP/SIP service Provider :   There are many Worldwide SIP Service providers such as Verizon in USA , BT in europe, Swisscom in Switzerland etc .

 

Building a SBC

The latest trends in Telecommunications industry demand an open standardized SBC to cater to growing and large array of SIP Trunking, Unified Multimedia Communications UC&C, VoLTE, VoWi-Fi, RCS and OTT services worldwide . Building an SBC requires that it meet the following prime requirements :

  • software centric
  • Cloud Deploybale
  • Rich multimedia (audio , video , files etc) processing
  • open interfaces
  • The end product should be flexible to be deployed as COTS ( Commercial Off the shelf) product or as a virtual network function in the NFV cloud.
  • Multi Configuration , should be supported such as Hosted or Cloud deployed .
  • Overcome inconsistencies in SIP from different Vendors
  • Security and Lawful Interception
  • Carrier Grade Scaling

Flow Diagram 

SBC WebRTC to SIP

Thus we see how SBC became important part of comm systems developed over SIP and MGCP. SBC offer B2BUA ( Back to Back user agent) behavior to control both signalling and media traffic.


 

Advertisements

Setting up ubuntu ec2 t2 micro for webrtc and socketio

Setting up a ec2 instance on AWS for web real time communication platform over nodejs and socket.io using WebRTC .

Primarily a Web Call  , Chat and conference platform uses WebRTC for the media stream and socketio for the signalling . Additionally used technologies are nosql for session information storage , REST Apis foe getting sessions details to third parties.

Below is a comprehensive setup if ec2 t2.micro free tier instance  ,  installation with a webrtc project module and samples of customization and usuage .

Technologies used are listed below :

Server

  1. ec2 instance t2.micro covered under free tier
  2. domain name
  3. SSL certificate

Core module for Web Calling feature

  1. WebRTC
  2. Node.js
  3. socket.io

UI components

  1. javascript
  2. css
  3. html5
  4. bootstrap
  5. jquerry

Supporting setup for session management

  1. Code version-ing  and maintenance
  2. git
  3. npm

Amazon’s free tier ec2

Amazon EC2
ec2 instances are elastic compute general purpose storage servers that mean that they can resize the compute capacity in the cloud based on load .
750 hours per month of Linux, RHEL, or SLES t2.micro instance usage
Expires 12 months after sign-up.

Some other products are also covered under free tier which may come in handy for setting up the complete complatorm .Here is a quick summary

1.Amazon S3
it is a storage server. Can be used to store media file like image s, music , videos , recorded video etc .

2.Amazon RDS
It a relational database server . If one is using mysql or postgress for storing session information or user profile data . It is good option .

3.Amazon SES
email service. Can be used to send invites and notifications to users over mail for scheduled sessions or missed calls .

4.Amazon CloudFront
It is a CDN ( content delivery network ) . If one wants their libraries to be widly available without any overheads . CDN is a good choice .

Server Setup

Set up environment by installing nvm  , npm  and git ( source version control)

1. NVM ( node version manager )

cURL:

curl -o- https://raw.githubusercontent.com/creationix/nvm/v0.31.1/install.sh | bash

or Wget:

wget -qO- https://raw.githubusercontent.com/creationix/nvm/v0.31.1/install.sh | bash</code>

To check installation

command -v nvm
nvm

2. NPM( node package manager)

sudo apt-get install npm

Screenshot from 2016-05-16 12-41-42

2. Git

sudo apt-get install git

Screenshot from 2016-05-17 11-25-01

 SSL certificates

Since 2015 it has become mandatory to have only https origin request WebRTC’s getUserMedia API ie Voice, video, geolocation , screen sharing require https origins.
Note that this does not apply to case where its required to only serve peer’s media Stream or using Datachannels . Voice, video, geolocation , screen sharing now require https origins

For A POC purpose here is th way of generating a self signed certificate
Transport Layer Security and/or Secure Socket Layer( TLS/SSL) is a public/private key infrastructure.Following are the steps

1.create a private key
openssl genrsa -out webrtc-key.pem 2048

2.Create a “Certificate Signing Request” (CSR) file
openssl req -new -sha256 -key webrtc-key.pem -out webrtc-csr.pem

3.Now create a self-signed certificate with the CSR,
openssl x509 -req -in webrtc-csr.pem -signkey webrtc-key.pem -out webrtc-cert.pem

However in production or actual implementation it is highly recommended to use a signed certificate by CA as For examples include
Godaddy (https://ca.godaddy.com/web-security/ssl-certificate) , Comoddo (https://ssl.comodo.com/) , Global Sign (https://www.globalsign.com/en/ssl/managed-ssl/) , Symantec (https://www.symantec.com/ssl-certificates) etc .

Web Server

create https certificate using self generate or purchased SSL certificates using fs , node-static and https modules . To know how to create self generated SSL certificates follow section above on SSL certificates.

var fs = require(‘fs’);
var _static = require(‘node-static’);
var https = require(‘https’);

var file = new _static.Server("./", {
cache: 3600,
gzip: true,
indexFile: "index.html"
});

var options = {
key: fs.readFileSync(‘ssl_certs/webrtc-key.pem’),
cert: fs.readFileSync(‘ssl_certs/webrtc-cert.pem’),
ca: fs.readFileSync(‘ssl_certs/webrtc-csr.pem’),
requestCert: true,
rejectUnauthorized: false
};

var app = https.createServer(options, function(request, response){
request.addListener(‘end’, function () {
file.serve(request, response);
}).resume();
});

app.listen("8080");

Web servers work with the HTTP (and HTTPS) protocol which is TCP based. As a genral rule TCP establishes connection whereas UDP send data packets

 

Scoketio signalling server as npm

Socket.io determines which of the following real-time communication method is suited to the particular client and its network bandwidth .

  • WebSocket
  • Adobe Flash Socket
  • AJAX long polling
  • AJAX multipart streaming
  • Forever Iframe
  • JSONP Polling

The socket.io server needs a HTTP Server for initial handshake.

The general steps for socketio signalling server are:

1.require socket.io and keep the reference. like
var io = require(‘socket.io’)

2.Create your http / https server
outline in section on webserver

3.bind your http and https servers (.listen)
io.listen(app, {
log: false,
origins: ‘*:*’
});

4. Optionally set transport
io.set(‘transports’, [
‘websocket’
]);

4.setup io events as
io.sockets.on(‘connection’, function (socket) {

//Do domething
});

Note that Socket.io or websockets require an http server for the initial handshake.
<pre>Install ssocketio npm module</pre><pre>
npm install socket.io
[/sourcecode ]

Complete code for signalling server

var io = require(‘socket.io’).listen(app, {
log: false,
origins: ‘*:*’
});

io.set(‘transports’, [
‘websocket’
]);

var channels = {};

io.sockets.on(‘connection’, function (socket) {

console.log("connection ");
var initiatorChannel = ”;

if (!io.isConnected) {
io.isConnected = true;
}

socket.on(‘namespace’,function(data){
onNewNamespace(data.channel, data.sender);
});

socket.on(‘new-channel’, function (data) {
if (!channels[data.channel]) {
initiatorChannel = data.channel;
}
console.log("————new channel ", data.channel , " by " , data.sender);
channels[data.channel] = {
channel: data.channel,
users:[data.sender]
};

});

socket.on(‘join-channel’, function (data) {
console.log("————join channel ", data.channel , " by " , data.sender);
channels[data.channel].users.push(data.sender);
});

socket.on(‘presence’, function (channel) {
var isChannelPresent = !! channels[channel.channel];
console.log("presence for channel " ,isChannelPresent);
socket.emit(‘presence’, isChannelPresent);
});

socket.on(‘disconnect’, function (channel) {
});

socket.on("admin_enquire",function(data){
switch (data.ask){
case "channels":
socket.emit(‘response_to_admin_enquire’, channels);
break;
case "channel_clients":
socket.emit(‘response_to_admin_enquire’, io.of(‘/’ + data.channel).clients());
break;
default :
socket.emit(‘response_to_admin_enquire’, channels);
}

});

});

function onNewNamespace(channel, sender) {
console.log(" —–> onNewNamespace ", channel);

io.of(‘/’ + channel).on(‘connection’, function (socket) {

var username;
if (io.isConnected) {
io.isConnected = false;
socket.emit(‘connect’, true);
}

socket.on(‘message’, function (data) {
if (data.sender == sender) {
if(!username) username = data.data.sender;
socket.broadcast.emit(‘message’, data.data);
}
});

socket.on(‘disconnect’, function() {
if(username) {
socket.broadcast.emit(‘user-left’, username);
username = null;
}
});
});
}

 

WebRTC main HTML5  project

This is the front  end section of the whole exercise . It contains JavaScript , css and html5 to make a webrtc call

<html lang=en>
<head>
<title>WebRTC Call</title>

<meta http-equiv=Content-Type content="text/html; charset=UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1">

	<link rel=stylesheet href="https://ec2-54-193-124-35.us-west-1.compute.amazonaws.com:8084/minScripts/webrtcdevelopment_header.css">
<script src="https://ec2-54-193-124-35.us-west-1.compute.amazonaws.com:8084/minScripts/webrtcdevelopment_header.js"> </script>

<style type="text/css">
video{
width:100% !important;
}
body{
background: #2B2B2B;
}
</style>
</head>

<body id="pagebody">
<div id="elementToShare" class="container-fluid">
<!-- ................................ top panel ....................... -->
<div class="row topPanelClass" >
<div id="topIconHolder" >
<ul id="topIconHolder_ul">
	<li hidden> <span id="username" class="userName" hidden>a</span></li>
	<li hidden> <span id="numbersofusers" class="numbers-of-users" hidden></span></li>
	<li> <span id="HelpButton" class="btn btn-info glyphicon glyphicon-question-sign topPanelButton" data-toggle="modal" data-target="#helpModal" > Help </span></li>
</ul>
</div>
</div>
<!-- .............alerts................. -->
<div class="row" id="alertBox" hidden="true"></div>
<!-- .......................... Row ................................ -->
<div class="row thirdPanelClass">
<div class="col-xs-12 videoBox merge" id="videoHold">
<div class="row users-container merge" id="usersContainer" >
<div class="CardClass" id="card">

<!-- when no remote -->
<div id="local" class="row" hidden="">
<video name="localVideo" autoplay="autoplay" muted="true" />
</div>
<!-- when remote is connected -->
<div id ="remote" class="row" style="display:inline" hidden>
<div class="col-sm-6 merge" class="leftVideoClass" id="leftVideo">
<video name="video1" hidden autoplay="autoplay" muted="true" ></video>
</div>
<div class="col-sm-6 merge" class="rightVideoClass" id="rightVideo" >
<video name="video2" hidden autoplay="autoplay" ></video>
</div>
</div>
</div>
</div>
</div>
</div>
<!--modal help -->
<div class="modal fade" id="helpModal" role="dialog">
<div class="modal-dialog modal-lg">
<div class="modal-content">
<div class="modal-header">
<button type="button" class="close" data-dismiss="modal">&times;</button>
<h4 class="modal-title">Help</h4>
</div>
<div class="modal-body">
WebRTC Runs in only https due to getusermedia security contraints
</div>
<div class="modal-footer">
<button type="button" class="btn btn-default" data-dismiss="modal">Close</button>
</div>
</div>
</div>
</div>
</div>
</body>

	<link rel=stylesheet href="https://ec2-54-193-124-35.us-west-1.compute.amazonaws.com:8084/minScripts/webrtcdevelopment.css">
<script src="https://ec2-54-193-124-35.us-west-1.compute.amazonaws.com:8084/minScripts/webrtcdevelopment.js"> </script>

<script>
$('document').ready(function(){

 sessionid= init(true);

 var local={
 localVideo: "localVideo",
 videoClass:"",
 userDisplay:false,
 userMetaDisplay:false 
 };

 var remote={
 remotearr: ["video1" , "video2"],
 videoClass:"",
 userDisplay:false,
 userMetaDisplay:false 
 };

 webrtcdomobj= new WebRTCdom(
 local,remote
 );

 var session ={
 sessionid : sessionid,
 socketAddr: "https://localhost:8084/"
 };

 var webrtcdevobj = new WebRTCdev ( session, null, null , null );

 startcall();
});
</script>
</html>
Screenshot from 2016-05-17 12-12-37.png

Common known issues:

1.Opening page https://<web server ip>:< web server port>/index.html says insecure

This is beacuse the self signed certificates produced by open source openSSL is not recognized by a trusted third party Certificate Agency.
A CA ( Certificate Authority ) issues digital certificate to certify the ownership of a public key for a domain.

To solve the access issue goto https://<web server ip>:< web server port> and given access permission such as outlined in snapshot below

image

2.Already have given permission to Web Server , page loads but yet no activity .

if you open developer console ( ctrl+shift+I on google chrome ) you will notice that there migh be access related errros in red .
If you are using different server for web server and signalling server or even if same server but different ports you need to explicity go to the signalling server url and port and give access permission for the same reason as mentione above.
3.no webcam capture on opening the page

This could happen due to many reasons

  •  page is not loaded on https
  • browser is not webrtc compatible
  • Media permission to webcam are blocked
  • the machine does have any media capture devices attached
  •  Driver issues in the client machine while accessing webcams and mics .

4.socketio + code: 0, message: “Transport unknown”

Due to the version  v1.0.x of socket.io while performing handshake . To auto correct this , downgrade to v0.9.x

 

 

TFX Widgets Development

TFX is a modular widget based WebRTC communication and collaboration solution. It is a customizable solution where developers can create and add their own widget over the underlying WebRTC communication mechanism . It can support extensive set of user activity such as video chat , message , play games , collaborate on code , draw something together etc . It can go as wide as your imagination . This post describes the process of creating widgets to host over existing TFX platform .

Prerequisites

It is required to have TFX Chrome extension installed and running from Chrome App Store under above . To do this follow the steps described in TangoFX v0.1 User’s manual.

Test TFX Sessions  ?

TFX Sessions uses the browser’s media API’s , like getUserMedia and Peerconnection to establish p2p media connection . Before media can traverse between 2 end points the signalling server is required to establish the path using Offer- Answer Model . This can be tested by making unit test cases on these function calls .

TFX Sessions uses socketio based handshake between peers to ascertain that they are valid endpoints to enter in a communication session . This is determined by SDP ( Session Description Parameters ) . The same can be observed in chrome://webrtc-internals/ traces and graphs .

TangoFX v8 Developer's manual - Google Docs

How to make widgets using TFX API ?

Step 1:  To make widgets for TFX , just write your simple web program which should consist of one main html webpage and associated css and js files for it .

Step 2 : Find an interesting idea which is requires minimal js and css . Remember it is a widget and not a full fleshed web project , however js frameworks like requirejs , angularjs , emberjs etc , work as well.

Step 3: Make a compact folder with the name of widget and put the respective files in it. For example the html files or view files would go to src folder , javascript files would goto js folder , css files would goto css folder , pictures to picture folder , audio files to sound folder and so on .

Step 4 : Once the widget is performing well in standalone environment , we can add a sync file to communicate the peer behaviors across TFX network . For this we primarily use 2 methods .

  • SendMessage : To send the data that will be traversed over DataChannel API of TFX . The content is in json format and will be shared with the peers in the session .
  • OnMessage : To receive the message communicated by the TFX API over network

Step 5: Submit the application to us or test it yourself by adding the plugin description in in widgetmanifest.json file . Few added widgets are

[
{
"plugintype": "code",
"id" 	:"LiveCode",
"type" 	: "code",
"title" : "Code widget",
"icon" 	: "btn btn-style glyphicon glyphicon-tasks", 
"url"	: "../plugins/AddCode/src/codewindow.html"
},

{
"plugintype": "draw",
"id" 	:"Draw",
"type" 	: "draw",
"title" : "Code widget",
"icon" 	: "btn btn-style glyphicon glyphicon-pencil", 
"url"	: "../plugins/AddDraw/src/drawwindow.html"
},

{
"plugintype": "pingpong",
"id"   :"Pingpong",
"type" : "pingpong",
"title": "ping pong widget",
"icon" : "btn btn-style glyphicon glyphicon-record", 
"url"  : "../plugins/AddPingpong/src/main.html"
}
]

Step 6 : For proper orientation of the application make sure that overflow is hidden and padding to left is atleast 60 px so that it doesnt overlap with panel
padding-left: 60px;
overflow: hidden;

Step 7 : Voila the widget is ready to go .


Simple Messaging Widget

For demonstration purpose I have summarised the exact steps followed to create the simple messaging widget which uses WebRTC ‘s Datachannel API in the back and TFX SendMessage & OnMessage API to achieve

Step 1 : Think of a general chat scenario as present in various messaging si

Step 2: Made a folder structure with separation for js , css and src. Add the respective files in folder. It would look like following figure:

TangoFX v8 Developer's manual - Google Docs (1)
Step 3 : The html main page is

<html>
<head>
<!-- jquerry -->
<script src="../../../../js/jquery/jquery.min.js"></script>
<link rel="stylesheet" type="text/css" href="../../../../css/jquery-ui.min.css"/>
<link rel="stylesheet" type="text/css" href="../css/style.css">
</head>

<body>
<div id="messages" class="message_area">
<textarea disabled class="messageHistory" rows="30" cols="120" id="MessageHistoryBox"></textarea>
   <br/>
<input type ="text" id="MessageBox" size="120"/>
</div>
</body>
<script src="../js/sync.js" type="text/javascript"></script>
</html>

Step 4 : The contents of css file are

body {padding: 0; margin: 0; overflow: hidden;}

.message_area{
	  padding-left: 60px;
}
.messageHistory{
	background-color: transparent;
	background: transparent;
}

Step 5 : The contents of js file

//send message when mouse is on mesage dicv ans enter is hit
$("#messages").keyup(function(event){
    if(event.keyCode == 13){
    var msg=$('#MessageBox').val(); 
    //send to peer 
    var data ={
       "msgcontent":msg
      }
    sendMessage(data);
    addMessageLog(msg);
    $("#MessageBox").val('');
    }
});

function addMessageLog(msg){
    //add text to text area for message log for self 
 $('#MessageHistoryBox').text( $('#MessageHistoryBox').text() + '\n'+ 'you : '+ msg);
}

// handles send message
function sendMessage(message) {
      var widgetdata={
      "type":"plugin",
      "plugintype":"relaymsg",
      "action":"update",
      "content":message
      };
  // postmessage
  window.parent.postMessage(widgetdata,'*');
}

//to handle  incoming message
function onmessage(evt) {
    //add text to text area for message log from peer
  if(evt.data.msgcontent!=null ){
      $('#MessageHistoryBox').text( $('#MessageHistoryBox').text() +'\n'+ 'other : '+ evt.data.msgcontent );
  }
}

window.addEventListener("message",onmessage,false);

Step 6: The end result is :

TangoFX v8 Developer's manual - Google Docs (2)


Developing a cross origin Widget ( XHR)

Let us demonstrate the process and important points to create a cross- origin widget :

step 1 : Develop a separate web project and run it on a https

step 2 : Add the widget frame in TFX . Following is the code I added to make an XHR request over GET

var xmlhttp;
xmlhttp=new XMLHttpRequest();
xmlhttp.onreadystatechange=function()
  {
  if (xmlhttp.readyState==4 && xmlhttp.status==200)
    {
    document.getElementById("myDiv").innerHTML=xmlhttp.responseText;
    }
  }

xmlhttp.open("GET","https://192.168.0.119:8000/TFXCrossSiteProj/files/document1.txt",true);
xmlhttp.send();

step 3 : Using self made https we have have to open the url separately in browser and give it explicit permission to open in advanced setting. Make sure the original file is visible to you at the widgets url .

TangoFX v8 Developer's manual - Google Docs (3)
step 4: Adding permission to manifest for access the cross origin requests

"permissions": [
  "tabs","http://*.google.com/",
  "https://192.168.0.119:8000/TFXCrossSiteProj"
],

Step 5 : Rest of the process are similar to develop a regular widget ie css and js .

Step 6: Resulting widget on TFX

TangoFX v8 Developer's manual - Google Docs (4)

Note 1 :
In absence of changes to manifest file the cross origin request is meet with a Access-Control-Allow-Origin error .

Note 2:
While using POST the TFX responds with Failed to load resource: the server responded with a status of 404 (Not Found)

Note 3:
Also if instead of https http is used the TFX still responds with Failed to load resource: the server responded with a status of 404 (Not Found)


TFX WebRTC SaaS ( Software as a Service )

TFX  is WebRTC based communication platform built entirely on open standards making it extensively scalable. The underlying API completely masks the communication  aspect and lets the user enjoy an interactive communication session. It also supports easy to build widgets framework which can be used to build applications on the TFX platform .

TFX Sessions

TFX sessions is a part of TFX . It is a free Chrome extension WebRTC client that enables parties communicating and collaborating, to have an interactive and immersive experience. You can find it on Chrome Webstore here .

Features of TFX Sessions:

Through TFX, users can have instant multimedia Internet call sessions .

The core  features are :

  • No signin or account management
  • No additional requirement like Flash , Silverlight or Java
  • URL based session management
  • secure WebRTC based communication
  • complete privacy with no user tracking or media flow interruption
  • Ability to share session on social network platforms like Facebook , twitter , linkedin , gmail , google plus etc

Screenshot from 2015-05-19 11:47:42

  • ability to choose between multiple cameras

Screenshot from 2015-05-19 11:53:06

The TFX platform has developer friendly APIs to help build widgets. Some of the pre-built widgets available on TFX are:

  • Coding

screen4

  • Drawing

Screenshot from 2015-04-13 16:28:22

  • Multilingual chat

screen6

  • Screen sharing

screen5

TFX sessions is free for personal use and can be downloaded from Chrome Webstore.

What is the differentiator with other internet call services?

  • No registration , login for account management required
  • Communication is directly between peer to peer ie information privacy.
  • Third party apps , services can be included as widgets on TFX platform.
  • Can be skimmed to be embedded inside Mobile app webview , iframe, other portals etc anytime .

TFX Sessions Integration Models

The 3 possible approaches for TFX Integration  in increasing order of deployment time are  :

  1. WebSite’s widget on TFX chrome extension .
  2. Launch TFX extension in an independent window from website
  3. TFX call from embedded Window inside the website page

1. WebSite’s widget on TFX chrome extension .

This outlines the quickest deliverable approach of building the websites own customized widget on TFX widgets API and deployed on existing TFX communication setup .

Step 1 : Login using websites credentials to access the content

WebSite’s widget on TFX chrome extension

Step 2 : Access the website with the other person inside the TFX “ Pet Store “ Widget

WebSite’s widget on TFX chrome extension

2. Launch TFX in an independent window from “Click to Call” Button on website

This approach outlines the process of launching TFX in an independent window from a click of a button on website. However it is a prerequisite to have TFX extension installed on your Chrome browser beforehand.

Step 1 : Have TFX installed on chrome browser
Step 2 : Trigger and launch TFX chrome extension window on click of button on webpage

Launch TFX extension in an independent window from website

3. TFX call from embedded Window inside Website page

This section if for the third approach which is of being able to make TFX calls from embedded Window inside of the webpage. Refer to sample screen below :

Step 1 : Have TFX embedded in an iframe inside the website

TFX Integration Models (3)
Step 2 : Make session on click of button inside the iframe.

TFX Integration Models (4)


Technical Details about TFX like architecture , widgets development , components description etc can be found here : TFX Platform

To get more information about TFX and/or making custom widgets get i touch with me at tara181989@gmail.com or email to info@above-inc.com

TFX platform

So I haven’t written anything worthy in a while , just published some posts that were lying around in my drafts . Here I write about the main thing . some thing awesome that I was trying to accomplish in the last quarter .

<< TFX is now live in chrome store , open and free for public use . No signin or account required , no advertisements   : https://chrome.google.com/webstore/detail/tfx-sessions/aochimdcllmgleokpnlabijehdlmkdga >>

TFX Sessions is a plug and play platform for VoIP ( voice over IP ) scenarios.  Intrinsically it  is a very lightweight API package and shipped in form of a Chrome Extension . It is a turn-key solution when parties want instant audio/audio communication without any sign-in ,plugin installation or additional downloads  . Additionally TFX Sessions is packaged with some interesting plugins which enable the communicating parties to get the interactive and immersive experience as in a face to face meeting.

There is a market requirement of making a utterly simple WebRTC API  that has everything needed to build bigger aggregate projects but the available solutions are either just to basic or much too complex . So I initially started writing my own getuserMedia APIs, but left it midway and picked up simplewebrtc API instead for want of time .Then I focused on the main crux  of the project which was the widget API and ease of integration.

How Does TFX Sessions Work ?

  • Signalling channel establishes the session using Offer- Answer Model
  • Browser’s  media API’s , like getUserMedia and Peerconnection are used for media flow
  • Media only flows peer to peer
TFX WebRTC platform architecture . socket io signalling

TFX WebRTC platform architecture . socket io signalling

A Widget is essentially any web project that wants communication over webrtc channel . Once the platform is ready I have core APIs , widgets and signalling server. Then came up the subject of enterprise internet blocking my communication stream . Time for TURN ( Coturn in my case ).

TFX create / join room

TFX create / join room

TFX startup screen

TFX startup screen

Components of TFX

Client Side Components of tangoFX :

broplug API
Inhouse master library for TangoFX. Makes the TFX sessions platform .Masks the low level webrtc and socketio functions .
Provides simple to use handles for interesting plugins development in platform .
simplewebrtc
performs webrtc support , peer configuration , wildemitter , utils , event emitters , JSON formatter , websocket , socket namespace , transport , XHR more like so
socket.io.js
exports and listener for real time event based bidirectional communication
Jquerry
JS library for client side scripting
Bootstrap
HTML and CSS-based design templates for typography, forms, buttons, navigation and other interface components.

Server Side components

Signalling Server -signal master
socket.io server for webrtc signalling
TURN -Coturn
TURN protocol based media traversal for connecting media across restricting domains ie firewalls, network policies etc .
redis Data structure to maintain current and lost sessions

TFXSessions Components

Architecture

So here is the final architecture of TFX chrome extension widget based platform .

  • The client side contains widgets , chore extension APIs , chrome’s WebRTC API’s , socket.io client for signalling , HTML5 , Jquerry , Javascript , and CSS for styling.
  • The Server Side of the solution contains socket.io server for signalling , manuals and other help/support materials , HTTPS certificate and TURN server implementation for NAT .
TFX whitepaper v2.0

TFX platform Server client components . WebRTC media and socketio communication . Build as chrome Extension

Salient Features

  • The underlying technology of TangoFX is  webrtc with socket based signalling  . Also it adheres to the latest standards of W3C , IETF and ITU on internet telephony .
  • TangoFX sessions is extremely scalable and flexible due to the abstraction between communication and service development. This make it a piece of  cake for any web developer using  TangoFX interface to add his/ her own service easily and quickly without diving into the nitty gritties .
  • TangoFX is currently packaged in a chrome extension supported on chrome browser on desktop operating system like window , mac , linux etc .
  • The call is private to both the parties as it is peer-to-peer meaning that the media / information exchanged by the parties over TangoFX does not pass through an intervening server as in other existing internet calling solutions.
  • TangoFX is very adaptive to slow internet and can be used across all kinds of networks such as corporate to public without being affected by firewall or restricting policies  .

TFX Widget Screens

Alright so that’s there . Tada the platform is alive and kicking . Right now in beta stage however . Intensive testing going on here . However here are some screenshots that are from my own developer version .

TFX recording widget

TFX recording widget

TFX face detection and overlay widget

TFX face detection and overlay widget

TFX multilingual communication

TFX multilingual communication

TFX screen-sharing

TFX screen-sharing

TFX video Filters

TFX video Filters

TFX audio visualizer

TFX audio visualizer

TFX text messaging widget

TFX text messaging widget

TFX cross domian access . flicker here

TFX cross domain access . flicker here

TFX draw widget

TFX draw widget

TFX code widget supportes many programming languages

TFX code widget supportes many programming languages

TFX  webrtc dynamic stats

TFX webrtc dynamic stats

TFX  introduction widget

TFX introduction widget

Note that the widgets described above have been made with the help of third party APIs.

TFX Sessions Summary

We saw that TFX is WebRTC based communication and collaboration solution .It is build on Open standards from w3c , IETF , Google etc.
Scalable and customizable. Immersive and interactive experience .
Easy to build widgets framework using TangoFX APIs.

TFX User Manual : https://tfxserver.above-inc.com/static/manuals/material/TangoFXv0.1Usersmanual.pdf

TFX Developer’s Manual : https://tfxserver.above-inc.com/static/manuals/material/TangoFXv0.1Developersmanual.pdf


Steps for building and deploying WebRTC solution

Step 1  : USE Local machine to test the client server WebRTC funcationality

Pick any WebRTC API and run its demos . It works kool . download and run in local-machine with nodejs server . Awesome . Everything is Awesome !!

You can learn more about some WS based WebRTC API here:  https://altanaitelecom.wordpress.com/2014/12/02/current-state-of-webrtc/.

If you are a diehard telecom engineer and only want SIP based WebRTC solutions go here : https://altanaitelecom.wordpress.com/2014/07/16/interoperability-between-webrtc-sip-phones-and-others/

Steps for building and deploying WebRTC solution Step 1 : Pick a WebRTC API and run locally ( ie open 2 browsers and run on local machine )

Steps for building and deploying WebRTC solution
Step 1 : Pick a WebRTC API and run locally ( ie open 2 browsers and run on local machine )

Step 2 : Use cloud Server and different client Browsers  

Now what good is it doing to anyone if its running locally on my machine with addresses like localhost and 127.0.0.1  . Let us put it on the cloud and at-least let my colleague / friends enjoy it .  Cloud Web Server and Nodejs signalling server . That is okay use amazon’s Ec2. works for most of the people most of the time .

Steps for building and deploying WebRTC solution Step 2 : Put Server on cloud and WebRTC clients on different machine

Steps for building and deploying WebRTC solution
Step 2 : Put Server on cloud and WebRTC clients on different machine

Here is when we discover the issues of ICE ( Interactive Connectivity Establishment ) I have mentioned this in detail on the post NAT Traversal using STUN and TURN .  Briefly ICE helps us in coping up with NAT ( Network Address Traversal and Firewalls ) .

Note that this step only works if everyone you want to connect to is either on same intranet or on public internet without and UDP blocks / firewalls / restriction .

As we try to connect 2 WebRTC clients from different machine and different networks we find that network address from client’s OS and network card fails to connect to Signalling Server due to either Firewalls issues or other Network policies . We therefore use a STUN server to map the private IP to a publicly accessible IP that will help in completing the signalling

The Signalling is establishes using a STUN server for address mapping and NAT . One can use google’s default STUN server stun.l.google.com:19302. Easy and free .

Steps for building and deploying WebRTC solution Step 2.1 : Put Server on cloud and WebRTC clients on different machine + STUN for address discovery ( NAT traversal )

Steps for building and deploying WebRTC solution
Step 2.1 : Put Server on cloud and WebRTC clients on different machine + STUN for address discovery ( NAT traversal )

There you go everything is looking good from here now , both peers join the session successfully  , but the video may appear black . This is so because the media under most inter network conditions fails to flow between private and public network .

This is where step 3 comes into picture ie using a TURN ( media relay ) server .

Step 3: TURN server to Call people in a inter-network fashion 

Sure the architecture I have setup is bound to work everywhere where the network is open and public . However error in connectivity , errors in console , blank video are the problems that might appear when one tries to connect from private to public connections.

To bypass network firewalls , corporate net policies , UDP blocks and filters we require a TURN server which help in media traversal across different networks in a relay mechanism.

Now we have 3 options to choose from

1.  Use a wildly popular http://numb.viagenie.ca/

2. Build your own TURN server with RFC 5766 ( COTURN )  , or rather easier would be to use any open source TURN server code available in Github.

3. Pay and use a commercial TURN service provider or you can even use their trail version to see if things work out for you ( example Xirsys)  .

Remember you can use any TURN service it does not affect your WebRTC API functionality . All we need to do is add it to Peerconnection configuration like

</address><address>peerConnectionConfig: {
iceServers:[</address><address>{"url": < stunserver address >},</address><address>{"username":"xx","url":< turn server address transport=udp>,"credential":"yy"},</address><address>{"username":"xx","url":< turn server address transport=tcp> , "credential":"yy"}]</address><address>},</address><address>

There we go , now anyone from anywhere should be able to use our WebRTC setup for making audio , video calls or just exchanging data via DataChannel ( like screen-sharing , file transfer , messages , playing games , collaborative office work etc )  .

Steps for building and deploying WebRTC solution TURN based media Relay for WebRTC traffic

Steps for building and deploying WebRTC solution
TURN based media Relay for WebRTC traffic

The setups covers scenarios wherein user is on office corporate network , home network , mobile network , no problem as long as he / she has a webrtc enables browser ( read Chrome , Mozilla , Opera ) .

It is noteworthy that ideally voice should be traversing on TCP while video and data can go around in UDP however unless restrained the WebRTC API’s self determine the best protocol to route the packets / stream .


Read more about best WebRTC frameworks and code in this book

WebRTC SIP / IMS solution

We started in winters on 2012 with Webrtc . At time time it just looked like a new car design that might fade away when new ones comes .

What really is WebRTC ? I made an entry on it  here .

Around nov – dec 2012 , team and I spend the time learning the nitty grities of HTML5 based media operation and Javascript sip stack of SIPML. I remeber toward the end of the year ie before Christmas , We were done with the explanation and education aspects of WebRTC , a technology that will revolutionize communication in ages to come , at-least so says the numerous other blogs ,  and documents i read so far .


The flow between

client BOB -> webrtc2sip Gateway -> SIP server -> client Alice

can be  understood with the callflow of a simple SIP Invite initiated from one html page towards another which passes through the configuration of gateway to IMS world ,  SIP Telecom Application server , Database , nodes of IMS environment etc.

For the purpose of a simple Explanation a simplified call flow ca be depicted as ,

webrtccallflow

A very high level architecture of solution deployment in IMS world could be

solution arch2

As the solution matures into a full fleshed project . The alpha version has been released with the following feature set . The WebRTC platform Suite offers a easily deploy-able solution to enable communication

Alpha Release WebRTC platform Suite

  • Single Sign On
    •  Login with id and password to access all services
  • Audio / Video Call
    •      Call Hold / Call Transfer
  • Messaging:
    •      SIP Instant Messaging
    •      Message to Facebook Messenger
    •      Message delivered as Email
  • Chatroom :
    • group chat between multiple users . Room is created for set of users .
  • Video Conferencing :
    • video chat between multiple parties . Room is created for set of users .
  • File Transfer :
    • Sharing of files from local to remote , in peer-to-peer and broadcasting fashion .
  • Third party Webservices :
    • Widgets like calendar , weather , stocks , twitter are embedded.
  • Visual Voice Mail :
    • Record and deliver voice message to recipients voice mail inbox which can be accessed/ played from web client .
  • Phonebook :
    •      cloud integration
    •      add new entries
    •      add photos to contacts identity
    •      import contacts from google account
  • Click to Call :
    •      Drop down list of contacts form mail call console
    •      2 step Click to call from Phonebook
  •  Presence :
    •      Publish online / offline status
    •      Use Subscribe / notify requests of SIP
  • WS to SIP Gateway :
    • Conversion between the signal coming from the  WebRTC  and SIP client  to the  IMS core
    • Conversion of  “voice/video ” media  between sRTP and RTP
    • Conversion of other media (data channel) towards MSRP and Transcoding.
  • Support of ICE procedure .
    • Implementation of a STUN server. 
    • QoS Support  

Beta Release WEBRTC PLATFORM SUITE

  • Logs :
    •      calls logs
    •      Message logs
  • User Profile :
    •      user details like address , email and social networking accounts
    •      Phonenumber for GSM integration through SMS
    •      User’s Media storage like Pictures , profile picture , Audio , video
    •      File sharing documents storage for future access in the same format
  • Real Time and Offline Analytics
    • service usage with graphical and tabular history  trends
  • Session Management :
    •       Single Sign-on
    •       Forgot password regeneration using secure question
    •       Registration of new user account
    •       Logout and clearance of session parameters
  • Security :
    •      No redirection to any page through url entry without valid session
    •      No going back to home page after logout by back button on browser
    •      No data vulnerability
    •      Multiple login through different devices handled
  • OAuth :
    •      Login via IMAP / token through facebook and Google
  • Phonebook with Presence functionality inbuild .
  • Directory Service based on country / region
  • Geolocation of approximate location detection of device logged in and visibility to  others

webrtc solution

WebRTC client deployment view , accessible devices , network elements

WebRTC deploymenet overview and inetraction with other network elemets such as gateway , cloud storage ,  sipserver , IMS

WebRTC deploymenet overview and inetraction with other network elemets such as gateway , cloud storage , sipserver , IMS