Kamailio DNS and NAT

  • DNS sub-system in Kamailio
    • DNS failover
    • DNS load balancing
    • NAT ( Network Address Translation)
  • NAT ( Network Address Translation)
  • Why is NAT is important in SIP?
  • Types of NAT solutions
  • NAT behaviours
  • RTP NAT
  • Fixing NAT
  • NAT Traversal Module
    • Why use keepalive when Registrations are already there for NATing ?
    • How keepalives work for NATing ?
    • function nat_keepalive()
    • Params
    • Functions
    • client_nat_test()
      • fix_contact()
      • nat_keepalive()
    • Pseudo Variables
    • Statistics
  • NATHelper Module
    • NAT pinging types
      • UDP packet
      • SIP request
    • params
    • functions
    • Pseudo Variables
    • RPC Commands

In this article, we discuss Nating in a SIP Server like Kamailio. Types of NAT pings, their behaviour and types. Also some implementation of some of the Kamailios modules like

  • NAT Traversal module
  • NAT helper module
  • STUN module

A repository for extensive application of kamailio modules for various usecases is https://github.com/altanai/kamailioexamples

DNS sub-system in Kamailio

To resolve hostname into IPs Kamailio can do either of below

  • use libresolv and a combination of the locally configured DNS server /etc/hosts and the local Network Information Service (NIS/YP a.s.o) or
  • cache the query results and first look into internal cache

DNS failover – if destination resolves to multiple addresses tm can try all of them until it finds one to which it can successfully send the packet or it exhausts all of them, with internal DNS cache. Also used when the destination host doesn’t send any reply to a forwarded invite within the SIP timeout interval (tm fr_timer parameter).

DNS load balancing – SRV based load balancing with weight value in the DNS SRV record.

  • (-) Only the locally configured DNS server (usually in /etc/resolv.conf) is
    used for the requests (/etc/hosts and the local Network Information Service are ignored).
    • optional: disable the DNS cache (use_dns_cache=off or compile without -DUSE_DNS_CACHE).
  • (-) DNS cache uses extra memory
    • optional: disable the DNS cache.
  • (-) DNS failover introduces a very small performance penalty
    • optional: disable the DNS failover (use_dns_failover=off).
  • (-) DNS failover increases the memory usage (the internal structures used to represent the transaction are bigger when the DNS failover support is compiled).
    • optional: compile without DNS failover support (DUSE_DNS_FAILOVER).Turning it off from the config file is not enough in this case (the extra memory will still be used).

NAT ( Network Address Translation)

Network address translation replaces the IP address within packets with a different IP address which internet endpoints can relate with. This enables multiple hosts in a private subnet with their pwn private address ( 10.x.x.x or 192.x.x.x etc ) to share single public IP address interface, to access the Internet.

NAT is bidirectional- If the private ip:port got translated to public ip:port on the inside interface while entering outside internet, on arriving from outside interface it will get translated from public ip:port to private ip:port.

For a SBC ( Session border controller ) or where the kamailio server is directly customer facing, where you dont have a private line or VPN to clients, then it is often encountered with NATed endpoints. Read more about NAT traversal using STUN and TURN here

Why is NAT is important in SIP?

These characteristics of SIP design and operation flows demonstrate why NAT solutions are so important

  • RFC 3261 for SIP presumed end-to-end reachability and does not specify much around ANT issues .
  • No NLRI (Network Layer Reachability Information) translation layer exists, such as DNS or ARP
  • SIP is designed to used RTP which uses dynamically allocated ports to stream media.
    It is comparable to FTP which creates ephemeral connections on unpredictable dynamic ports to send multiplexed data and “metadata”, instead of protocol like HTTP where all data is sent on same connection.
  • UDP (default transport for SIP) is connection less and session tracking requires these be mapped onto a statelful flow, rigorous keepalives and other such techniques like using TCP instead have their own tradeoffs
  • since sip packets put network and transport information right on sip header they are limited by the rateability and awareness of their network interface thereby prevent other endpoint from reaching its ip or port

Types of NAT solutions

  • Client-side NAT traversal – clients are responsible for identifying their WAN NLRI and adding ip and port to navigate them in outside world
  • Server-side NAT traversal – SIP server should discover the client’s WAN addressing while clients continue to work transparently behind NAT. Requires that DIP server look at the source and destination ip and port of actual packets instead of relying on the encapsulated sip headers and SDP body.
  • ALG (Application Layer Gateways) – mostly applied at router itself. wodk by susbtitung public IP/port information inplace of provate and vice versa for return packets. Limitataions – they dont provide a fullproof fix example they may fix Via but not the Contact address or SDP body or RTP ports
WebRTC signalling and media flow on Open public network
Open network leading to smooth p2p media stream

Far-end NAT traversal solution ( TURN server)

WebRTC media flow when peers are behind NAT . Uses TURN relay mechanism
public private ip mapping , firewalls and private network obstruct p2p media flow. TURN is useful to relay media pckets

NAT behaviours

Cone NATSymmetric NAT
Local client performs an outbound connection to a remote UA and a dynamic rule is created for the destination IP tuple, allowing the remote machine to connect back.Local client allows inbound connections from a specific source IP address and port, also NAT assigns a new random source port for each destination IP tuple
Further subdivied into:
1. Full Cone NAT
2. Restricted Cone NAT – all requests from the same internal IP address and port are mapped to the same external IP address and port.
-3. Port-Restricted Cone NAT

RTP NAT

NAT not only applies to sip signalling packets but also to RTP. Even RTP packets are not able to transverse accross private -public network interfaces to the right place across a NAT’d connection.

To solve two-way media RTP performs RTP latching, where client listens for at least one RTP frame arriving at the destination port it advertised, and harvests the source IP and port from that packet and uses that for the return RTP path. RTP latching works out of the box for public RTP endpoints but not for ones behind NAT.

It is thus recommended to use an intermediate RTP relay such as RTPengine on kamailio. It is controlled via a UDP control socket by kamailio as an external process. More on installation and descrition of RTP engine on kamailio is covered here. When RTPengine control module receives RTP offer /answer from Kamailio, it opens a pair of RTP/RTCP ports to receive traffic and substitues in SDP. Doing so for both ends makes RTP engine come in the path of media stream packets for both directions.

A first INVITE has no no corresponding on NAT ( as no port has been allocated yet ) so the c= contact and m = media lines would look like

c=IN IP4 192.0.2.13.
m=audio 23767 RTP/AVP 0 101.

We can force RTP to go through a proxy by changing c line and m line to

c=IN IP4 address-of-proxy  
m=audio port-on-proxy RTP/AVP 0 101

A separate daemon called an RTP proxy (with a public IP address) that both user agents can send their audio to, can be setup by calling force_rtp_proxy().

Fixing NAT

When the client is behind NAT, following needs to be taken careof to provide smooth operation

  1. Ensuring Tranactional replies are sent to correct source address ( maybe using ;rport param and forcerport() method ) instead of just relying on via header transport protocol and port.
    example:
if (client_nat_test("3")){
    //CALL RE-INVITE/UPDATE Nat DETECTED $ci\n");
    force_rport();
    fix_contact();
    ...
}

also Change Media ip address to public IP

if(nat_uac_test("8") && search("Content-type: application/sdp")) {
        // RE-INVITE/UPDATE CALL fix SDP- NAT
        fix_nated_sdp("2");
}

Any far-end NAT traversal solution ( TURN server) if employed should stay in path of entire Dialog not just for initial INVITE transaction which many times results in ACK being dropped. This can be achived by adding Record-Route header of rr module to the initial INVITE request itself

Set the advertised address of the public-facing inetrface to the Public NAT IP using “listen” parameter

Ensure contact URI is NAT processed by using NATHelper modules which rewrites the domain portion of the Contact URI to contain the source IP and port of the request or reply. add_contact_alias([ip_addr, port, proto]) in NAThelper module which adds “;alias=ip~port~transport” parameter to the contact URI containing either received ip, port, and transport protocol or those given as parameters

Implement RTP proxy which performs NAT for streams such as rtpengine module

NAT Traversal Module

Provides far-end NAT traversal to kamailio’s SIP signalling. Its role is

  • detect user agents behind NAT
  • manipulate SIP headers so that user agents can continue working behind NAT transparently
  • keepalives to UA behind NAT to preserve their visibility in network

Some pros and cons of NATTravsersal module

  • (+) detect even UAs behind multiple cascaded NAT boxes, complex distributed env with multiple proxies
  • (+) handle env where incoming and outgoing paths are diff for SIP messages
  • (+) handle cases when routing path may even change between consecutive dialogs
  • (+) can work for other than registered UA’s also
  • (-) built for IPv4 NAT handling not adapted to support IPv6 session keepalives.

Why use keepalive when Registrations are already there for NATing ?

  1. NAT binding works for registered users who want incoming calls. However for cases like outgoing calls or for presence subscription notifications, failings registration implies inability to receive further in-dialog messages after the NAT binding expires. This artificial binding for registrations makes system unreliable and volatile as it doesnot guarantee the delivery of in-dialog messages for outgoing calls without registration renewal. Therefore keepalive are adopted which also works for unregistered users.
  2. Minimizes the traffic as only border proxies send keepalives which send keepalives statelessly, instead of having to relay messages generated by the registrars.
  3. Also for situations when DNS resolves diff proxies for outgoing or incoming path traditional register based keepalives fail to associate or dissociate correct routes.

How keepalives work for NATing ?

This mechanism works by sending a SIP request to a user agent behind NAT to make that user agent send back a reply. The purpose is to have packets sent from inside the NAT to the proxy often enough to prevent the NAT box from timing out the connection.

Module sends Keeplaives to preserve their visibility only in :

  • Registration – for user agent that have registered to for incoming calls, triggering keepalive for a REGISTER request.
  • Subscription – for presence agents that have subscribed to some events for receiving back notifications with SUBSCRIBE request.
  • Dialogs – for user agents that have initiated an outgoing call for receiving further in-dialog messages.
    When all the conditions to keepalive a NAT endpoint will disappear, that endpoint will be removed from the list with the NAT endpoints that need to be kept alive.

function nat_keepalive()

  • the function needs to be called on proxy directly interacting with UA behind NAT.
  • call only once for the requests (REGISTER, SUBSCRIBE or outgoing INVITEs) that triggers the need for network visibility.
  • call before the request gets either a stateless reply or it is relayed with t_relay()
  • for outgoing INVITE , it triggers dialog tracing for that dialog and will use the dialog callbacks to detect changes in the dialog state.

Dependencies – sl , tm and dialog module

Params

  • keepalive_interval – time interval between sending a keepalive message to all the endpoints that need being kept alive. A negative value or zero will disable the keepalive functionality.
modparam("nat_traversal", "keepalive_interval", 30) // 30 seconds keeplaive inetrval
  • keepalive_method – SIP method to use to send keepalive messages.usual ones are NOTIFY and OPTIONS. Default value is “NOTIFY”.
modparam("nat_traversal", "keepalive_method", "OPTIONS")
  • keepalive_from – SIP URI to use in the From header of the keepalive requests. default sip:keepalive@proxy_ip,with IP address of the outgoing interface
modparam("nat_traversal", "keepalive_from", "sip:keepalive@altanai.com")
  • keepalive_extra_headers – extra headers that should be added to the keepalive messages. Header must also include the CRLF (\r\n) line separator. Multiple headers can be specified by concatenating with \r\n separator.
modparam("nat_traversal", "keepalive_extra_headers", "User-Agent: Kamailio\r\nX-MyHeader: some_value\r\n")
  • keepalive_state_file – filename where information about the NAT endpoints and the conditions for which they are being kept alive is saved . It is used when Kamailio starts to restore its internal state and continue to send keepalive messages to the NAT endpoints that have not expired in the meantime. Also used at kamailio restart as it avoids losing keepalive state information about the NAT endpoints.
modparam("nat_traversal", "keepalive_state_file", "/var/run/kamailio/keepalive_state")

Functions

  • client_nat_test ()– Check if the client is behind NAT. Tests to be performed gievn by int can be :
    1 – tests if client has a private IP address or one from shared address space in the Contact field of the SIP message.
    2 – tests if client has contacted Kamailio from an address that is different from the one in the Via field.
    4 – tests if client has a private IP address or one from shared address space in the top Via field of the SIP message.

For example calling client_nat_test(“3”) will perform test 1 and test 2 and return true if at least one succeeds, otherwise false.

  • fix_contact() – replace the IP and port in the Contact header with the IP and port the SIP message was received from. Usually called after a succesfull call to client_nat_test(type)
if (client_nat_test("3")) {
    fix_contact();
}
  • nat_keepalive() – Triggers keepalive functionality for the source address of the request. When called it only sets some internal flags, which will trigger later the addition of the endpoint to the keepalive list if a positive reply is generated/received (for REGISTER and SUBSCRIBE) or when the dialog is started/replied (for INVITEs). For this reason, it can be called early or late in the script. The only condition is to call it before replying to the request or before sending it to another proxy. If the request needs to be sent to another proxy, t_relay() must be used to be able to intercept replies via TM or dialog callbacks.

If stateless forwarding is used, the keepalive functionality will not work. Also for outgoing INVITEs, record_route() should also be used to make sure the proxy that keeps the caller endpoint alive stays in the path.

if ((method=="REGISTER" || method=="SUBSCRIBE" ||
    (method=="INVITE" && !has_totag())) && client_nat_test("3"))
{
    nat_keepalive();
}

Pseudo Variables

  • $keepalive.socket(nat_endpoint)
  • $source_uri

Statistics

  • keepalive_endpoints – total number of NAT endpoints that are being kept alive.
  • registered_endpoints – NAT endpoints kept alive for registrations
  • subscribed_endpoints – NAT endpoints kept alive for subscriptions.
  • dialog_endpoints – Indicates how many of the NAT endpoints are kept alive for taking part in an INVITE dialog.

NATHelper Module

NAT traversal and reuse of TCP connections.
Helps symmetric UAs who are not able to determine their public address.

NAT pinging types

UDP packet
4 bytes (zero filled) UDP packets are sent to the contact address.
SIP request
a stateless SIP request is sent to the UDP contact address.
(+) low bandwitdh traffic,
(+) easy to generate by Kamailio;
(+) bidirectional traffic through NAT
(+) since each PING request from Kamailio (inbound traffic) will force the SIP client to generate a SIP reply (outbound traffic) – the NAT bind will be surely kept open.
(-) unidirectional traffic through NAT (inbound – from outside to inside);
if many NATs do update the bind timeout only on outbound traffic, the bind may expire and closed.
(-) higher bandwitdh traffic
(-) more expensive (as time) to generate by Kamailio;

Dependencies – usrloc

Params

  • force_socket – Socket to be used when sending NAT pings for UDP communication.
modparam("nathelper", "force_socket", "127.0.0.1:5060")
  • natping_interval
  • ping_nated_only
  • natping_processes – How many timer processes should be created by the module for the exclusive task of sending the NAT pings.
  • natping_socket
  • received_avp – AVP used to store the URI containing the received IP, port, and protocol by fix_nated_register
  • sipping_bflag
  • sipping_from
  • sipping_method
  • natping_disable_bflag
  • nortpproxy_str
  • keepalive_timeout
  • udpping_from_path
  • append_sdp_oldmediaip
  • filter_server_id

Functions

  • fix_nated_contact() – rewrites the “Contact” header field with request’s source address:port pair
  • fix_nated_sdp() – adds the active direction indication to SDP and updates ource ip address information too
  • add_rcv_param() – add a received parameter to the “Contact” header fields or the Contact URI.
  • fix_nated_register() exports the request’s source address:port into an AVP to be used during save()
  • nat_uac_test()- check if client’s request originated behind a nat
  • is_rfc1918()
  • add_contact_alias() – Adds an “;alias=ip~port~transport” parameter to the contact URI
  • handle_ruri_alias() – Checks if the Request URI has an “alias” parameter and if so, removes it and sets the “$du” based on its value.
  • set_contact_alias()

Pseudo Variables

  • $rr_count – Number of Record Routes in received SIP request or reply.
  • $rr_top_count – If topmost Record Route in received SIP request or reply is a double Record Route, value of $rr_top_count is 2.

RPC Commands

nathelper.enable_ping 

References :

Asterisk – installation and dial plans for WebRTC supported PJSIP clients

Asterisk is a framework or toolkit designed for VOIP systems . It can support Enterprise communication systems like PBXs, call distributors, VoIP gateways , conference bridges etc . It is open source and free to use . It is developed in C and runs in linux .

Technically , Asterisk has protocol support for many telephony technologies and protocols such as SIP , H323 . It can connect old PSTN or copper line and VOIP .

Asterisk is a framework for building multi-protocol, real-time communications applications and solutions. Asterisk is to realtime voice and video applications as what Apache is to web applications

– asterisk.org

Combine the SIP channel, the PSTN interface channel and some Dialplan script and you have a gateway.
Change the Dialplan to drop calls into a ConfBridge session and you have a conference server.
Alter it once more to route calls into voice mailboxes and you have a voicemail server.
Tie it all together and you have an amazingly powerful phone system.

– asterisk.org

A repo containing Asterisk configuration and common applications can be found at https://github.com/altanai/asteriskexamples

Asterisk as a Central signalling SIP application Server in VoIP Platform

Due to the wide array of call flow processing and media, channel, bridge management module support of asterisk, it is ideal to sit at the core of a VoIP of Communication platform solution. A WebRTC capable CPaaS overall architecture with inbound and outbound Kamailio proxy and central asterisk signaller and integration to telecom service providers over SIP trunks can be described as below

Quick Installation of Asterisk

goto /usr/src and download the preferred the version of asterisk code from
http://downloads.asterisk.org/pub/telephony/asterisk/

I am using the latest release candidate at the time writing this article asterisk-16.2.0-rc1-patch.tar.gz

Some external Dependencies apt-get install subversion

Then install the source dependencies

 sudo su
contrib/scripts/get_mp3_source.sh

This will install mp3 related programs such as


A addons/mp3
A addons/mp3/decode_ntom.c
A addons/mp3/interface.c
A addons/mp3/MPGLIB_README
A addons/mp3/common.c
A addons/mp3/huffman.h
A addons/mp3/tabinit.c
A addons/mp3/Makefile
A addons/mp3/README
A addons/mp3/decode_i386.c
A addons/mp3/dct64_i386.c
A addons/mp3/MPGLIB_TODO
A addons/mp3/mpg123.h
A addons/mp3/layer3.c
A addons/mp3/mpglib.h
Exported revision 202.

Actual dependencies will be installed via install_prereq script

contrib/scripts/install_prereq install

Output snippet

Run configure which will create scripts for next processes

 ./configure

Build third party scripts

make -j2

After build , to run the installation

make install

Asterisk PBX setup

make basic-pbx

The output should be

Installing basic-pbx config files…
Installing file configs/basic-pbx/README
Installing file configs/basic-pbx/asterisk.conf
Installing file configs/basic-pbx/cdr.conf
Installing file configs/basic-pbx/cdr_custom.conf
Installing file configs/basic-pbx/confbridge.conf
Installing file configs/basic-pbx/extensions.conf
Installing file configs/basic-pbx/indications.conf
Installing file configs/basic-pbx/logger.conf
Installing file configs/basic-pbx/modules.conf
Installing file configs/basic-pbx/musiconhold.conf
Installing file configs/basic-pbx/pjsip.conf
Installing file configs/basic-pbx/voicemail.conf
Updating asterisk.conf

Also run make config to make pbx configs

make config 

start asterisk

systemctl start asterisk

connect to asterisk tool for cli

asterisk -vvvr
Asterisk 16.2.0-rc1, Copyright (C) 1999 - 2018, Digium, Inc. and others. Created by Mark Spencer markster@digium.com
Asterisk comes with ABSOLUTELY NO WARRANTY; type 'core show warranty' for details.
This is free software, with components licensed under the GNU General Public License version 2 and other licenses; you are welcome to redistribute it under certain conditions. Type 'core show license' for details.

Connected to Asterisk 16.2.0-rc1 currently running on ip-172-31-45-26 (pid = 13388)
ip-172-31-45-26*CLI> 

Register sip phones with asterisk PBX and make / receive calls

To make calls among users, we need to configure channel driver with sip support . Using the sip protcol the phones within the enterprise will be able to send call signals out to one another. Open pjsip.conf

Take any endpoint from template , such as

;================================
;Laverne Roberts
;Software Engineer
[1113 (endpoint-internal-d70)
auth = 1113
aors = 1113
callerid = Laverne Roberts <1113>
[1113 (auth-userpass)
password = xxxxxxxx
username = xxxxxxxx
[1113 (aor-single-reg)
mailboxes = 1113@example

and set the values in sip softphone like zoiper , register with provided creds

registering sip phone zoiper with newly created asterisk PBX

If the registration creds used are not matching with the ones defines in pjsip.conf then REGISTER request failed message is displayed

Request 'REGISTER' from '' failed for 'x.x.x.x:18475' (callid: hp8iN6oWLRdER4zvEBdiUg..) - No matching endpoint found

On correct creds used the server prints traces such as

-- Added contact 'sip:1113@x.x.x.x:44312;transport=UDP;rinstance=b8aceff08623b51e' to AOR '1113' with expiration of 60 seconds
== Endpoint 1113 is now Reachable
-- Removed contact 'sip:1113@x.x.x.x:44312;transport=UDP;rinstance=b8aceff08623b51e' from AOR '1113' due to request
== Contact 1113/sip:1113@x.x.x.x:44312;transport=UDP;rinstance=b8aceff08623b51e has been deleted
== Endpoint 1113 is now Unreachable
-- Added contact 'sip:1113@x.x.x.x:18475;transport=UDP;rinstance=5af431512ae0af3a' to AOR '1113' with expiration of 60 seconds
== Endpoint 1113 is now Reachable

Alternatively one can also create new sip endpoints

Dialplan Applications

Playback Hello World

[internal_users]
exten => 6000,1,Answer()
exten => 6000,2,Verbose("---------------- Tring Tring -------")
exten => 6000,3,Wait(1)
exten => 6000,n,Playback(silence/1&hello-world)
exten => 6000,n,Hangup()

NoOP and printing channel variables

exten=>6124,1,Verbose(2,The channel name is ${CHANNEL})
same => n,Verbose(2,The unique id is ${UNIQUEID})
same => n,Verbose(2,The caller id is ${CALLERID(all)})
same => n,Verbose(2,The datetime is ${DATETIME})
same => n,Verbose(2,The timestamp is ${TIMESTAMP})
same => n,Verbose(2,The context is ${CONTEXT})
same => n,Verbose(2,The SYSTEMNAME is ${SYSTEMNAME})
same => n,Verbose(2,The PRIORITY is ${PRIORITY})
same => n,Verbose(2,The CHANNEL is ${CHANNEL})

settings varaibles and Say

exten=>6009,1,Verbose("------------ Set variable -------")
same => n,Set(COUNT=3)
same => n,SayNumber(${COUNT})
same => n,Set(${COUNT}=10)
same => n,SayNumber(${COUNT}) 

Simple Voicemail ( also need configuration on voicemail.conf)

exten => 1235,1,VoiceMail(1235,u)               

PBX and applications

PBX cores settings 
Version:                     16.2.0-rc1
   Build Options:               BUILD_NATIVE, OPTIONAL_API
   Maximum calls:               Not set
   Maximum open file handles:   1024
   Root console verbosity:      5
   Current console verbosity:   5
   Debug level:                 0
   Maximum load average:        0.000000
   Minimum free memory:         0 MB
   Startup time:                10:27:35
   Last reload time:            10:27:35
   System:                      Linux/4.15.0-1021-aws built by root on x86_64 2019-02-11 11:48:29 UTC
   System name:                 
   Entity ID:                   0e:28:c0:44:39:5e
   PBX UUID:                    a2df96bb-6d1a-4f64-a953-cf02030e9851
   Default language:            en
   Language prefix:             Enabled
   User name and group:         /
   Executable includes:         Disabled
   Transcode via SLIN:          Enabled
   Transmit silence during rec: Disabled
   Generic PLC:                 Disabled
   Generic PLC on equal codecs: Disabled
   Min DTMF duration::          80
   Cache media frames:          Enabled
   RTP use dynamic payloads:    1
   RTP dynamic payload types:   35-63,96-127

Subsystems
   Manager (AMI):               Disabled
   Web Manager (AMI/HTTP):      Disabled
   Call data records:           Enabled
   Realtime Architecture (ARA): Disabled

 Directories
   Configuration file:          /etc/asterisk/asterisk.conf
   Configuration directory:     /etc/asterisk
   Module directory:            /usr/lib/asterisk/modules
   Spool directory:             /var/spool/asterisk
   Log directory:               /var/log/asterisk
   Run/Sockets directory:       /var/run/asterisk
   PID file:                    /var/run/asterisk/asterisk.pid
   VarLib directory:            /var/lib/asterisk
   Data directory:              /var/lib/asterisk
   ASTDB:                       /var/lib/asterisk/astdb
   IAX2 Keys directory:         /var/lib/asterisk/keys
   AGI Scripts directory:       /var/lib/asterisk/agi-bin

Asterisk Applications

Some of the application are listed below . Note that this is not an extensive list as more application are added and old ones are removed every minor release .

AddQueueMember: Dynamically adds queue members.
AlarmReceiver: Provide support for receiving alarm reports from a burglar or fire alarm panel.
AMD: Attempt to detect answering machines.
Answer: Answer a channel if ringing.
AttendedTransfer: Attended transfer to the extension provided and TRANSFER_CONTEXT
BackGround: Play an audio file while waiting for digits of an extension to go to.
BackgroundDetect: Background a file with talk detect.
BlindTransfer: Blind transfer channel(s) to the extension and context provided
Bridge: Bridge two channels.
BridgeAdd: Join a bridge that contains the specified channel.
BridgeWait: Put a call into the holding bridge.
Busy: Indicate the Busy condition.
ChanSpy: Listen to a channel, and optionally whisper into it.
ConfBridge: Conference bridge application.
Dial: Attempt to connect to another device or endpoint and bridge the call.
Dictate: Virtual Dictation Machine.
Echo: Echo media, DTMF back to the calling party
ExtenSpy: Listen to a channel, and optionally whisper into it.
ExternalIVR: Interfaces with an external IVR application.
FollowMe: Find-Me/Follow-Me application.
ForkCDR: Forks the current Call Data Record for this channel.
Hangup: Hang up the calling channel.
Log: Send arbitrary text to a selected log level.
MailboxExists: Check to see if Voicemail mailbox exists.
Milliwatt: Generate a Constant 1004Hz tone at 0dbm (mu-law).
MixMonitor: Record a call and mix the audio during the recording. Use of StopMixMonitor is required to guarantee the audio file is available for processing during dialplan execution.
Monitor: Monitor a channel.
Morsecode: Plays morse code.
MusicOnHold: Play Music On Hold indefinitely.
NoCDR: Tell Asterisk to not maintain a CDR for this channel.
NoOp: Do Nothing (No Operation).
Originate: Originate a call.
Park: Park yourself.
PauseMonitor: Pause monitoring of a channel.
PauseQueueMember: Pauses a queue member.
Pickup: Directed extension call pickup.
Playback: Play a file.
PlayTones: Play a tone list.
PrivacyManager: Require phone number to be entered, if no CallerID sent
Proceeding: Indicate proceeding.
Progress: Indicate progress.
Queue: Queue a call for a call queue.
Read: Read a variable.
SendImage: Sends an image file.
SendText: Send a Text Message on a channel.
SendURL: Send a URL.
Set: Set channel variable or function value.
SetAMAFlags: Set the AMA Flags.
SMS: Communicates with SMS service centres and SMS capable analogue phones.
SoftHangup: Hangs up the requested channel.
SpeechActivateGrammar: Activate a grammar.
SpeechBackground: Play a sound file and wait for speech to be recognized.
SpeechCreate: Create a Speech Structure.
SpeechDeactivateGrammar: Deactivate a grammar.
SpeechDestroy: End speech recognition.
SpeechLoadGrammar: Load a grammar.
SpeechProcessingSound: Change background processing sound.
SpeechStart: Start recognizing voice in the audio stream.
SpeechUnloadGrammar: Unload a grammar.
StackPop: Remove one address from gosub stack.
StartMusicOnHold: Play Music On Hold.
Stasis: Invoke an external Stasis application.
StopMixMonitor: Stop recording a call through MixMonitor, and free the recording’s file handle.
StopMonitor: Stop monitoring a channel.
StopMusicOnHold: Stop playing Music On Hold.
StopPlayTones: Stop playing a tone list.
StreamEcho: Echo media, up to ‘N’ streams of a type, and DTMF back to the calling party
VMSayName: Play the name of a voicemail user
VoiceMail: Leave a Voicemail message.
Record: Record to a file.
RemoveQueueMember: Dynamically removes queue members.
ResetCDR: Resets the Call Data Record.
RetryDial: Place a call, retrying on failure allowing an optional exit extension.
Return: Return from gosub routine.
Ringing: Indicate ringing tone.
SayNumber: Say Number.
SMS: Communicates with SMS service centres and SMS capable analogue phones.
SoftHangup: Hangs up the requested channel.
SpeechActivateGrammar: Activate a grammar.
SpeechBackground: Play a sound file and wait for speech to be recognized.
SpeechCreate: Create a Speech Structure.
SpeechDeactivateGrammar: Deactivate a grammar.
SpeechDestroy: End speech recognition.
SpeechLoadGrammar: Load a grammar.
SpeechProcessingSound: Change background processing sound.
SpeechStart: Start recognizing voice in the audio stream.
StartMusicOnHold: Play Music On Hold.
Stasis: Invoke an external Stasis application.
StopMixMonitor: Stop recording a call through MixMonitor, and free the recording’s file handle.
StopMonitor: Stop monitoring a channel.
StopMusicOnHold: Stop playing Music On Hold.
StopPlayTones: Stop playing a tone list.
StreamEcho: Echo media, up to ‘N’ streams of a type, and DTMF back to the calling party
System: Execute a system command.
VoiceMail: Leave a Voicemail message.
VoiceMailMain: Check Voicemail messages.
VoiceMailPlayMsg: Play a single voice mail msg from a mailbox by msg id.
Wait: Waits for some time.
Zapateller: Block telemarketers with SIT.

Webrtc support for asterisk over res_pjsip

Refernce project https://github.com/altanai/asteriskexamples/tree/master/webrtc_pjsip

To connect video based webrtc endpoints ensure you load the codecs and also libsrtp . Overwrite the selective conf in this folders with the existing conf of asterisk to run a basic webrtc video call . These were tested with jssip on asterisk v17 with res_pjsip.

confirm using pjsip – since chan_sip is depriciate. Confirm that yo are not using chan_sip and instead ensure using re_pjsip

module unload chan_sip
module show like res_pjsip

If pjssip is not found load it wither form menuselect or by using cli load module command .

pjssip show endpoints

*CLI> pjsip show endpoints

 Endpoint:  <Endpoint/CID.....................................>  <State.....>  <Channels.>
    I/OAuth:  <AuthId/UserName...........................................................>
        Aor:  <Aor............................................>  <MaxContact>
      Contact:  <Aor/ContactUri..........................> <Hash....> <Status> <RTT(ms)..>
  Transport:  <TransportId........>  <Type>  <cos>  <tos>  <BindAddress..................>
   Identify:  <Identify/Endpoint.........................................................>
        Match:  <criteria.........................>
==========================================================================================

 Endpoint:  7000                                                 Invalid       0 of inf
     InAuth:  7000/7000
        Aor:  7000                                               1

 Endpoint:  8000                                                 Invalid       0 of inf
     InAuth:  8000/8000
        Aor:  8000                                               1

Generating self-signed certs

use the “ast_tls_cert” script in the “contrib/scripts” Asterisk source directory to make a self-signed certificate authority and an Asterisk certificate.

sh ast_tls_cert.sh -C localhost -O "altanai" -d .

after creating the self signed keys start the server

asterisk -vvvvvvc

channels

Peer             User/ANR         Call ID          Format           Hold     Last Message    Expiry     Peer      
10.10.10.10     1060             e8ae107f-ce90-2  (ulaw)           No       Rx: ACK                    1060 

Codecs – check for the webrtc supported audio and video codesc in the list , if not found install the modules and reload.

  31 image png          png              (PNG Image)
   6 audio g726         g726             (G.726 RFC3551)
   4 audio alaw         alaw             (G.711 a-law)
   2 audio g723         g723             (G.723.1)
  20 audio speex        speex            (SpeeX)
  21 audio speex        speex16          (SpeeX 16khz)
  22 audio speex        speex32          (SpeeX 32khz)
  24 audio g722         g722             (G722)
  25 audio siren7       siren7           (ITU G.722.1 (Siren7, licensed from Polycom))
  32 video h261         h261             (H.261 video)
  33 video h263         h263             (H.263 video)
   8 audio adpcm        adpcm            (Dialogic ADPCM)
  36 video h265         h265             (H.265 video)
  44 audio silk         silk8            (SILK Codec (8 KHz))
  45 audio silk         silk12           (SILK Codec (12 KHz))
  46 audio silk         silk16           (SILK Codec (16 KHz))
  47 audio silk         silk24           (SILK Codec (24 KHz))
  28 audio g719         g719             (ITU G.719)
  34 video h263p        h263p            (H.263+ video)
  35 video h264         h264             (H.264 video)
  19 audio g729         g729             (G.729A)
   9 audio slin         slin             (16 bit Signed Linear PCM)
  10 audio slin         slin12           (16 bit Signed Linear PCM (12kHz))
  11 audio slin         slin16           (16 bit Signed Linear PCM (16kHz))
  12 audio slin         slin24           (16 bit Signed Linear PCM (24kHz))
  13 audio slin         slin32           (16 bit Signed Linear PCM (32kHz))
  14 audio slin         slin44           (16 bit Signed Linear PCM (44kHz))
  15 audio slin         slin48           (16 bit Signed Linear PCM (48kHz))
  16 audio slin         slin96           (16 bit Signed Linear PCM (96kHz))
  17 audio slin         slin192          (16 bit Signed Linear PCM (192kHz))
   3 audio ulaw         ulaw             (G.711 u-law)
  18 audio lpc10        lpc10            (LPC10)
  27 audio testlaw      testlaw          (G.711 test-law)
  43 audio none         none             (<Null> codec)
  42 image t38          t38              (T.38 UDPTL Fax)
  39 video vp9          vp9              (VP9 video)
  38 video vp8          vp8              (VP8 video)
   5 audio gsm          gsm              (GSM)
  37 video mpeg4        mpeg4            (MPEG4 video)
  23 audio ilbc         ilbc             (iLBC)
  40 text  red          red              (T.140 Realtime Text with redundancy)
  41 text  t140         t140             (Passthrough T.140 Realtime Text)
  29 audio opus         opus             (Opus Codec)
  30 image jpeg         jpeg             (JPEG image)
   7 audio g726aal2     g726aal2         (G.726 AAL2)
   1 audio codec2       codec2           (Codec 2)
  26 audio siren14      siren14          (ITU G.722.1 Annex C, (Siren14, licensed from Polycom))

Webrtc clients

Here is a succesful run using PJSIP as the webrtc client to communicated to another pjsip client via Asterisk server .

Note that all signalling and media isi getting proxied via the Asterisk server signalling and media plane which is in contrast to the peer to peer nature of WebRTC

Register with SIP Registrar on Asterisk
Specify the contact URI as the Asterisk server too
Incoming Call
WebRTC Call in progress via asterisk SIP and Media Server
Signalling and Media Proxy Via Asterisk

Debugging

set debug all

pjsip set logger on
rtp set debug on

setting debugger on a peer

SIP SET DEBUG PEER 1061

Chrome WebRTC Internals

References :


TURN server for WebRTC – RFC5766-TURN , Coturn, Xirsys , Twillio

STUN (Session Traversal Utilities for NAT) and TURN (Traversal Using Relays around NAT) are protocols that can be used to provide NAT traversal for VoIP and WebRTC. These projects provide a VoIP media traffic NAT traversal server and gateway.

TURN Server is a VoIP media traffic NAT traversal server and gateway.

This article describes working with some of the TURN servers 


We know that a STUN only server such as the one below , may not work under firewalls like in enterperises or even in universities resulting in black video , one way video , inconcsistent streaming or even no video experience .

// google STUN 
var iceservers_array=[{urls: 'stun:stun.l.google.com:19302'}] ;

To overcome this we rely on a publicaly avaiable TURN server which can step in and do ICE exchange to seup relay routes for the media stream. Some of the options for both self hosted and TURN as a service are described below .

rfc5766-turn-server

It is a legacy project mainly archives for reference( links at the end of section). This is a VoIP gateway for inter network communication which is opensource  MIT licensed .

platforms supported : Any client platform is supported, including Android, iOS, Linux, OS X, Windows, and Windows Phone. This project can be successfully used on other *NIX platforms ( Aamazon EC2) too. It supports flat file or Database based user management system ( MySQL , postgress , redis ). The source code project contains ,  TURN server ,  TURN client messaging library and some sample scripts to test various modules like protocol , relay , security etc .

Protocols : Protocols between the TURN client and the TURN server – UDP, TCP, TLS, and DTLS. Relay protocol – UDP , TCP .

Authentication : The authentication mechanism is using key which is calculated over the user name, the realm, and the user password. Key for the HMAC depends on whether long-term or short-term credentials are in use. For long-term credentials, the key is 16 bytes: key = MD5(username “:” realm “:” SASLprep(password))

Installation :  Since I used my Ubuntu Software center for installing the RFC turn server 5677 .

Screenshot from 2015-03-05 15:22:30

More information is on Ubuntu Manuals : http://manpages.ubuntu.com/manpages/trusty/man1/turnserver.1.html

The content got stored inside /usr/share/rfc5766-turn-server. Also install mysql for record keeping

sudo apt-get install mysql-server
mysql
mysql2
mysql4

Intall MySQL workbench to monitor the values feed into the turn database server in MysqL. connect to MySQL instance using the following screenshot

mysql5

The database formed with mysql after successful operation is as follows . We  shall notice that the initial db is absolutely null

mysql8empty

Terminal Commands

These terminal command ( binary images ) get stored inside etc/init.d after installing

turnadmin – Its turn relay administration tool used for generating , updating keys and passwords . For generating a key to get long term crdentaial use -k command and for aading or updateing a long -term user use the -a command. Therefore a simple command to generate a key is

format : turnadmin -k -u -r -p

 turnadmin -k -u turnwebrtc -r mycompany.com -p turnwebrtc

The generated key is displayed in console . For example the following screenshot shows this :

rfc5677turnkey

To fill in user with long term credentails

Format : turnadmin -a [-b | -e | -M | -N ] -u -r -p

turnadmin -a -M "host=localhost dbname=turn user=turn password=turn" -u altanai -r mycompany.com -p 123456

Check the values reflected in MySQL workbench for long term user table . ( screenshot depicts two entries for altanai and turnwebrtc user )

turnkeylongterm

you can also check it on console using the -l command

format :turnadmin -l –mysql-userdb=””

turnadmin -l --mysql-userdb="host=127.0.0.1 dbname=turn user=turnwebrtc password=turnwebrtc connect_timeout=30"
longtermuserlcommand

or we can also check using the terminal based mySQL client

mysql> use turn;
Database changed
mysql> select * from turnusers_lt;
+------------+----------------------------------+
| name | hmackey |
+------------+----------------------------------+
| altanai | 57bdc681481c4f7626bffcde292c85e7 |
| turnwebrtc | 6066cbe0b5ee14439b2ddfc177268309 |
+------------+----------------------------------+
2 rows in set (0.00 sec)

turnserver – Its command to handle the turnserver itself . We can use the simple turnserver command to start it without any db support using just turnserver. Screenshot for this is

turnserverstart

We can use a database like mysql to start it with db connection string

Format : turnserver –mysql-userdb=””

turnserver --mysql-userdb="host=127.0.0.1 dbname=turn user=turnwebrtc password=turnwebrtc connect_timeout=30"

turnservermysqldb

turnutils_uclient: emulates multiple UDP,TCP,TLS or DTLS clients.

turnutils_peer: simple stateless UDP-only “echo” server. For every incoming UDP packet, it simply echoes it back.

turnutils_stunclient: simple STUN client example that implements RFC 5389 ( using STUN as endpoint to determine the IP address and port allocated to it , keep-alive , check connectivity etc) and RFC 5780 (experimental NAT Behavior Discovery STUN usage) .

turnutils_rfc5769check: checks the correctness of the STUN/TURN protocol implementation. This program will perform several checks and print the result on the screen. It will exit with 0 status if everything is OK, and with (-1) if there was an error in the protocol implementation.

Test

1. Test vectors from RFC 5769 to double-check that our STUN/TURN message encoding algorithms work properly. Run the utility to check all protocols :

$ cd examples
$ ./scripts/rfc5769.sh

2. TURN functionality test (bare minimum TURN example).

If everything compiled properly, then the following programs must run together successfully, simulating TURN network routing in local loopback networking environment:

console 1 :

$ ./scripts/basic/relay.sh

console2 :

$ ./scripts/peer.sh

If the client application produces output and in approximately 22 seconds prints the jitter, loss and round-trip-delay statistics, then everything is fine.

Usage

iceServers:[
 { 'url': 'stun: altanai@mycompany.com'},
 { 'url': 'turn: altanai@mycompany.com', 'credential': '123456'}]

Insert the above piece of code on peer connection config . Now call from one network environment to another . For example call from a enterprise network behind a Wifi router to a public internet datacard webrtc agent . The call should connect with video flowing smoothly between the two .

tooltips
TURN working accross Enterprise firrewall on a WebRTC video platform written with SimpeWebRTC lbrary . Project tango FX

website : https://code.google.com/p/rfc5766-turn-server/

Download the executable from : http://turnserver.open-sys.org/downloads/v3.2.5.4/

coturn

Project Coturn evolved from rfc5766-turn-server project with many new advanced TURN specs beyond the original RFC 5766 document. The databses supported are : SQLite , MySQL , PostgreSQL , Redis , MongoDB

Protocols :

The implementation fully supports the following client-to-TURN-server protocols: UDP  , TCP  , TLS  SSL3/TLS1.0/TLS1.1/TLS1.2; ECDHE , DTLS versions 1.0 and 1.2. Supported relay protocols UDP (per RFC 5766) and TCP (per RFC 6062)

Authetication :

Supported message integrity digest algorithms:

  • HMAC-SHA1, with MD5-hashed keys (as required by STUN and TURN standards)
  • HMAC-SHA256, with SHA256-hashed keys (an extension to the STUN and TURN specs)

Supported TURN authentication mechanisms:

Installation

Install libopenssl and libevent plus its dev or extra libraries .
OpenSSL has to be installed before libevent2 for TLS beacuse When libevent builds it checks whether OpenSSL has been already installed, and its version.

Download coturn readonly  from

svn checkout http://coturn.googlecode.com/svn/trunk/ coturn-read-only

extract the tar contents
$ tar xvfz turnserver-.tar.gz

go inside the extracted folder and run the following command to build
$ ./configure
$ make
$ make install

Adding users in the format using turnadmin
$ Sudo turnadmin -a -u -r -p

$ Sudo turnadmin -a -u altanai -r myserver.com -p 123456

Start the turn Server using turnserver from inside of /etc/init.d using the start command

$ sudo /etc/init.d/coturn start
Screenshot from 2015-01-06 12-08-15

The logs are usually stored in /var/log . Screenshot of log file

tuenlog2

The default configured port is 3478.If other port is needed, change the file /etc/turnserver.conf

Usuage:

Specify the  values in Peer Connection

iceServers:[
{ ‘url’: ‘stun: altanai@myserver.com’},
{ ‘url’: ‘turn: altanai@myserver.com’, ‘credential’: ‘123456’}]

Specifications:

TURN specs:

STUN specs:

  • RFC 3489 – STUN – Simple Traversal of User Datagram Protocol (UDP) Through Network Address Translators (NATs)
  • RFC 5389 – Session Traversal Utilities for NAT (STUN)
  • RFC 5769 – test vectors for STUN protocol testing
  • RFC 5780 – NAT behavior discovery support
  • RFC 7443 – Application-Layer Protocol Negotiation (ALPN) Labels for STUN and TURN

ICE :

  • RFC 5245 – ICE
  • RFC 5768 – ICE–SIP
  • RFC 6336 – ICE–IANA Registry
  • RFC 6544 – ICE–TCP
  • RFC 5928 – TURN Resolution Mechanism

website : https://code.google.com/p/coturn/

Chorme WebRTC Internals Page for COTURN connection

Xirsys

Xirsys is a provider for WebRTC infrastructure which included stun and turn server hosting as well .

The process of using their services includes singing up for a account and choosing whether you want a paid service capable of handling more calls simultaneously or free one handling only upto 10 concurrent turn connections .

The dashboard appears like this :

xirsys1

To receive the api one need to make a one time call to their service , the result of which contains the keys to invoke the turn services from webrtc script .

window.addEventListener("load", function (e) {
    let xhr = new XMLHttpRequest();
    xhr.onreadystatechange = function ($evt) {
        if (xhr.readyState == 4 && xhr.status == 200) {
            let res = JSON.parse(xhr.responseText);
            console.log("response: ", res);
            iceservers_array.push(res.v.iceServers);
           alert( iceservers_array);
        }
    };
    xhr.open("PUT", "https://global.xirsys.net/_turn/webrtc", true);
    xhr.setRequestHeader("Authorization","Basic " + btoa("altanai:<sec rettoken>"));
    xhr.setRequestHeader("Content-Type","application/json");
    xhr.send(JSON.stringify({"format": "urls"}));
});

The resulting output should look like ( my keys are hidden with a red rectangle ofcourse )

xirsysedited

The process of adding a TURN / STUN to your webrtc script in JS is as follows :

iceServers:[
 {"url":"stun:turn2.xirsys.com"},
 {"username":"< put your API username>","url":"turn:turn2.xirsys.com:443?transport=udp","credential":"< put your API credentail>"},
 {"username":"< put your API username>","url":"turn:turn2.xirsys.com:443?transport=tcp","credential":"< put your API credentail>"}]

website : http://xirsys.com/technology/

Twillio TURN Server

// before unload update status on main site
window.addEventListener("load", function (e) {
    let xhr = new XMLHttpRequest();
    xhr.onreadystatechange = function ($evt) {
        if (xhr.readyState == 4 && xhr.status == 200) {
            let res = JSON.parse(xhr.responseText);
            console.log("response: ", res);
            iceservers_array = res.iceServers;
            console.log("iceservers_array: ", iceservers_array);
            CallSessionBegins();
        }
    };
    xhr.open("POST", "https://<ourdomain>:3000/token", true);
    xhr.setRequestHeader("Content-Type","application/json");
    xhr.send(JSON.stringify({"format": "urls"}));
});

Asterisk TURN module and service

Asterisk is an open source carrer grade SIP server which also provides Firewall traversal . A github repo containing some asterisk dialplan examples is https://github.com/altanai/asteriskexamples. An article discussing Asterisk and its implementation

Asterisk – installation and dial plans for WebRTC supported PJSIP clients

Asterisk is a framework or toolkit designed for VOIP systems . It can support Enterprise communication systems like PBXs, call distributors, VoIP gateways , conference bridges etc . It is open source and free to use . It is developed in C and runs in linux . Technically , Asterisk has protocol support for many…

Kamailio NAT and STUN modules in Server

It is also interesting to note that majority of popular open source SIP servers also have an implementation , libabary or module for STUN/TURN such as Kamailio’s STUN module https://www.kamailio.org/docs/modules/devel/modules/stun.html

You can read more about the kamailio DNS and NATing here

Kamailio DNS and NAT

DNS sub-system in Kamailio DNS failover DNS load balancing NAT ( Network Address Translation) NAT ( Network Address Translation) Why is NAT is important in SIP? Types of NAT solutions NAT behaviours RTP NAT Fixing NAT NAT Traversal Module Why use keepalive when Registrations are already there for NATing ? How keepalives work for NATing…


NAT traversal using STUN and TURN


WebRTC : Web-based real-time communications is a gamechanger for real-time communication systems. WebRTC is one such open-source, royalty-free, unencumbered browser-based platform using the browser’s embedded media application programming interface (API). It allows developers to add custom JavaScript & HTML5 to control the media setup and flow. WebRTC has enabled developers to build apps, sites, widgets, plugins and extensions capable of delivering simultaneous audio, video, data, and screen-sharing capability in a peer to peer fashion.

Issues accross Networks : But something which escapes our attention is how media is traversing across the network. Of course, the webrtc sessions run smoothly when both the peers are on the open public internet without any restrictions or firewall blocks. But the real problem begins when one of the peers is behind a Corporate/Enterprise network or using a different Internet service provider with some security restrictions. In such a case the normal ICE capability of WebRTC is not sufficient to set up a bidirectional media streaming setup. For network restriction what is required is a NAT ( Network Address Traversal) mechanism that performs address discovery.

NAT and ICE Solution : STUN and TURN server protocols handle session initiations with handshakes between peers in different network environments. In the case of a firewall blocking a STUN peer-to-peer connection, the system fallback to a TURN server which provides the necessary traversing mechanism through the NAT.

Lets study from the start ie ICE.

NAT

Network Address Translation provides a mapping of internal to external IP addresses. This helps in network address modification for packets which in transit accross a tarfic routinig node such as inter networks.

A private address on the inside of the NAT is mapped to an external public address. Port address translation (PAT) resolves conflicts that arise when multiple hosts happen to use the same source port number to establish different external connections at the same time.

Some ways to acheive this

  • Application Layer Gateway (ALG) 
  • Interactive Connectivity Establishment ( ICE )
  •  UPnP Internet Gateway Device Protocol
  • propertiary SIP based Session Border Controller, so on

Lets us just look at ICE in detail which is the default implementation for WebRTC

What is ICE and why is it used ?

ICE (Interactive Connectivity Establishment )  framework ( mandatory by WebRTC standards  ) find network interfaces and ports in Offer / Answer Model to exchange network based information with participating communication clients. ICE makes use of the Session Traversal Utilities for NAT (STUN) protocol and its extension, Traversal Using Relay NAT (TURN)

ICE is defined by RFC 5245 – Interactive Connectivity Establishment (ICE): A Protocol for Network Address Translator (NAT) Traversal for Offer/Answer Protocols.

Sample WebRTC offer holding ICE candidates :

type: offer, sdp: v=0
o=- 3475901263113717000 2 IN IP4 127.0.0.1
s=-
t=0 0
a=group:BUNDLE audio video data
a=msid-semantic: WMS dZdZMFQRNtY3unof7lTZBInzcRRylLakxtvc
m=audio 9 RTP/SAVPF 111 103 104 9 0 8 106 105 13 126
c=IN IP4 0.0.0.0
a=rtcp:9 IN IP4 0.0.0.0
a=ice-ufrag:/v5dQj/qdvKXthQ2
a=ice-pwd:CvSEjVc1z6cMnhjrLlcbIxWK
a=ice-options:google-ice
a=fingerprint:sha-256 F1:A8:2E:71:4B:4E:FF:08:0F:18:13:1C:86:7B:FE:BA:BD:67:CF:B1:7F:19:87:33:6E:10:5C:17:42:0A:6C:15
a=setup:actpass
a=mid:audio
a=sendrecv
a=rtcp-mux
a=rtpmap:111 opus/48000/2
a=fmtp:111 minptime=10
a=rtpmap:103 ISAC/16000
a=rtpmap:104 ISAC/32000
a=rtpmap:9 G722/8000
a=rtpmap:0 PCMU/8000
a=rtpmap:8 PCMA/8000
a=rtpmap:106 CN/32000
a=rtpmap:105 CN/16000
a=rtpmap:13 CN/8000
a=rtpmap:126 telephone-event/8000
a=maxptime:60
m=video 9 RTP/SAVPF 100 116 117 96
c=IN IP4 0.0.0.0
a=rtcp:9 IN IP4 0.0.0.0
a=ice-ufrag:/v5dQj/qdvKXthQ2
a=ice-pwd:CvSEjVc1z6cMnhjrLlcbIxWK
a=ice-options:google-ice
a=fingerprint:sha-256 F1:A8:2E:71:4B:4E:FF:08:0F:18:13:1C:86:7B:FE:BA:BD:67:CF:B1:7F:19:87:33:6E:10:5C:17:42:0A:6C:15
a=setup:actpass
a=mid:video
a=sendrecv
a=rtcp-mux
a=rtpmap:100 VP8/90000
a=rtcp-fb:100 ccm fir
a=rtcp-fb:100 nack
a=rtcp-fb:100 nack pli
a=rtcp-fb:100 goog-remb
a=rtpmap:116 red/90000
a=rtpmap:117 ulpfec/90000
a=rtpmap:96 rtx/90000
a=fmtp:96 apt=100
m=application 9 DTLS/SCTP 5000
c=IN IP4 0.0.0.0
a=ice-ufrag:/v5dQj/qdvKXthQ2
a=ice-pwd:CvSEjVc1z6cMnhjrLlcbIxWK
a=ice-options:google-ice
a=fingerprint:sha-256 F1:A8:2E:71:4B:4E:FF:08:0F:18:13:1C:86:7B:FE:BA:BD:67:CF:B1:7F:19:87:33:6E:10:5C:17:42:0A:6C:15
a=setup:actpass
a=mid:data
a=sctpmap:5000 webrtc-datachannel 1024

Notice the ICE candidates under video and audio. Now take a look at the SDP answer

type: answer, sdp: v=0
o=- 6931590438150302967 2 IN IP4 127.0.0.1
s=-
t=0 0
a=group:BUNDLE audio video data
a=msid-semantic: WMS R98sfBPNQwC20y9HsDBt4to1hTFeP6S0UnsX
m=audio 1 RTP/SAVPF 111 103 104 0 8 106 105 13 126
c=IN IP4 0.0.0.0
a=rtcp:1 IN IP4 0.0.0.0
a=ice-ufrag:WM/FjMA1ClvNb8xm
a=ice-pwd:8yy1+7x0PoHZCSX2aOVZs2Oq
a=fingerprint:sha-256 7B:9A:A7:43:EC:17:BD:9B:49:E4:23:92:8E:48:E4:8C:9A:BE:85:D4:1D:D7:8B:0E:60:C2:AE:67:77:1D:62:70
a=setup:active
a=mid:audio
a=sendrecv
a=rtcp-mux
a=rtpmap:111 opus/48000/2
a=fmtp:111 minptime=10
a=rtpmap:103 ISAC/16000
a=rtpmap:104 ISAC/32000
a=rtpmap:0 PCMU/8000
a=rtpmap:8 PCMA/8000
a=rtpmap:106 CN/32000
a=rtpmap:105 CN/16000
a=rtpmap:13 CN/8000
a=rtpmap:126 telephone-event/8000
a=maxptime:60
m=video 1 RTP/SAVPF 100 116 117 96
c=IN IP4 0.0.0.0
a=rtcp:1 IN IP4 0.0.0.0
a=ice-ufrag:WM/FjMA1ClvNb8xm
a=ice-pwd:8yy1+7x0PoHZCSX2aOVZs2Oq
a=fingerprint:sha-256 7B:9A:A7:43:EC:17:BD:9B:49:E4:23:92:8E:48:E4:8C:9A:BE:85:D4:1D:D7:8B:0E:60:C2:AE:67:77:1D:62:70
a=setup:active
a=mid:video
a=sendrecv
a=rtcp-mux
a=rtpmap:100 VP8/90000
a=rtcp-fb:100 ccm fir
a=rtcp-fb:100 nack
a=rtcp-fb:100 nack pli
a=rtcp-fb:100 goog-remb
a=rtpmap:116 red/90000
a=rtpmap:117 ulpfec/90000
a=rtpmap:96 rtx/90000
a=fmtp:96 apt=100
m=application 1 DTLS/SCTP 5000
c=IN IP4 0.0.0.0
b=AS:30
a=ice-ufrag:WM/FjMA1ClvNb8xm
a=ice-pwd:8yy1+7x0PoHZCSX2aOVZs2Oq
a=fingerprint:sha-256 7B:9A:A7:43:EC:17:BD:9B:49:E4:23:92:8E:48:E4:8C:9A:BE:85:D4:1D:D7:8B:0E:60:C2:AE:67:77:1D:62:70
a=setup:active
a=mid:data
a=sctpmap:5000 webrtc-datachannel 1024
STUNTURN
address discovery for global IP:portallocates its own address as interface to the client
binary protocolextension of STUN
doesnt stay in path after connectionstays in path after connection.
tunnels and relays media
higher priority lower priority
server and peer reflexive ICE candidates relay ICE candidates
Failed WebRTC ICE Conection
succesfull STUN Connection
ICE candidate grid

Call Flow for STUN protocol exchange

Client -> Server : binding request with attributes – CHANGE-REQUEST

Server -> Cient : binding response with attributes – MAPPED-ADDRESS, RESPONSE-ORIGIN, OTHER-ADDRESS, XOR-MAPPED-ADDRESS

STUN call flow for WebRTC Offer Answer
STUN call flow for WebRTC Offer Answer
WebRTC STUN binding request
WebRTC STUN Binding success response

WebRTC needs SDP Offer to be sent to B from A.
Client B uses this SDP offer to generate an SDP Answer for A.
The SDP ( as seen on chrome://webrtc-internals/ ) includes ICE candidates which map open ports in the firewalls.

However, in case both sides are symmetric NATs, the media flow gets blocked. For such a case TURN is used which tries to give a public IP and port mapped to internal IP and port. This relay path provides an alternative routing mechanism like a packet mirror. It can open a DTLS connection and use it to key the SRTP-DTLS media streams.

NAT types

Some types of NAT are described below

Full Cone ( Normal)

All requests from the same internal IP address and port are mapped to the same external IP address and port. It also allows external hosts to send packet to internal host by using the mapped external address.

Full cone ( credits wikipedia)

Restricted Cone

All requests from the same internal IP address and port are mapped to the same external IP address and port, but external hosts can send packet to internal host only if  internal host had previously sent a packet to that IP address.

Address Restricted cone ( credits wikipedia)

Port Restricted Cone

All requests from the same internal IP address and port are mapped to the same external IP address and port, but external hosts can send packet to internal host only if  internal host had previously sent a packet to that IP address and that port.

Port Restricted cone ( credits wikipedia)

Symmetric

All requests from the same internal IP address and port, to a specific destination IP address and port, are mapped to the same external IP address and port. Any traffic from same internal IP+port to a different destination uses a new mapping. Also external hosts which receives a packet can send a UDP packet back to internal host.

Symmetric NAT ( credits wikipedia).
Address and Port-Dependent Mapping (“APDM”) and APDF (Address and Port-Dependent Filtering).
Web Archieve 2018
credit https://web.archive.org/web/20180829131023/http://nattest.net.in.tum.de/results.php

Network Scenarios for NAT

In order to Understand this better consider various scenarios that determine the NAT Mapping Behavior one could run tests using cli or network analyzer tools and checking checking the XOR-MAPPED-ADDRESS value of the Binding Response message that the client receives

Mapping behaviour

  •  Endpoint-Independent Mapping NAT (EIM-NAT)
  • Address-Dependent Mapping NAT (ADM-NAT)
  • Address and Port-Dependent Mapping NAT (APDM-NAT)

Filtering behaviour

  • Endpoint-Independent Filtering NAT (EIF-NAT)
  • Address-Dependent Filtering NAT (ADF-NAT)
  • Address and Port-Dependent Filtering NAT (APDF-NAT)

Hole Punching

As long as one end of the connection is able to determine the dynamic association of thee other [arty by NAT and send data , hole punching can work.

Permissive NAT mapping techniques which map the same internal address/port consistently to an external address/port are suitable for hole punching such as full cone , address or port restricted NAT. However pure symmetric NAT have inconsistent destination specific port mapping and thus cannot do hole punching.

1 . No Firewall present on either peer. Both connected to open public internet

Diagrammatic representation of  this shown as follows :

WebRTC signalling and media flow on Open public network
WebRTC signalling and media flow on Open public network

In this case there is no restriction to signal or media flow and the call takes places smoothly in p2p fashion.

2.  Either one or both the peer ( could be many in case of multi conf call ) are present behind a firewall  or  restrictive connection or router configured for intranet

In such a case the signal may pass with the use of default ICE candidates or simple ppensource google Stun server such as

iceServers:[
 { 'url': "stun:stun.l.google.com:19302"}]

Diagram :

WebRTC signalling when peers are behind  firewalls
WebRTC signalling when peers are behind firewalls

However the media is restricted resulting in a black / empty / no video situation for both peers  . To combat such situation a relay mechanism such as TURN is required which essentially maps public ip to private ips thus creating a alternative route for media and data to flow through .

WebRTC media flow when peers are behind NAT . Uses TURN relay mechanism
WebRTC media flow when peers are behind NAT . Uses TURN relay mechanism

Peer config should look like :

var configuration =  {
  iceServers: [
 { "url':"stun::"},
 { "url":"turn::"}
  ]};

var pc = new RTCPeerConnection(configuration);

3. When the TURN server is also behind a firewall

The config file of the turn server need to be altered to map the public and private IP. The diagrammatic description of this is as follows :

WebRTC media flow when peers are behind NAT and TURN server is behind NAT as well . TURN config files bind a public interface to private interface address.
WebRTC media flow when peers are behind NAT and TURN server is behind NAT as well . TURN config files bind a public interface to private interface address .

References :