GDPR , Europe’s digital privacy legislation passed in 2018, replaces the 1995 EU Data Protection Directive. It is rules designed to give EU citizens more control over their personal data & strengthen privacy rights. It aims to simplify the regulatory environment for business and citizens.
GDPR (General Data Protection Regulation) in European Union 2018,
California Consumer Privacy Act (CCPA) 2019,
Personal Data Protection Bill (PDP) – India 2018 and
also specifications against Robocalls and SPIT ( SPAM over Internet Telephony) among others
Multinational companies will predominantly be regulated by the supervisory authority where they have their “main establishment” or headquarter. However, the issue concerning GDPR is that it not only applies to any organisation operating within the EU, but also to any organisations outside of the EU which offer goods or services to customers or businesses in the EU.
Key Principles of GDPR are
Lawfulness, fairness and transparency
Purpose limitation
Data minimisation
Accuracy
Storage limitation
Integrity and confidentiality (security)
Accountability
GDPR consists of 7 projects (DPO, Impact assessment, Portability, Notification of violations, Consent, Profiling, Certification and Lead authority) that will strengthen the control of personal data throughout the European Union.
Stakeholders
stakeholders of data protection regulation are Data Subject – an individual, a resident of the European Union, whose personal data are to be protected
Data Controller – an institution, business or a person processing the personal data e.g. e-commerce website.
Data Protection Officer – a person appointed by the Data Controller responsible for overseeing data protection practices.
Data Processor – a subject (company, institution) processing a data on behalf of the controller. It can be an online CRM app or company storing data in the cloud.
Data Authority – a public institution monitoring implementation of the regulations in the specific EU member country.
Extra-Territorial Scope
Any VoIP service provider may feel that since they are not based out of EU such as officially headquartered in the Asia Pacific or US region they may not be legally binding to GDPR. However, GDPR expands the territorial and material scope of EU data protection law. It applies to both controllers and processors established in the EU, and those outside the EU, who offer goods or services to or monitor EU data subject.
VoIP service providers as Data Processors
A processor is a “person, public authority, agency or other body which processes personal data on behalf of the controller”. Most VoIP service providers are multinational in nature with services offered directly or indirectly to all regions. The GDPR imposes direct statutory obligations on data processors, which means they will be subject to direct enforcement by supervisory authorities, fines, and compensation claims by data subjects. However, a processor’s liability will be limited to the extent that it has not complied with it’s statutory and contractual obligations.
Data minimization – It is now a good practise to store and process as less user’s personal data as necessary to render our services effectively. Also to maintain data for only a stipulated time ( approx 90 days of CDR for call details and logs )
Record Keeping, Accountability and governance
To show compliance with GDPR, a service provider maintain detailed records of processing activities. Also, they must implement technological and organisational measures to ensure, and be able to demonstrate, that processing is performed in accordance with the GDPR. Some ways to apply these are :
Contracts: putting written contracts in place with organisations that process personal data on your behalf
maintaining documentation of your processing activities
Organisational policies focus on Data protection by design and default – two-factor auth, strong passwords to guard against brute-force, encryption, focus on security in architecture
Rish analysis and impact assessments: for uses of personal data that are likely to result in a high risk to individuals’ interests
Audit by Data protection officer
Clear Codes of conduct
Certifications
As for a VOIP landscape thankfully every call or message session is followed by a CDR ( Calld Detail Record ) or MDR ( Message Detail Record).
Additionally, assign a unique signature to every data-access client the VoIP system and log every read/write operation carried out on data stores whether persistent datastores or system caches.
Privacy Notices to Subjects
User profile data such as :
Basic identity information, name, address and ID numbers
Web data such as location, IP address, cookie data and RFID tags
Health and genetic data
Bio-metric data
Racial or ethnic data
Political opinions
Sexual orientation
is protected strictly under GDPR rules
A service provider should provide indepth information to data subjects when collecting their personal data, to ensure fairness and transparency. They must provide the information in an easily accessible form, using clear and plain language.
Consent
The GDPR introduces a higher bar for relying on consent , requiring clear affirmative action. Silence, pre ticked boxes or inactivity will not be sufficient to constitute consent. Data subjects can withdraw their consent at any time, and it must be easy for them to do so.
Lawful basis for processing Data now include
In Article 6 of the GDPR , there are six available lawful bases for processing.
(a) Consent: the individual has given clear consent for you to process their personal data for a specific purpose.
(b) Contract: the processing is necessary for a contract you have with the individual, or because they have asked you to take specific steps before entering into a contract.
(c) Legal obligation: the processing is necessary for you to comply with the law (not including contractual obligations).
(d) Vital interests: the processing is necessary to protect someone’s life.
(e) Public task: the processing is necessary for you to perform a task in the public interest or for your official functions, and the task or function has a clear basis in law.
(f) Legitimate interests: the processing is necessary for your legitimate interests or the legitimate interests of a third party, unless there is a good reason to protect the individual’s personal data which overrides those legitimate interests.
File such as PCAPS , Recordings and transcripts of calls hold sensitive information from end users , these should be encryoted and inaccssible to even the dev teams within the org without explicit consent of end user .
Individuals’ Rights
The GDPR provides individuals with new and enhanced rights to Data subjects who will have more control over the processing of their personal data. A data subject access request can only be refused if it is manifestly unfounded or excessive, in particular because of its repetitive character.
Rights of Data Subjets include
Right of Access
Right to Rectification
Right to Be Forgotten
Right to Restriction of Processing
Right to Data Portability
Right to Object
Right to Object to Automated Decisionmaking
For a VoIP service provider if a user opts for redaction then none of his calls or messages should be traced in logs . Also replace distinguishable end user identifier such as phone number and sip uri with *** charecters
Provide option for “Account Deletion” and purge account – If a user wished to close his/her account , his/her detaisl should be deleted form the sustem except for the bare bones detaisl which are otherwise required for legal , taxation and accounting requirnments
Breach Notification
A controller is a “person, public authority, agency or other body which, alone or jointly with others, determines the purposes and means of processing of personal data”,
A controller will have a mandatory obligation to notify his supervisory authority of a data breach within 72 hours unless the breach is unlikely to result in a risk to the rights of data subjects. Will also have to notify affected data subjects where the breach is likely to result in a “high risk” to their rights. A processor, however, will only be obliged to report data breaches to controllers
International Data Transfers
Data transfers to countries outside the EEA(European Economic Area) continue to be prohibited unless that country ensures an adequate level of protection. The GDPR retains existing transfer mechanisms and provides for additional mechanisms, including approved codes of conduct and certification schemes.
The GDPR prohibits any non-EU court, tribunal or regulator from ordering the disclosure of personal data from EU companies unless it requests such disclosure under an international agreement, such as a mutual legal assistance treaty.
One of the biggest challenges for a service provider is the identification & categorization of GDPR impacted data sets in disparate locations across the enterprise. A dev team must flag tables, attributes and other data objects that are categorically covered under GDPR regulations and then ensure that they are not transferred to a server outside of EU.
In the present age of Virtual shared server instance, cloud computing and VoIP protocol it is operational a very tough task for a communication service provider to ensure that data is not transferred outside of EU such as a VoIP call from origination in US and destination in EU will require information exchanges via SDP, vcard , RTP stream via media proxies etc.
Sanctions
The GDPR provides supervisory authorities with wide-ranging powers to enforce compliance, including the power to impose significant fines. You will face fines of up to €20m or 4% of your total worldwide annual turnover of the preceding financial year. In addition, data subjects can sue you for pecuniary or non-pecuniary damages (i.e. distress). Supervisory authorities will have a discretion as to whether to impose a fine and the level of that fine.
Data Protection officer (DPO)
Under the terms of GDPR, an organisation must appoint a Data Protection Officer (DPO) if it carries out large-scale processing of special categories of data, carries out large scale monitoring of individuals such as behaviour tracking or is a public authority.
supports OAuth 2.0 based authentication. The application, acting as the OAuth Client, is responsible for refreshing the credential information and updating the ICE Agent with fresh new credentials before the accessToken expires. The OAuth Client can use the RTCPeerConnection setConfiguration method to periodically refresh the TURN credentials.
RTCOAuthCredential Dictionary
describe the OAuth auth credential information which is used by the STUN/TURN client (inside the ICE Agent) to authenticate against a STUN/TURN server
ICE candidate policy [JSEP] to select candidates for the ICE connectivity checks
relay – use only media relay candidates such as candidates passing through a TURN server. It prevents the remote endpoint/unknown caller from learning the user’s IP addresses
all – ICE Agent can use any type of candidate when this value is specified.
RTCBundlePolicy Enum
balanced – Gather ICE candidates for each media type (audio, video, and data). If the remote endpoint is not bundle-aware, negotiate only one audio and video track on separate transports.
max-compat – Gather ICE candidates for each track. If the remote endpoint is not bundle-aware, negotiate all media tracks on separate transports.
max-bundle – Gather ICE candidates for only one track. If the remote endpoint is not bundle-aware, negotiate only one media track. If the remote endpoint is bundle-aware, all media tracks and data channels are bundled onto the same transport.
If the value of configuration.bundlePolicy is set and its value differs from the connection’s bundle policy, throw an InvalidModificationError.
RTCRtcpMuxPolicy Enum
what ICE candidates are gathered to support non-multiplexed RTCP.
negotiate – Gather ICE candidates for both RTP and RTCP candidates. If the remote-endpoint is capable of multiplexing RTCP, multiplex RTCP on the RTP candidates. If it is not, use both the RTP and RTCP candidates separately.
require – Gather ICE candidates only for RTP and multiplex RTCP on the RTP candidates. If the remote endpoint is not capable of rtcp-mux, session negotiation will fail.
If the value of configuration.rtcpMuxPolicy is set and its value differs from the connection’s rtcpMux policy, throw an InvalidModificationError. If the value is “negotiate” and the user agent does not implement non-muxed RTCP, throw a NotSupportedError.
An RTCPeerConnection object has an operations chain which ensures that only one asynchronous operation in the chain executes concurrently.
Also an RTCPeerConnection object MUST not be garbage collected as long as any event can cause an event handler to be triggered on the object. When the object’s internal slot is true ie closed, no such event handler can be triggered and it is therefore safe to garbage collect the object.
CreateOffer() – generates a blob of SDP that contains an RFC 3264 offer with the supported configurations for the session, including
descriptions of the local MediaStreamTracks attached to this RTCPeerConnection,
codec/RTP/RTCP capabilities
ICE agent (usernameFragment, password , local candiadtes etc )
DTLS connection
var pc = new RTCPeerConnection();
pc.createOffer({
mandatory: {
OfferToReceiveAudio: true,
OfferToReceiveVideo: true
},
optional: [{
VoiceActivityDetection: false
}]
}).then(function(offer) {
return pc.setLocalDescription(offer);
})
.then(function() {
// Send the offer to the remote through signaling server
})
.catch(handleError);
CreateAnswer() – generates an SDPanswer with the supported configuration for the session that is compatible with the parameters in the remote configuration
var pc = new RTCPeerConnection();
pc.createAnswer({
OfferToReceiveAudio: true
OfferToReceiveVideo: true
})
.then(function(answer) {
return pc.setLocalDescription(answer);
})
.then(function() {
// Send the answer to the remote through signaling server
})
.catch(handleError);
Codec preferences of an m= section’s associated transceiver is said to be the value of the RTCRtpTranceiver with the following filtering applied
If direction is “sendrecv”, exclude any codecs not included in the intersection of RTCRtpSender.getCapabilities(kind).codecs and RTCRtpReceiver.getCapabilities(kind).codecs.
If direction is “sendonly”, exclude any codecs not included in RTCRtpSender.getCapabilities(kind).codecs.
If direction is “recvonly”, exclude any codecs not included in RTCRtpReceiver.getCapabilities(kind).codecs.
RTCPriorityType Priority and QoS Model which can be
“very-low”, “low”, “medium”, “high”
RTP Media API
Send and receive MediaStreamTracks over a peer-to-peer connection. Tracks, when added to an RTCPeerConnection, result in signaling; when this signaling is forwarded to a remote peer, it causes corresponding tracks to be created on the remote side.
The actual encoding and transmission of MediaStreamTracks is managed through objects called RTCRtpSenders. Similarly, the reception and decoding of MediaStreamTracks is managed through objects called RTCRtpReceivers. These are associated with one track.
RTCRtpTransceivers are created implicitly when the application attaches a MediaStreamTrack to an RTCPeerConnection via the addTrack(), or explicitly when the application uses the addTransceiver(). They are also created when a remote description is applied that includes a new media description.
dictionary RTCRtpCodecParameters {
required octet payloadType;
required DOMString mimeType;
required unsigned long clockRate;
unsigned short channels;
DOMString sdpFmtpLine;
};
payloadType – identify this codec. mimeType – codec MIME media type/subtype. Valid media types and subtypes are listed in [IANA-RTP-2] clockRate – expressed in Hertz channels – number of channels (mono=1, stereo=2). sdpFmtpLine – “format specific parameters” field from the “a=fmtp” line in the SDP corresponding to the codec
voiceActivityFlag of type boolean – Only present for audio receivers. Whether the last RTP packet, delivered from this source, contains voice activity (true) or not (false).
RTCRtpTransceiver Interface
Each SDP media section describes one bidirectional SRTP (“Secure Real Time Protocol”) stream. RTCRtpTransceiver describes this permanent pairing of an RTCRtpSender and an RTCRtpReceiver, along with some shared state. It is uniquely identified using its mid property.
Thus it is combination of an RTCRtpSender and an RTCRtpReceiver that share a common mid. An associated transceiver( with mid) is one that’s represented in the last applied session description.
Method stop() – Irreversibly marks the transceiver as stopping, unless it is already stopped. This will immediately cause the transceiver’s sender to no longer send, and its receiver to no longer receive. stopping transceiver will cause future calls to createOffer to generate a zero port in the media description for the corresponding transceiver and stopped transceiver will cause future calls to createOffer or createAnswer to generate a zero port in the media description for the corresponding transceiver
Methods setCodecPreferences() – overrides the default codec preferences used by the user agent.
Example setting codec Preferebec for OPUS in audio
peer = new RTCPeerConnection();
const transceiver = peer.addTransceiver('audio');
const audiocapabilities = RTCRtpSender.getCapabilities('audio');
let codec = [];
codec.push(audiocapabilities.codecs[0]);
transceiver.setCodecPreferences(codec);
Access to information about the Datagram Transport Layer Security (DTLS) transport over which RTP and RTCP packets are sent and received by RTCRtpSender and RTCRtpReceiver objects, as well other data such as SCTP packets sent and received by data channels. Each RTCDtlsTransport object represents the DTLS transport layer for the RTP or RTCP component of a specific RTCRtpTransceiver, or a group of RTCRtpTransceivers if such a group has been negotiated via [BUNDLE].
“new”- DTLS has not started negotiating yet. “connecting” – DTLS is in the process of negotiating a secure connection and verifying the remote fingerprint. “connected”- DTLS has completed negotiation of a secure connection and verified the remote fingerprint. “closed” – transport has been closed intentionally like close_notify alert, or calling close(). “failed” – transport has failed as the result of an error like failure to validate the remote fingerprint
Protocols multiplexed with RTP (e.g. data channel) share its component ID. This represents the component-id value 1 when encoded in candidate-attribute while ICE candadte for RTCP has component-id value 2 when encoded in candidate-attribute.
This interface candidate Internet Connectivity Establishment (ICE) configuration used to setup RTCPeerconnection. To facilitate routing of media on given peer connection, both endpoints exchange several candidates and then one candidate out of the lot is chosen which will be then used to initiate the connection.
candidate – transport address for the candidate that can be used for connectivity checks.
component – candidate is an RTP or an RTCP candidate
foundation – unique identifier that is the same for any candidates of the same type , helps optimize ICE performance while prioritizing and correlating candidates that appear on multiple RTCIceTransport objects.
ip , port
priority
protocol – tcp/udp
relatedAddress , relatedPort
sdpMid – candidate’s media stream identification tag
sdpMLineIndex
usernameFragment – randomly-generated username fragment (“ice-ufrag”) which ICE uses for message integrity along with a randomly-generated password (“ice-pwd”).
active – An active TCP candidate is one for which the transport will attempt to open an outbound connection but will not receive incoming connection requests.
passive – A passive TCP candidate is one for which the transport will receive incoming connection attempts but not attempt a connection.
so – An so candidate is one for which the transport will attempt to open a connection simultaneously with its peer.
UDP candidate type
host – actual direct IP address of the remote peer
srflx – server reflexive , generated by a STUN/TURN server
prflx – peer reflexive ,IP address comes from a symmetric NAT between the two peers, usually as an additional candidate during trickle ICE
usernameFragment – randomly-generated username fragment (“ice-ufrag”) which ICE uses for message integrity along with a randomly-generated password (“ice-pwd”).
Access to information about the ICE transport over which packets are sent and received. Each RTCIceTransport object represents the ICE transport layer for the RTP or RTCP component of a specific RTCRtpTransceiver, or a group of RTCRtpTransceivers if such a group has been negotiated via [BUNDLE].
“new” – ICE agent is gathering addresses or is waiting to be given remote candidates
“checking” –
“connected” – Found a working candidate pair, but still performing connectivity checks to find a better one.
“completed” – Found a working candidate pair and done performing connectivity checks.
“disconnected”,
“failed”,
“closed”
RTCIceRole Enum
“unknown”, // agent who role is not yet defined “controlling”, // controlling agent “controlled” // controlled agent
RTCIceComponent Enum
“rtp”, // ICE Transport is used for RTP (or RTCP multiplexing) “rtcp” // ICE Transport is used for RTCP
Peer-to-peer Data API
-tbd
Peer-to-peer DTMF
-tbd
Statistics Model
The browser maintains a set of statistics for monitored objects, in the form of stats objects. A group of related objects may be referenced by a selector( like MediaStreamTrack that is sent or received by the RTCPeerConnection).
Statistics API extends the RTCPeerConnection interface
RTCReceivedRTPStreamStats, all required attributes from its inherited dictionaries, and also attributes packetsReceived, packetsLost, jitter, packetsDiscarded
RTCInboundRTPStreamStats, all required attributes from its inherited dictionaries, and also attributes trackId, receiverId, remoteId, framesDecoded, nackCount
RTCRemoteInboundRTPStreamStats, all required attributes from its inherited dictionaries, and also attributes localId, bytesReceived, roundTripTime
RTCSentRTPStreamStats, with all required attributes from its inherited dictionaries, and also attributes packetsSent, bytesSent
RTCOutboundRTPStreamStats, with all required attributes from its inherited dictionaries, and also attributes trackId, senderId, remoteId, framesEncoded, nackCount
RTCRemoteOutboundRTPStreamStats, with all required attributes from its inherited dictionaries, and also attributes localId, remoteTimestamp
RTCPeerConnectionStats, with attributes dataChannelsOpened, dataChannelsClosed
SBC ( Session Borde Controllers ) are basically gateways that provide interconnectivity between the hosted IP-PBX of the enterprise to the outside world endpoints such as telco service provider, PSTN/ TDM , SIP trunking providers or even third party OTT provider apps like skype for business etc.
If you have a hosted IPPBX or PBX in your data-centre or on premise and you need controlled but heavy outflowing traffic, it is a good idea to integrate a resilient and efficient SBC to provide seamless interconnectivity.
Hosted PBX
For an enterprises such as an Trading floor or warehouse with multiple phone types , softphones , hardphones , turrets etc distributed across various geographies and zones a device agnostic architectural setup is prime . Listing the essentials for setting up such a system. Note supplementary services are data-services , logging , licensing etc are important but kept out of scope to keep focus on functional aspects .
An enterprise application usually is structured in tiers or layers
Client tier – the networks clients communication to the central java programs . Runs on client machines
web tier – state full communication between client and business tier . Runs in server machine.
business tier- handles the logic of the application. The business tier uses the Enterprise Java Bean (EJB) container, which manages the execution of the beans
data tier – encompasses DB drivers . Runs on separate machines for database storage
Event services for Line status notifications
providers lines status notification across enterprise for inter zone and softphone to hardphone .
Routing services
routing calls within enterprise and hardphone sites read more about resource zones later in the article
Call Control Manager (CCM)
consolidated set of all service and component that make up the VOIP platform besides media handlers . It includes SIP adapters , bridge managers , call processing frameworks , API frameworks , healthchecks etc .
Call processing framework ( CPF)
signalling and call routing logic , mostly in SIP and trunks . Manages identities such as Call Line information , Called Party Information , line status etc in shared memory.
Multiple shared Lines and their statuses
Incases where there is a need to process multiple calls from a single User agent device such as a softphone or hardphone ( common scenario for a turret phone) , the design involves assigning it multiple sip uris and each sip uri will establish a line.
When caller calls callee , the line is said to be BUSY , otherwise said to be IDLE. Transition of a shared sip line from IDLE to BUSY is transmitted to others via SIP PUBLISH as other UAs holding the same sip
Similarly any other event like transfer is propagated to other via SIP UPDATE
Clustering Call control managers (CCM)
A Call Communication manager (CCM) from various zones should be able to cowork on call and session management and advanced features such as routing from home guest zone to home zone , call transfer , refer , barge etc. Designing a clustered setup will also provide elasticity , fail-over and high availability. Can use clustered , HA compliant framework such as Oracle Communication Application Server , suited for enterprise level deployments.
Call Replication and distributed memory management
A node will store two types of data: active sessions and passive sessions. The active sessions are used by the node and stored in cache. The passive sessions are the replicas from the other nodes’ active sessions. The passives sessions are stored on a persistent storage.
Controlling Line Calls using AOR and Resource Zones
When dealing with many SIP endpoints , now referred to as resource, it is best to assign the resources to their respective zones. Thus a resource’s status updates will be only updated by its active resource zone while can be read by any resource zone.
Incoming request Zone vs Active Resource Zone
For an Incoming request such a INVITE , check whether the zone sending the request is its active resource zone or not .If the Active Resource Zone is the same zone on which the INVITE came in, then the call is handled by that zone. If the Active Resource Zone is a different zone, then the call needs to be forwarded to the Active Resource Zone.
Bridges for Local Media connections
Although call signalling is handled by a resources active resource zone only, we can still create media bridges in local zone of the resource .
Local MM bridges are used to auto answer an incoming sip line call and create trunk , especially from hardphones which do not support provisional responses.
Interzone proxy Handler
proxies call control messages between active and non active resource zones. Primarily mapping the sip messages with all custom headers inbetween the communication device interfaces.
Dial Trunk using multiple dedicated sip lines and connect via Media Bridge
To save up on call routing /connection time and to support te ability to add as many users on call at runtime , a dedicated media bridge is established for every call.
A sip line activated is auto-answered by MM , creates a trunk and waits for other endpoint to join the bridge. The flow is as follows :
As INVITE arrives for an IDLE sip line , it is connected to a trunk and auto answered by a local MM bridge .
Since the call is already answered , when caller dials number for callee , collect the DTMF digits over RTP using RFC 2833 DTMF events.
Run inter-digit timer for digit collection and detect end of dialing on timeout.
The dialed trunk connection is made and call is added to media bridge
When provisional responses are received on the trunk connection, generate in-band call progress tones (ringing, proceeding etc) via the MM
When the line answers, the progress tones have to be stopped and the called party gets bridged to the calling party via the media bridge.
Call Diversion involves forwarding calls from zone to another zone. joinjed parties get call UPDATE status and forward response .
Call barge is the processing of joining an ongoing call . The barge event is usually propagated to joined parities via SIP INFO. Private lines do not allow barge in and are exclusively reserved for only few users.
Interconnectivity provided by an SBC ( Session Border Controller)
Hold-Resume and Music on Hold in multi-line evironment
While a regular p2p call involves simple reinvite based hold and resume with varrying SDP, the scenario is slightly more detailed for hold resume on bridged trunk connection , as explained below.
As the calls made are on bridge , a hold signal involves a RE-INIVITE with held-SDP to media manager (MM). If hold status on trunk is 200 OK the hold status will be sent to other call interfaces connected on the trunk. Else if hold is denied ,403 is sent back to hold-initiates.
Music on hold is an one way RTP mostly from media server.
For a bridged scenarios , separate Music on hold bridges are kept on Media Managers. When an UA has to hold , it is removed from original bridge and place on music on hold bridge . To be unhold/ resume it is placed back into the orignal bridge from music on hold bridge .
Conference
user initiates conference, the conference feature can execute on the zone where the user was logged on, irrespective of zones where the other conference attendees join from . The Call processing framework of originators zone completes the SDP exchange to establish two-way speech path among all the parties.
Incases there are multiple connections from a zone , a local MM conference bridge can be created for them which would connect back to originators MM conf bridge . this two part conf bridge will be transparent to the sip line sand users .
For provisioning inputs and settings setup a Diagnostics , Administration and Configuration platform which can process APIs for data services , licences , alarms or do remote device control such as using SNMP
Session Border Controllers (SBC)
At network level SBC operations include
bridging multiple interfaces in different networks even between the IPv4 and IPv6 networks
auto NAT discovery and STUN
protocol conversion such as TLS to UDP etc
Flood detection and IP filtering
For SIP specific functionalities , SBC does
SIP validation involving checks on syntax and message contents also consistency checks are performed.
stateful and call aware. tracing, monitoring and checking for validitya and health of all the SIP messages
Topology hiding
Traffic filtering
Codec filtering , reordering , media pinning, transcoding, or call recording
Data replication brings High Availability (HA) with hot backups or even Active-Active solutions.
Traffic sharing and routing roles of SBC can include
IP-based and Digest-based authentication
limiting traffic by number of concurrent calls or calling rate.
Dialplan and/or Custom routing
Dispatching/Load-balancing to a backend cluster of servers
SBC’s can be physical hardware boxes or software based applications, as the name suggests their purpose is to control the session at border between the enterprise and external service provider.
SIP to PSTN – SIP is an IP protocol whereas PSTN is a TDM one , achieving interoperability is also the KRA of an SBC
SIP trunking – SBC provide a secure sip connectivity to connect calls to sip trunks which provide bulk calls functionality at a flat pricing.
support for various fixed or mobile endpoints – SBC ensure they are RFC compliant and can extend SIP to any kind of telecom endpoint like PSTN , GSM, fax , Skype , sipphone , IP phones etc.
NAT / Network address translator – To meet the packet routing challenges across a firewall or even during private -public mapping. A combo of DHCP servers and NAT provider comes very handy to reroute or perform hole punching such that signalling and media packets are not dropped and meet the required endpoint. More about NAT here – NAT traversal using STUN and TURN.
Load balancing – Reverse proxies and Load balancers is a much adopted industry practise to mask the inner IPs of the VoIP platform and also route traffic appropriately between control and media server .
Security , QoS and Regulatory compliance – since SBCs are required to typically support a large array of clients they adhere to regulatory and industry accepted standards ,which also involves security features like AAA, TLS/SSL and other means for quality of assurance like logging and fault detection, preventing DDoS etc . In many cases SBC can also encrypt / decrypt RTP streams for probing , tapping or lawful inspection .
Terminating at carriers , PSTN and IP gateways
Additional SBC features
Inaddition to above it is good to have if an SBC provides extra features like forking , emergency number dialing ( 911 ) or active directory integration . Real Time Analysis and monitoring of call and metrics are also expected from a SBC since they reside on edge of the network and are more vulnerable to threats . For example Dialogic Mediant SBC’s and gateways , Audio Codes SBCs
With the shift from on premise PBXs to cloud based VM or microservice architecture , SBC vendors adopt a lager umbrella of services also including automation scripts for checks , reporting tools / consoles , developer friendly APIs to manage sessions via SBC and even WebRTC gateways to connect browser endpoints .
Usage Scenarios
Any VOIP dependant system which deals with bulksome voice / video traffic from external endpoints is a usages scenarios. Listing few
Contact Call centres
Remote work / offsite monitoring
CRM solution for sales/marketing
Connecting webrtc click to dial from webpage to enterprise representatives
connecting enterprise UCC clients to PSTN endpoints
Developing SIP-based applications – basic call routing, media management
SIP platform Development – NAt and DNS , Cross-platform and integration to External Telecommunication provider landscape , Databases
The contenst of this article are
PCAP Collection
CICD over Jenkins
Configuration management using chef cookbooks
virtualization and containerization using Docker
Infrastructure management using terraform / Kubernetes
Logs Analysis and Alarming
Overview of VoIP platform DevOPS tools
PCAP Collection
Data to be captured from Pcap,
DTMF – Both in-band and out of band DTMF for every call, along with the time stamp.
Codec negotiations – Extracting codecs from PCAP lets us
Validate later whether there were codec changes without prior SIP message,
If the call has been hung up with 488 error code then it was due to which codec
SIP errors – track deviations from standard SIP messaging.
Identify known erroneous SIP messaging scenarios such as for MITM or replay attacks
RTCP Media stats – extract Jitter, Loss, RTT with RTCP reports for both the incoming and outgoing stream.
Identify Media or ACK Timeouts
Check whether a party has not sent any media packet for > 60 s (media time out threshold duration)
When a call is hung up due to ACK time out.
Audio stream – After GDPR, take explicit permission from users before storing audio streams
Continuous Integration and Delivery Automation using Jenkins
CICD provides continous delivery hub , distribute work across multiple machines, helping drive builds, tests and deployments across multiple platforms .
Jenkins jobs is a self-contained Java-based program extensible using plugins.
Jenkins pieline– orchestrates and automates building project in Jenkins
Configuration management using chef cookbooks
Alternatives like puppet and Ansible, which are also a cross-platform configuration management platform
Compute virtualization and containerization using Docker
Docker containers can be used instead of virtual machines such as VirtualBox , to isolates applications and be OS and platform independent
Makes distributed development possible and automates the deployment possible
unpause Unpause all processes within one or more containers
update Update configuration of one or more containers
wait Block until one or more containers stop, then print their exit codes
see all iamges
> docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
sipcapture/homer-cron latest fb2243f90cde 3 hours ago 476MB
sipcapture/homer-kamailio latest f159d46a22f3 3 hours ago 338MB
sipcapture/heplify latest 9f5280306809 21 hours ago 9.61MB
<none> <none> edaa5c708b3a
See all stats
> docker stats
CONTAINER ID NAME CPU % MEM USAGE / LIMIT MEM % NET I/O BLOCK I/O PIDS
f42c71741107 homer-cron 0.00% 52KiB / 994.6MiB 0.01% 2.3kB / 0B 602MB / 0B 0
0111765091ae mysql 0.04% 452.2MiB / 994.6MiB 45.46% 1.35kB / 0B 2.06GB / 49.2kB 22
Run command from within a docker instnace
docker exec -it bash
First see all processes
docker ps
select a process and enter its bash
docker exec -it 0472a5127fff bash
to edit or update a file inside docker either install vim everytime u login in resh docker conainer like
apt-get update
apt-get install vim
or add this to dockerfile
RUN [“apt-get”, “update”] RUN [“apt-get”, “install”, “-y”, “vim”]
see if ngrep is install , if not then install and run ngrep to get sip logs isnode that docker container
apt update
apt install ngrep
ngrep -p "14795778704" -W byline -d any port 5060
docker volume – Volumes are used for persisting data generated by and used by Docker containers. docker volumes have advantages over blind mounts such as easier to backup or migrate , managed by docker APIs, can be safely shared among multiple containers etc
docker stack – Lets to manager a cluster of docker containers thorugh docker swarm can be defined via docker-compose.yml file
docker service
create Create a new service
inspect Display detailed information on one or more services
logs Fetch the logs of a service or task
ls List services
ps List the tasks of one or more services
rm Remove one or more services
rollback Revert changes to a service’s configuration
scale Scale one or multiple replicated services
update Update a service
Run docker containers
sample run command
docker run -it -d --name opensips -e ENV=dev imagename:2.2
-it flags attaches to an interactive tty in the container.
-e gives envrionment variables
-d runs it in background and prints container id
Remove docker entities
To remove all stopped containers, all dangling images, and all unused networks:
docker system prune -a
To remove all unused volumes
docker system prune --volumes
To remove all stopped containers
docker container prune
sometimes docker images keep piling with stopped congainer such as
REPOSITORY TAG IMAGE ID CREATED SIZE d1dcfe2438ae 15 minutes ago 753MB 2d353828889b 16 hours ago 910MB ...
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
0dd6698a7517 2d353828889b "/entrypoint.sh" 13 minutes ago Exited (137) 13 minutes ago hardcore_wozniak
to remove such images and their conainer , first stop and remove confainers
docker stop $(docker ps -a -q)
docker rm $(docker ps -a -q)
Terraform is used for building, changing and versioning infrastructure. Infra as Code – can run single application to datacentres via configuration files which create execution plan. It can manage low-level components such as compute instances, storage, and networking, as well as high-level components such as DNS entries, SaaS features, etc. Resource Graph – builds a graph of all your resources
tfenv can be used to manage terraform versions
> brew unlink terraform
tfenv install 0.11.14
tfenv list
Terraform configuration language
used for declaring resources and descriptions of infrastructure .tf or .tf.json file extension Group of resources can be gathered into a module Terraform configuration consists of a root module, where evaluation begins, along with a tree of child modules created when one module calls another.
Exmaple : launch a single AWS EC2 instance , fle server1.tf
console Interactive console for Terraform interpolations
destroy Destroy Terraform-managed infrastructure
env Workspace management
fmt Rewrites config files to canonical format
get Download and install modules for the configuration
graph Create a visual graph of Terraform resources
import Import existing infrastructure into Terraform
init Initialize a Terraform working directory
output Read an output from a state file
plan Generate and show an execution plan
providers Prints a tree of the providers used in the configuration
refresh Update local state file against real resources
show Inspect Terraform state or plan
taint Manually mark a resource for recreation
untaint Manually unmark a resource as tainted
validate Validates the Terraform files
version Prints the Terraform version
workspace Workspace management
0.12upgrade Rewrites pre-0.12 module source code for v0.12
debug Debug output management (experimental)
force-unlock Manually unlock the terraform state
push Obsolete command for Terraform Enterprise legacy (v1)
state Advanced state management
terraform init
initialize a working directory containing Terraform configuration files.
terraform validate
checks that verify whether a configuration is internally-consistent, regardless of any provided variables or existing state.
Kubernetes
container orchestration platform , automating deployment, scaling, and management of containerized applications. Can deploy to cluster of computers, automating the distribution and scheduling as well
Service discovery and load balancing – gives Pods their own IP addresses and a single DNS name for a set of Pods, and can load-balance across them.
Automatic bin packing – Automatically places containers based on their resource requirements and other constraints, while not sacrificing availability. Mix critical and best-effort workloads in order to drive up utilization and save even more resources.
Storage orchestration – Automatically mount the storage system of your choice, whether from local storage, a public cloud provider such as GCP or AWS, or a network storage system such as NFS, iSCSI, Gluster, Ceph, Cinder, or Flocker.
Self-healing – Restarts containers that fail, replaces and reschedules containers when nodes die, kills containers that don’t respond to your user-defined health check, and doesn’t advertise them to clients until they are ready to serve.
Automated rollouts and rollbacks – progressively rolls out changes to your application or its configuration, while monitoring application health to ensure it doesn’t kill all your instances at the same time.
Secret and configuration management – Deploy and update secrets and application configuration without rebuilding your image and without exposing secrets in your stack configuration.
Batch execution– manage batch and CI workloads, replacing containers that fail, if desired.
Horizontal scaling – Scale application up and down with a simple command, with a UI, or automatically based on CPU usage.
Starting Kubernetes…minikube version: v1.3.0
commit: 43969594266d77b555a207b0f3e9b3fa1dc92b1f
minikube v1.3.0 on Ubuntu 18.04
Running on localhost (CPUs=2, Memory=2461MB, Disk=47990MB) …
OS release is Ubuntu 18.04.2 LTS
Preparing Kubernetes v1.15.0 on Docker 18.09.5 …
kubelet.resolv-conf=/run/systemd/resolve/resolv.conf
Pulling images …
Launching Kubernetes …
Done! kubectl is now configured to use "minikube"
dashboard was successfully enabled
Kubernetes Started
Basic Commands
start Starts a local kubernetes cluster
status Gets the status of a local kubernetes cluster
stop Stops a running local kubernetes cluster
delete Deletes a local kubernetes cluster
dashboard Access the kubernetes dashboard running within the minikube cluster
Images Commands:
docker-env Sets up docker env variables; similar to ‘$(docker-machine env)’
cache Add or delete an image from the local cache.
Configuration and Management Commands:
addons Modify minikube’s kubernetes addons
config Modify minikube config
profile Profile gets or sets the current minikube profile
update-context Verify the IP address of the running cluster in kubeconfig.
Networking and Connectivity Commands:
service Gets the kubernetes URL(s) for the specified service in your local cluster
tunnel tunnel makes services of type LoadBalancer accessible on localhost
Advanced Commands:
mount Mounts the specified directory into minikube
ssh Log into or run a command on a machine with SSH; similar to ‘docker-machine ssh’
kubectl Run kubectl
Troubleshooting Commands:
ssh-key Retrieve the ssh identity key path of the specified cluster
ip Retrieves the IP address of the running cluster
logs Gets the logs of the running instance, used for debugging minikube, not user code.
update-check Print current and latest version number
kubectl
controls the Kubernetes cluster manager.
Basic Commands (Beginner):
create Create a resource from a file or from stdin.
expose Take a replication controller, service, deployment or pod and expose it as a new Kubernetes Service
run Run a particular image on the cluster
set Set specific features on objects
explain Documentation of resources
get Display one or many resources
edit Edit a resource on the server
delete Delete resources by filenames, stdin, resources and names, or by resources and label selector
Deploy Commands:
rollout Manage the rollout of a resource
scale Set a new size for a Deployment, ReplicaSet, Replication Controller, or Job
autoscale Auto-scale a Deployment, ReplicaSet, or ReplicationController
Cluster Management Commands:
certificate Modify certificate resources.
cluster-info Display cluster info
top Display Resource (CPU/Memory/Storage) usage.
cordon Mark node as unschedulable
uncordon Mark node as schedulable
drain Drain node in preparation for maintenance
taint Update the taints on one or more nodes
Troubleshooting and Debugging Commands:
describe Show details of a specific resource or group of resources
logs Print the logs for a container in a pod
attach Attach to a running container
exec Execute a command in a container
port-forward Forward one or more local ports to a pod
proxy Run a proxy to the Kubernetes API server
cp Copy files and directories to and from containers.
auth Inspect authorization
Advanced Commands:
diff Diff live version against would-be applied version
apply Apply a configuration to a resource by filename or stdin
patch Update field(s) of a resource using strategic merge patch
replace Replace a resource by filename or stdin
wait Experimental: Wait for a specific condition on one or many resources.
convert Convert config files between different API versions
kustomize Build a kustomization target from a directory or a remote url.
Settings Commands:
label Update the labels on a resource
annotate Update the annotations on a resource
completion Output shell completion code for the specified shell (bash or zsh)
Other Commands:
api-resources Print the supported API resources on the server
api-versions Print the supported API versions on the server, in the form of “group/version”
config Modify kubeconfig files
plugin Provides utilities for interacting with plugins.
version Print the client and server version information
DevOps monitoring tools nagios
Manage Docker configs
create Create a config from a file or STDIN
inspect Display detailed information on one or more configs
ls List configs
rm Remove one or more configs
Manage containers
attach Attach local standard input, output, and error streams to a running container
commit Create a new image from a container’s changes
cp Copy files/folders between a container and the local filesystem
create Create a new container
diff Inspect changes to files or directories on a container’s filesystem
exec Run a command in a running container
export Export a container’s filesystem as a tar archive
inspect Display detailed information on one or more containers
kill Kill one or more running containers
logs Fetch the logs of a container
ls List containers
pause Pause all processes within one or more containers
port List port mappings or a specific mapping for the container
prune Remove all stopped containers
rename Rename a container
restart Restart one or more containers
rm Remove one or more containers
run Run a command in a new container
start Start one or more stopped containers
stats Display a live stream of container(s) resource usage statistics
stop Stop one or more running containers
top Display the running processes of a container
unpause Unpause all processes within one or more containers
update Update configuration of one or more containers
wait Block until one or more containers stop, then print their exit codes
Alternatives, Senu multi-cloud monitoring or Raygun
Monitoring, debugging, logs analysis and alarms
Aggregate logs into logstash and provide search and filtering via Elastic Search and Kibana. Can also trigger alerts or notifications on specific keyword searches in logs such as WARNING or ERRRO or call_failed.
Some common alert scenarios include :
SBC and proxy gateways failures – check states of VM instance
DNS caching alerts – Domain Name System (DNS) caching, a Dynamic Host Configuration Protocol (DHCP) server, router advertisement and network boot alerts from service such as dnsmasq
Disk usage alert – setup alerts for 80% usage and trigger an alarm to either manually prune or create automatic timely archive backups. check the percentage of DISK USAGE
df -h
Mostly it is either the logs file or pcap recorder which need to be archieved in external storage.
Use logrotate – it can rotates, compresses, and mails system logs
config file for logrorate – logrotate -vf /etc/logrotate.conf
Elevated Call failure SIP 503 or Call timeout SIP 408 – high frequency of failed calls indicate an internal issue and must be followed up by smoke testing the entire system to identify any probable issue such as undetected frequent crashes of any individual component or any blacklisting by a destination endpoint etc
sudo tail -f sip.log | grep 503
or
sudo tail -f sip.log | grep WARNING
cron service or processed alerts –
ps axf
PID TTY STAT TIME COMMAND
2 ? S 0:00 [kthreadd]
3 ? I< 0:00 \_ [rcu_gp]
4 ? I< 0:00 \_ [rcu_par_gp]
5 ? I 0:00 \_ [kworker/0:0-eve]
6 ? I< 0:00 \_ [kworker/0:0H-kb]
7 ? I 0:00 \_ [kworker/0:1-eve]
8 ? I 0:00 \_ [kworker/u4:0-nv]
9 ? I< 0:00 \_ [mm_percpu_wq]
10 ? S 0:00 \_ [ksoftirqd/0]
11 ? I 0:00 \_ [rcu_sched]
12 ? S 0:00 \_ [migration/0]
13 ? S 0:00 \_ [cpuhp/0]
14 ? S 0:00 \_ [cpuhp/1]
15 ? S 0:00 \_ [migration/1]
16 ? S 0:00 \_ [ksoftirqd/1]
17 ? I 0:00 \_ [kworker/1:0-eve]
18 ? I< 0:00 \_ [kworker/1:0H-kb]
or checks cron status
service cron status
● cron.service - Regular background program processing daemon
Loaded: loaded (/lib/systemd/system/cron.service; enabled; vendor preset: enabled)
Active: active (running) since Fri 2016-06-26 03:00:37 UTC; 1min 17s ago
Docs: man:cron(8)
Main PID: 845 (cron)
Tasks: 1 (limit: 4383)
CGroup: /system.slice/cron.service
└─845 /usr/sbin/cron -f
Jun 26 03:00:37 ip-172-31-45-21 systemd[1]: Started Regular background program processing daemon.
Jun 26 03:00:37 ip-172-31-45-21 cron[845]: (CRON) INFO (pidfile fd = 3)
Jun 26 03:00:37 ip-172-31-45-21 cron[845]: (CRON) INFO (Running @reboot jobs)
restart or start cron service if required
DB connections / connection pool process – keep listening for any alerts on DB connections failure or even warnings as this can be due to too many read operations such as in DDOS and can escalate very quickly
cron zombie process checks – zombie process or defunct process is a process that has completed execution (via the exit system call) but still has an entry in the process table: it is a process in the “Terminated state”. List xombie process and kill them with pid to free up .
kill -9 <PID1>
bulk calls checks – consult ongoing call cmd commands for application server such as For Freeswitch use
Incase of DDOS or other macious attacker IP identification block the IP
iptables -I INPUT -s y.y.y.y -j DROP
Can also use fail2ban
>apt-get update && apt-get installfail2ban
Additionally check how many dispatchers are responding on outbound gateway
opensipsctl dispatcher dump
Process control supervisor or pm2 checks – supervisor is a Linux Process Control System that allows its users to monitor and control a number of processes
ps axf | grep supervisor
for pm2
> pm2 status
[PM2] Spawning PM2 daemon with pm2_home=/Users/altanai/.pm2
[PM2] PM2 Successfully daemonized
┌─────┬───────────┬─────────────┬─────────┬─────────┬──────────┬────────┬──────┬───────────┬──────────┬──────────┬──────────┬──────────┐
│ id │ name │ namespace │ version │ mode │ pid │ uptime │ ↺ │ status │ cpu │ mem │ user │ watching │
htop to check memeory and CPU
Health and load on the reverse proxy, load balancer as Nginx – perform a direct curl request to host to check if Nginx responds with a non 4xx / 5xx response or not
curl -v <public-fqdn-of-server>
Incase of error response , restart
/etc/init.d/nginx start
Incase of updates restart ngnix config
nginx -s reload
For HTTP/SSL proxy daemon such as tiny proxy which are used for fast resposne , set the MinSpareServers, MaxSpareServers , MaxClients , MaxRequestsPerChild etc appropriately
VPN checks – restart fireealls or IPsec incase of ssues
/etc/init.d/ipsec restart
Additionally also check ssh service
ps axf | grep sshd
restart sshd if required
SSL cert expiry checks – to keep the operations running securely and prevent and abrupt termination it is a good practise to run regular certificate expiry checks for SSL certs especially on secure HTTP endpoint like APIs , web server and also on SIP applications servers for TLS. If any expiry is due in < 10 days to trigger an alert to renew the certs
Health of Task scheduling services such as RabbitMQ, Celery Distributed Task Queue – remote debugging of these can be set up via pdb which supports setting (conditional) breakpoints and single stepping at the source line level, inspection of stack frames, source code listing, and evaluation of arbitrary Python code in the context of any stack frame.
It can also be set up via using the client libraries provided by these Queue services themselves
cluster status – setup an efficient health check service which monitors the cluster status for High Availability. Learn more about ensuring HA – JSON object depicting the status of cluster shards
fscli > show status UP 0 years, 0 days, 0 hours, 58 minutes, 33 seconds, 15 milliseconds, 58 microseconds FreeSWITCH (Version 1.6.20 git 987c9b9 2018-01-23 21:49:09Z 64bit) is ready 3 session(s) since startup 0 session(s) - peak 1, last 5min 1 0 session(s) per Sec out of max 30, peak 1, last 5min 1 1000 session(s) max min idle cpu 0.00/80.83 Current Stack Size/Max 240K/8192K
Programming or Syntax error in the production environment – mostly arising due to incomplete QA/testing before pushing new changes to production. Should trigger alerts for dev teams and meet with hot patches.
Many programing application development frameworks have inbuild libs for debugging , exceotion handling and reporting such as
SIPp is an opensource (GNU GPL license) performance testing tool for the SIP protocol and is widely used for Quality assurabce of callflows in voip applications for UAC / UASs cenarios.
It can emulate functioing of a sip phone such as REGISTER , establishes and releases multiple calls with the INVITE and BYE methods , send other SIP requests and wait for reponses based on dafult of custom xml scenario files.
Plus factor is the dynamic display of statistics about running tests (call rate, round trip delay, and message statistics), periodic CSV statistics dumps, TCP and UDP over multiple sockets or multiplexed with retransmission management, regular expressions and variables in scenario files, and dynamically adjustable call rates.
It is widley used as aperformnace and load testing tool since it can test SIP equipements like SIP proxies, B2BUAs, SIP media servers, SIP/x gateways, and SIP PBXes and can also emulate thousands of user agents calling your SIP system.
More on SIPp scripts and various exmaples can be read from
Pre-requisites to compile SIPp are: – C++ Compiler – curses or ncurses library – For TLS support: OpenSSL >= 0.9.8 – For pcap play support: libpcap and libnet – For SCTP support: lksctp-tools – For distributed pauses: Gnu Scientific Libraries
On the same host, run sipp with embedded client (uac) scenario:
sipp -sn uac 127.0.0.1 -trace_msg -trace_err
output for server
# sipp -sn uas
------------------------------ Scenario Screen -------- [1-9]: Change Screen --
Port Total-time Total-calls Transport
5060 32.95 s 61 UDP
0 new calls during 0.874 s period 1 ms scheduler resolution
19 calls Peak was 41 calls, after 28 s
0 Running, 63 Paused, 12 Woken up
0 dead call msg (discarded)
3 open sockets
———————————————– 2019-02-04 13:08:13.940159
UDP message sent (371 bytes):
ACK sip:service@127.0.0.1:5060 SIP/2.0
Via: SIP/2.0/UDP 192.x.x.x:5061;branch=z9hG4bK-52422-1-5
From: sipp ;tag=52422SIPpTag001
To: service ;tag=52416SIPpTag011
Call-ID: 1-52422@192.x.x.x
CSeq: 1 ACK
Contact: sip:sipp@192.x.x.x:5061
Max-Forwards: 70
Subject: Performance Test
Content-Length: 0
~ RTP
———————————————– 2019-02-04 13:08:13.941658
UDP message sent (371 bytes):
BYE sip:service@127.0.0.1:5060 SIP/2.0
Via: SIP/2.0/UDP 192.x.x.x:5061;branch=z9hG4bK-52422-1-7
From: sipp ;tag=52422SIPpTag001
To: service ;tag=52416SIPpTag011
Call-ID: 1-52422@192.x.x.x
CSeq: 2 BYE
Contact: sip:sipp@192.x.x.x:5061
Max-Forwards: 70
Subject: Performance Test
Content-Length: 0
———————————————– 2019-02-04 13:08:13.952888
UDP message received [313] bytes :
SIP/2.0 200 OK
Via: SIP/2.0/UDP 192.x.x.x:5061;branch=z9hG4bK-52422-1-7
From: sipp ;tag=52422SIPpTag001
To: service ;tag=52416SIPpTag011
Call-ID: 1-52422@192.x.x.x
CSeq: 2 BYE
Contact:
Content-Length: 0
Time
---------------------------- Repartition Screen ------- [1-9]: Change Screen --
Average Response Time Repartition 1
0 ms <= n < 10 ms : 293
10 ms <= n < 20 ms : 9
20 ms <= n < 30 ms : 0
30 ms <= n < 40 ms : 0
40 ms <= n < 50 ms : 0
50 ms <= n < 100 ms : 0
100 ms <= n < 150 ms : 0
150 ms <= n < 200 ms : 0
n >= 200 ms : 0
Average Call Length Repartition
0 ms <= n < 10 ms : 0
10 ms <= n < 50 ms : 0
50 ms <= n < 100 ms : 0
100 ms <= n < 500 ms : 0
500 ms <= n < 1000 ms : 0
1000 ms <= n < 5000 ms : 262
5000 ms <= n < 10000 ms : 0
n >= 10000 ms : 0
------------------------------ Sipp Server Mode -------------------------------
3 new calls during 0.286 s period 1 ms scheduler resolution
0 calls (limit 30) Peak was 25 calls, after 10 s
0 Running, 101 Paused, 7 Woken up
0 dead call msg (discarded) 0 out-of-call msg (discarded)
3 open sockets
----------------------------- Statistics Screen ------- [1-9]: Change Screen --
Start Time | 2019-02-04 13:08:03.908208 1549265883.908208
Last Reset Time | 2019-02-04 13:08:20.954289 1549265900.954289
Current Time | 2019-02-04 13:08:21.241152 1549265901.241152
-------------------------+---------------------------+--------------------------
Counter Name | Periodic value | Cumulative value
-------------------------+---------------------------+--------------------------
Elapsed Time | 00:00:00:286000 | 00:00:17:332000
Call Rate
Tracings
———————————————– 2019-02-04 13:08:13.934840
UDP message received [527] bytes :
INVITE sip:service@127.0.0.1:5060 SIP/2.0
Via: SIP/2.0/UDP 192.x.x.x:5061;branch=z9hG4bK-52422-1-0
From: sipp ;tag=52422SIPpTag001
To: service
Call-ID: 1-52422@192.x.x.x
CSeq: 1 INVITE
Contact: sip:sipp@192.x.x.x:5061
Max-Forwards: 70
Subject: Performance Test
Content-Type: application/sdp
Content-Length: 135
———————————————– 2019-02-04 13:08:13.948679
UDP message received [371] bytes :
ACK sip:service@127.0.0.1:5060 SIP/2.0
Via: SIP/2.0/UDP 192.x.x.x:5061;branch=z9hG4bK-52422-1-5
From: sipp ;tag=52422SIPpTag001
To: service ;tag=52416SIPpTag011
Call-ID: 1-52422@192.x.x.x
CSeq: 1 ACK
Contact: sip:sipp@192.x.x.x:5061
Max-Forwards: 70
Subject: Performance Test
Content-Length: 0
~ RTP
———————————————– 2019-02-04 13:08:13.949168
UDP message received [371] bytes :
BYE sip:service@127.0.0.1:5060 SIP/2.0
Via: SIP/2.0/UDP 192.x.x.x:5061;branch=z9hG4bK-52422-1-7
From: sipp ;tag=52422SIPpTag001
To: service ;tag=52416SIPpTag011
Call-ID: 1-52422@192.x.x.x
CSeq: 2 BYE
Contact: sip:sipp@192.x.x.x:5061
Max-Forwards: 70
Subject: Performance Test
Content-Length: 0
———————————————– 2019-02-04 13:08:13.949245
UDP message sent (313 bytes):
SIP/2.0 200 OK
Via: SIP/2.0/UDP 192.x.x.x:5061;branch=z9hG4bK-52422-1-7
From: sipp ;tag=52422SIPpTag001
To: service ;tag=52416SIPpTag011
Call-ID: 1-52422@192.x.x.x
CSeq: 2 BYE
Contact:
Content-Length: 0
time
---------------------------- Repartition Screen ------- [1-9]: Change Screen --
Average Response Time Repartition 1
0 ms <= n < 10 ms : 657
10 ms <= n < 20 ms : 20
20 ms <= n < 30 ms : 0
30 ms <= n < 40 ms : 0
40 ms <= n < 50 ms : 0
50 ms <= n < 100 ms : 0
100 ms <= n < 150 ms : 0
150 ms <= n < 200 ms : 0
n >= 200 ms : 0
Average Call Length Repartition
0 ms <= n < 10 ms : 649
10 ms <= n < 50 ms : 28
50 ms <= n < 100 ms : 0
100 ms <= n < 500 ms : 0
500 ms <= n < 1000 ms : 0
1000 ms <= n < 5000 ms : 0
5000 ms <= n < 10000 ms : 0
n >= 10000 ms : 0
------ [+|-|*|/]: Adjust rate ---- [q]: Soft exit ---- [p]: Pause traffic -----
Last Error: Overload warning: the major watchdog timer 3000ms has been t…
Run SIPp with embedded server (uas) scenario: ./sipp -sn uas On the same host, run SIPp with embedded client (uac) scenario: ./sipp -sn uac 127.0.0.1
Scenario file options:
-sd : Dumps a default scenario (embedded in the SIPp executable)
-sf : Loads an alternate XML scenario file. To learn more about XML scenario syntax, use the -sd option to dump embedded scenarios. They contain all the necessary help.
-oocsf : Load out-of-call scenario.
-oocsn : Load out-of-call scenario.
-sn : Use a default scenario (embedded in the SIPp executable). If this option is omitted, the Standard SipStone UAC scenario is loaded. Available values in this version:
‘uac’ : Standard SipStone UAC (default).
‘uas’ : Simple UAS responder.
‘regexp’ : Standard SipStone UAC – with regexp and variables.
‘branchc’ : Branching and conditional branching in scenarios – client.
‘branchs’ : Branching and conditional branching in scenarios – server.
Default 3pcc scenarios (see -3pcc option):
‘3pcc-C-A’ : Controller A side (must be started after all other 3pcc scenarios)
‘3pcc-C-B’ : Controller B side.
‘3pcc-A’ : A side.
‘3pcc-B’ : B side.
IP, port and protocol options
-t : Set the transport mode:
u1: UDP with one socket (default),
un: UDP with one socket per call,
ui: UDP with one socket per IP address. The IP addresses must be defined in the injection file.
t1: TCP with one socket,
tn: TCP with one socket per call,
l1: TLS with one socket,
ln: TLS with one socket per call,
c1: u1 + compression (only if compression plugin loaded),
cn: un + compression (only if compression plugin loaded). This plugin is not provided with SIPp.
-i : Set the local IP address for ‘Contact:’,’Via:’, and ‘From:’ headers. Default is primary host IP address.
-p : Set the local port number. Default is a random free port chosen by the system
-bind_local : Bind socket to local IP address, i.e. the local IP address is used as the source IP address. If SIPp runs in server mode it will only listen on the local IP address instead of all IP addresses.
-ci : Set the local control IP address
-cp : Set the local control port number. Default is 8888.
-max_socket : Set the max number of sockets to open simultaneously. This option is significant if you use one socket per call. Once this limit is reached, traffic is distributed over the sockets already opened. Default value is 50000
-max_reconnect : Set the the maximum number of reconnection.
-reconnect_close : Should calls be closed on reconnect?
-reconnect_sleep : How long (in milliseconds) to sleep between the close and reconnect?
-rsa : Set the remote sending address to host:port for sending the messages.
-tls_cert : Set the name for TLS Certificate file. Default is ‘cacert.pem
-tls_key : Set the name for TLS Private Key file. Default is ‘cakey.pem’
-tls_ca : Set the name for TLS CA file. If not specified, X509 verification is not activated.
-tls_crl : Set the name for Certificate Revocation List file. If not specified, X509 CRL is not activated.
-tls_version : Set the TLS protocol version to use (1.0, 1.1, 1.2) — default is autonegotiate
SIPp overall behavior options:
-v : Display version and copyright information.
-bg : Launch SIPp in background mode.
-nostdin : Disable stdin.
-plugin : Load a plugin.
-sleep : How long to sleep for at startup. Default unit is seconds.
-skip_rlimit : Do not perform rlimit tuning of file descriptor limits. Default: false.
-buff_size : Set the send and receive buffer size.
-sendbuffer_warn : Produce warnings instead of errors on SendBuffer failures.
-lost : Set the number of packets to lose by default (scenario specifications override this value).
-key : keyword value Set the generic parameter named “keyword” to “value”.
-set : variable value Set the global variable parameter named “variable” to “value”.
-tdmmap : Generate and handle a table of TDM circuits. A circuit must be available for the call to be placed. Format: -tdmmap {0-3}{99}{5-8}{1-31}
-dynamicStart : variable value Set the start offset of dynamic_id variable
-dynamicMax : variable value Set the maximum of dynamic_id variable
-dynamicStep : variable value Set the increment of dynamic_id variable
Call behavior options:
-aa : Enable automatic 200 OK answer for INFO, NOTIFY, OPTIONS and UPDATE.
-base_cseq : Start value of [cseq] for each call.
-cid_str : Call ID string (default %u-%p@%s). %u=call_number, %s=ip_address, %p=process_number, %%=% (in any order).
-d : Controls the length of calls. More precisely, this controls the duration of ‘pause’ instructions in the scenario, if they do not have a ‘milliseconds’ section. Default value is 0 and default unit is milliseconds.
-deadcall_wait : How long the Call-ID and final status of calls should be kept to improve message and error logs (default unit is ms).
-auth_uri : Force the value of the URI for authentication. By default, the URI is composed of remote_ip:remote_port.
-au : Set authorization username for authentication challenges. Default is taken from -s argument
-ap : Set the password for authentication challenges. Default is ‘password’
-s : Set the username part of the request URI. Default is ‘service’.
-default_behaviors: Set the default behaviors that SIPp will use. Possible values are:
all Use all default behaviors
none Use no default behaviors
bye Send byes for aborted calls
abortunexp Abort calls on unexpected messages
pingreply Reply to ping requests If a behavior is prefaced with a -, then it is turned off. Example: all,-bye
-nd : No Default. Disable all default behavior of SIPp which are the following:
On UDP retransmission timeout, abort the call by sending a BYE or a CANCEL
On receive timeout with no ontimeout attribute, abort the call by sending a BYE or a CANCEL
On unexpected BYE send a 200 OK and close the call
On unexpected CANCEL send a 200 OK and close the call
On unexpected PING send a 200 OK and continue the call
On any other unexpected message, abort the call by sending a BYE or a CANCEL
-pause_msg_ign : Ignore the messages received during a pause defined in the scenario
-callid_slash_ign: Don’t treat a triple-slash in Call-IDs as indicating an extra SIPp prefix.
Injection file options:
-inf : Inject values from an external CSV file during calls into the scenarios. First line of this file say whether the data is to be read in sequence (SEQUENTIAL), random (RANDOM), or user (USER) order. Each line corresponds to one call and has one or more ‘;’ delimited data fields. Those fields can be referred as [field0], [field1], … in the xml scenario file. Several CSV files can be used simultaneously (syntax: -inf f1.csv -inf f2.csv …)
-infindex : file field Create an index of file using field. For example -inf ../path/to/users.csv -infindex users.csv 0 creates an index on the first key.
-ip_field : Set which field from the injection file contains the IP address from which the client will send its messages. If this option is omitted and the ‘-t ui’ option is present, then field 0 is assumed. Use this option together with ‘-t ui’
RTP behaviour options:
-mi : Set the local media IP address (default: local primary host IP address)
-rtp_echo : Enable RTP echo. RTP/UDP packets received on port defined by -mp are echoed to their sender. RTP/UDP packets coming on this port + 2 are also echoed to their sender (used for sound and video echo).
-mb : Set the RTP echo buffer size (default: 2048).
-mp : Set the local RTP echo port number. Default is 6000.
-rtp_payload : RTP default payload type.
-rtp_threadtasks : RTP number of playback tasks per thread.
-rtp_buffsize : Set the rtp socket send/receive buffer size.
Call rate options:
-r : Set the call rate (in calls per seconds). This value can bechanged during test by pressing ‘+’, ‘_’, ‘*’ or ‘/’. Default is 10.
pressing ‘+’ key to increase call rate by 1 * rate_scale,
pressing ‘-‘ key to decrease call rate by 1 * rate_scale,
pressing ‘*’ key to increase call rate by 10 * rate_scale,
pressing ‘/’ key to decrease call rate by 10 * rate_scale.
-rp : Specify the rate period for the call rate. Default is 1 second and default unit is milliseconds. This allows you to have n calls every m milliseconds(by using -r n -rp m). Example: -r 7 -rp 2000 ==> 7 calls every 2 seconds. -r 10 -rp 5s => 10 calls every 5 seconds.
-rate_scale : Control the units for the ‘+’, ‘-‘, ‘*’, and ‘/’ keys.
-rate_increase : Specify the rate increase every -rate_interval units (default is seconds). This allows you to increase the load for each independent logging period. Example: -rate_increase 10 -rate_interval 10s ==> increase calls by 10 every 10 seconds.
-rate_max :
If -rate_increase is set, then quit after the rate reaches this value. Example: -rate_increase 10 -rate_max 100 ==> increase calls by 10 until 100 cps is hit.
-rate_interval : Set the interval by which the call rate is increased. Defaults to the value of -fd.
-no_rate_quit : If -rate_increase is set, do not quit after the rate reaches -rate_max.
-l : Set the maximum number of simultaneous calls. Once this limit is reached, traffic is decreased until the number of open calls goes down. Default: (3 * call_duration (s) * rate).
-m : Stop the test and exit when ‘calls’ calls are processed
-users : Instead of starting calls at a fixed rate, begin ‘users’ calls at startup, and keep the number of calls constant.
Retransmission and timeout options:
-recv_timeout : Global receive timeout. Default unit is milliseconds. If the expected message is not received, the call times out and is aborted.
-send_timeout : Global send timeout. Default unit is milliseconds. If a message is not sent (due to congestion), the call times out and is aborted.
-timeout : Global timeout. Default unit is seconds. If this option is set, SIPp quits after nb units (-timeout 20s quits after 20 seconds).
-timeout_error : SIPp fails if the global timeout is reached is set (-timeout option required).
-max_retrans : Maximum number of UDP retransmissions before call ends on timeout. Default is 5 for INVITE transactions and 7 for others.
-max_invite_retrans: Maximum number of UDP retransmissions for invite transactions before call ends on timeout.
-max_non_invite_retrans: Maximum number of UDP retransmissions for non-invite transactions before call ends on timeout.
-nr : Disable retransmission in UDP mode.
-rtcheck : Select the retransmission detection method: full (default) or loose.
-T2 : Global T2-timer in milli seconds
Third-party call control options:
-3pcc : Launch the tool in 3pcc mode (“Third Party call control”). The passed IP address depends on the 3PCC role.
When the first twin command is ‘sendCmd’ then this is the address of the remote twin socket. SIPp will try to connect to this address:port to send the twin command (This instance must be started after all other 3PCC scenarios). Example: 3PCC-C-A scenario.
When the first twin command is ‘recvCmd’ then this is the address of the local twin socket. SIPp will open this address:port to listen for twin command. Example: 3PCC-C-B scenario.
-master : 3pcc extended mode: indicates the master number
-slave : 3pcc extended mode: indicates the slave number
-slave_cfg : 3pcc extended mode: indicates the file where the master and slave addresses are stored
Performance and watchdog options:
-timer_resol Set the timer resolution. Default unit is milliseconds. This option has an impact on timers precision.Small values allow more precise scheduling but impacts CPU usage.If the compression is on, the value is set to 50ms. The default value is 10ms.
-max_recv_loops Set the maximum number of messages received read per cycle. Increase this value for high traffic level. The default value is 1000.
-max_sched_loops Set the maximum number of calls run per event loop. Increase this value for high traffic level. The default value is 1000.
-watchdog_interval : Set gap between watchdog timer firings. Default is 400.
-watchdog_reset : If the watchdog timer has not fired in more than this time period, then reset the max triggers counters. Default is 10 minutes.
-watchdog_minor_threshold: If it has been longer than this period between watchdog executions count a minor trip. Default is 500.
-watchdog_major_threshold: If it has been longer than this period between watchdog executions count a major trip. Default is 3000.
-watchdog_major_maxtriggers : How many times the major watchdog timer can be tripped before the test is terminated. Default is 10.
-watchdog_minor_maxtriggers: How many times the minor watchdog timer can be tripped before the test is terminated. Default is 120.
Tracing, logging and statistics options:
-f : Set the statistics report frequency on screen. Default is 1 and default unit is seconds.
-trace_stat : Dumps all statistics in <scenario_name>_.csv file. Use the ‘-h stat’ option for a detailed description of the statistics file content.
-stat_delimiter : Set the delimiter for the statistics file
-stf : Set the file name to use to dump statistics
-fd : Set the statistics dump log report frequency. Default is 60 and default unit is seconds.
-periodic_rtd : Reset response time partition counters each logging interval.
-trace_msg : Displays sent and received SIP messages in __messages.log
-message_file : Set the name of the message log file.
-message_overwrite: Overwrite the message log file (default true).
-trace_shortmsg : Displays sent and received SIP messages as CSV in <scenario file name>__shortmessages.log
-shortmessage_file: Set the name of the short message log file.
-shortmessage_overwrite: Overwrite the short message log file (default true).
-trace_counts : Dumps individual message counts in a CSV file.
-trace_err : Trace all unexpected messages in __errors.log.
-error_file : Set the name of the error log file.
-error_overwrite : Overwrite the error log file (default true).
-trace_error_codes: Dumps the SIP response codes of unexpected messages to <scenario file name>__error_codes.log.
-trace_calldebug : Dumps debugging information about aborted calls to <scenario_name>__calldebug.log file.
-calldebug_file : Set the name of the call debug file.
-calldebug_overwrite: Overwrite the call debug file (default true).
-trace_screen : Dump statistic screens in the <scenario_name>__screens.log file when quitting SIPp. Useful to get a final status report in background mode (-bg option).
-screen_file : Set the name of the screen file.
-screen_overwrite: Overwrite the screen file (default true).
-trace_rtt : Allow tracing of all response times in __rtt.csv.
-rtt_freq : freq is mandatory. Dump response times every freq calls in the log file defined by -trace_rtt. Default value is 200.
-trace_logs : Allow tracing of actions in __logs.log.
-log_file : Set the name of the log actions log file.
-log_overwrite : Overwrite the log actions log file (default true).
-ringbuffer_files: How many error, message, shortmessage and calldebug files should be kept after rotation?
-ringbuffer_size : How large should error, message, shortmessage and calldebug files be before they get rotated?
-max_log_size : What is the limit for error, message, shortmessage and calldebug file sizes.
Signal handling:
SIPp can be controlled using POSIX signals. The following signals are handled: USR1: Similar to pressing the ‘q’ key. It triggers a soft exit of SIPp. No more new calls are placed and all ongoing calls are finished before SIPp exits. Example: kill -SIGUSR1 732 USR2: Triggers a dump of all statistics screens in <scenario_name>__screens.log file. Especially useful in background mode to know what the current status is. Example: kill -SIGUSR2 732
Exit codes:
Upon exit (on fatal error or when the number of asked calls (-m option) is reached, SIPp exits with one of the following exit code: 0: All calls were successful 1: At least one call failed 97: Exit on internal command. Calls may have been processed 99: Normal exit without calls processed -1: Fatal error -2: Fatal error binding a socket
Debugging
Issue1 The commonName field needed to be supplied and was missing
Solution Given the common name while generating the certs
Issue2 If cmake error appears such as “command not found: cmake” then
GStreamer ( LGPL )ia a media handling library written in C for applicatioan such as streaming , recording, playback , mixing and editing attributes etc. Even enhnaced applicaiosn such as tsrancoding , media ormat conversion , streaming servers for embeeded devices ( read more about Gstreamer in RPi in my srticle here).
It encompases various codecs, filters and is modular with plugins developement to enhance its capabilities. Media Streaming application developers use it as part of their framework at either the broadcaster’s end or as media player.
Streaming / broadcasting Live Video call to non webrtc supported browsers and media players
attempts of streaming / broadcasting Live Video WebRTC call to non WebRTC supported browsers and media players such as VLC , ffplay , default video player in Linux etc .
continue : Streaming / broadcasting Live Video call to non webrtc supported browsers and media players
httontinuation to the attempts / outcomes and problems in building a WebRTC to RTP media framework that successfully stream / broadcast WebRTC content to non webrtc supported browsers ( safari / IE ) / media players ( VLC )
gstreamer-audio-1.0 GStreamer Audio library – Audio helper functions and base classes
gstreamer-plugins-bad-1.0 GStreamer Bad Plugin libraries – Streaming media framework, bad plugins libraries
gstreamer-rtsp-server-1.0 gst-rtsp-server – GStreamer based RTSP server
At the time of writing this article Gstreamer an much early version in 1.X , which was newer than its then stable version 0.x. Since then the library has updated many fold. summarising release highlights for major versions as the blog was updated over time .
Project : Making and IP survillance system using gstreamer and Janus
To build a turn-key easily deployable surveillance solution
Features :
Paring of Android Mobile with box
Live streaming from Box to Android
Video Recording inside the box
Auto parsing of recorded video around motion detection
Event listeners
2 way audio
Inbuild Media Control Unit
Efficient use of bandwidth
Secure session while live-streaming
Modules
Authentication ( OTP / username- password)
Livestreaming on Opus / vp8
Session Security and keepalives for live-streaming sessions
Sync local videos to cloud storage
Record and playback with timeline and events
Parsing and restructuring video ( transcoding may also be required )
Coturn server for NAT and ICE
Web platform on box ( user interface )+ NoSQL
Web platform on Cloud server ( Admin interface )+ NoSQL
REST APIs for third party add-ons ( Node based )
Android demo app for receiving the live stream and feeds
Varrying experiments and working gstreamer commands
Local Network Stream
To create /dev/video0
modprobe bcm2835-v4l2
To stream on rtspserver using rpicamsrc using h264 parse
Contains code for Android and ios Publishers , players on various platforms including HLS and Flash , streamings servers , Wowza playing mosules , webrtc broadcast
Gstreamer 1.8.0 – 24 March 2016
Features Hardware-accelerated zero-copy video decoding on Android
New video capture source for Android using the android.hardware.Camera API
Windows Media reverse playback support (ASF/WMV/WMA)
tracing system provides support for more sophisticated debugging tools
high-level GstPlayer playback convenience API
Initial support for the new Vulkan API
Improved Opus audio codec support: Support for more than two channels; MPEG-TS demuxer/muxer can handle Opus; sample-accurate encoding/decoding/transmuxing with Ogg, Matroska, ISOBMFF (Quicktime/MP4), and MPEG-TS as container; new codec utility functions for Opus header and caps handling in pbutils library. The Opus encoder/decoder elements were also moved to gst-plugins-base (from -bad), and the opus RTP depayloader/payloader to -good.
Asset proxy support in the GStreamer Editing Services
GStreamer 1.16.0 – 19 April 2019.
GStreamer WebRTC stack gained support for data channels for peer-to-peer communication based on SCTP, BUNDLE support, as well as support for multiple TURN servers.
AV1 video codec support for Matroska and QuickTime/MP4 containers and more configuration options and supported input formats for the AOMedia AV1 encoder
Closed Captions and other Ancillary Data in video
planar (non-interleaved) raw audio
GstVideoAggregator, compositor and OpenGL mixer elements are now in -base
New alternate fields interlace mode where each buffer carries a single field
WebM and Matroska ContentEncryption support in the Matroska demuxer
new WebKit WPE-based web browser source element
Video4Linux: HEVC encoding and decoding, JPEG encoding, and improved dmabuf import/export
Hardware-accelerated Nvidia video decoder gained support for VP8/VP9 decoding, whilst the encoder gained support for H.265/HEVC encoding.
Improvements to the Intel Media SDK based hardware-accelerated video decoder and encoder plugin (msdk): dmabuf import/export for zero-copy integration with other components; VP9 decoding; 10-bit HEVC encoding; video post-processing (vpp) support including deinterlacing; and the video decoder now handles dynamic resolution changes.
ASS/SSA subtitle overlay renderer can now handle multiple subtitles that overlap in time and will show them on screen simultaneously
Meson build feature-complete (with the exception of plugin docs) and it is now the recommended build system on all platforms. The Autotools build is scheduled to be removed in the next cycle.
GStreamer Rust bindings and Rust plugins module
GStreamer Editing Services allows directly playing back serialized edit list with playbin or (uri)decodebin
Market trends are really not in favor of Telecom Service /providers with increasing use of OTT ( Over The Top ) application like watsapp , Facebook messenger , Google hangouts , skype , viber , etc .
OTT ( Over The Top ) Applications
What is an OTT ?
An Over The Top ( OTT ) application is one which provides communication services over Internet . Therefore these bypass the communication billing system setup by a Telecom Operator , resulting in no gain or loss of revenue to Telecom Operator who is providing the Internet service to user in first place .
Hence we see that OTT are major threat and concern for Telecom Operators whose traditional and obviously expensive ( when compared to OTTs free service ) billing models are facing disruption .
Telecom Regulatory bodies around the world
The telecom regulatory authorities in some of the countries are for example listed as :
Canadian Radio-television and Telecommunications Commission (CRTC) – Canada
Ministry of Information Industry (MII) – China
Autorité de Régulation des Communications Électroniques et des Postes (ARCEP) – France
Bundesnetzagentur (BNA) – Germany
Telecom Regulatory Authority of India (TRAI) – India
Ministry for Communications and Informatization of the Russian Federation (Minsvyaz) – Russia
Infocomm Development Authority of Singapore (IDA) – Singapore
Independent Communications Authority of South Africa (ICASA) – south Africa
Federal Communications Commission (FCC) , National Association of Regulatory Utility Commissioners (regulators of individual states) (NARUC) , CTIA – The Wireless Association (CTIA) – USA
Such telecom regulatory bodies get to decide whether to enforce differential price to end consumers for using OTT so that telecom service providers can benefit or keep the Internet fair and open by passing Net Neutrality Laws and Bills and amendments .
what is Net Neaurality ?
The fundamental principle of Net Neurality is that Telecom Operators should not block , slow down or charge consumers extra for using other services as their means of communication. This states that it is wrong to charge users above the regular data rates for using VOIP apps and other internet based communication services.
The following counteries have adopted principles of Net Neutrality by passing bills or making law .
Chile – Chile’s General Law of Telecommunications, “No [ISP] can block, interfere with, discriminate, hinder, nor restrict the right of any Internet user of using, send, receive, or offer any content, application, or legitimate service through the Internet, as well as any activity or legitimate use conducted through the Internet.”
Brazil – ” Internet Bill of Rights ” makes equal access to internet mandatory in Brazil .
Netherlands – Even European Union has adopted Netherlands’ Net Neutrality amendment which reads “traffic should be treated equally, without discrimination, restriction or interference, independent of the sender, receiver, type, content, device, service or application.”
USA – Citizens make ‘We the People’ platform to ‘Restore Net Neutrality By Directing the Federal Communications Commission (FCC) to Classify Internet Providers as ‘Common Carriers‘. Therefore not allowing them to either throttle speed by paid prioritization , discriminate in pricing or block any broadband access to legal content . Above facts are from this tech.firstpost.com article.
Inspite of the fact that I Support Net Neutrality with all my heart , as a telecom engineer I understand the cost investment made by Telecom operators in providing am efficient communication network to its subscribers ( Access , Network and Application layers ). Therefor I do have my sympathies with the Telcos and to level out the wide ranging conflict between Telcos and ISP ( Internet Service Providers ) , I pen down the following points which reflect the Telecom Operators Problems and also highlight the solutions that can be adopted to counteract the OTT threat .
Depleting revenue for Telco
Messaging – OTT messaging cost operators $13.9 billion, or 9% of message revenue in 2013
Voice – Voice services under threat from VOIP services like Skype, Viber
OTT apps – Voice & Message apps have been the operator’s biggest headache. Its time Operator should launch its own OTT Services
Data Traffic – The utilization is yet to reach its peak. Will face challenges from WiFi access
Critical Pain areas – Erosion of Operator’s revenue from voice and (especially) messaging
Telco’s OTT aPPLICATION
At this stage it is crucial for a telecom Service provider / Operator to enter the Apps market and bring forth a Messenger which is more powerful , interactive and awesome than a OTT application. Fortunately the Operator can always couple this application with his background telecom infrastructure to provide the edge in performance and functionalists .
Road block while developing a OTT application for a Telecom Service Provider :
Investment in Data Network is not being utilized due to lack of service
Reuse of Existing business Logic and extending the service reach across devices and networks is tough
Operator already has full fledged network Infrastructure in Place
Desire for minimum CAPEX while investing in new technologies
compete with OTT players and open new revenue streams is a challenge
Next we find the way of solving the problems and integrating them together to form a Solution .
OTT Application for Telecom Service provider
Introduce new services to benefit from investment on Data Plans and Bandwidth
Expose REST API to enable 3trd party Integration with existing network Infrastructure
Partner with individual OTT players to make new services that do not compete on core competencies like billing etc
Use protocols like SIP that reduce CAPEX and have goto market more quickly
Go for enriched service that lead to better user experience
This writeup outlines the process of creating a OTT application for a Telecom Service Provider . Components for the application include cloud Address Book , Video Chatting , Location share , Contact synchronization ,REST based thin client , OS and device agnostic etc shown in the figure below
telco’s OTT app
The Application is designed to close knit with Operator’s own infrastructure hence the crucial entities like Network Address Book , Location Service are synced and fetched from Backend Network .
OTT application Feature Overview
Smart Address Book
Automatic: Get contacts from Gmail, Facebook
Fast search by first, last name, frequently
dialed number
Roadmap: View calendar events
Personal: Get image from Gmail and display in contacts list
Geo Location
Share own location during chatting
Get map for calculating the distance between two chat users
Roadmap : Trigger device (say Switch on/off AC before reaching home) from a threshold distance away from home location
Messaging
Ad-hoc Chat
Session Based Chat
Voice Input for texting
Presence information of contacts
RoadMap: Legacy message integration
Telephony
Voice call to mobile
Voice call to PSTN
Video call to other @imAll user
Share images during voice call to other
Device agnostic
Compatible with IOS, windows
Can run as native app on ipad
Can run as browser client on windows
RoadMap: native app for android, windows phone,blackberry10
Roadmap
To upgrade the application and provide enganced and enrich service support the I propose the following roadmap.
From plain vanilla voice and video calling ( supported by every other OTT application ) our application should progress towards legacy telecom support whihc included PSTN , GSM , ISDN etc . This requires backbone of telecom network and a good setup for media codec conversion to suit various legacy media codecs .
Road Map from Traditional to New age services
Voice and video calling
Legacy services support like MMS and SMS
Integration with 3rd party Vendors
Give new enriched services like Multilingual support , file transfer , screen-sharing etc
give facility to integrated web plugins for web calling
To keep the interest of customers it is essential that the application be supported on other popular OTT services like skype , Gtalk . for exmaple a caller should be able to make call from Skype / Gtalk to our application .Multilingual capabilities, support for larger protocol spectrum will just act like icing on the cake .
How does it benefit the Operator??
Saves on development cost and time
Device Agnostic OTT Applications
Simplified Service deployment
Saves licensing cost per client
Reuses existing Messaging and Address Book service logic.
Open New Revenue Streams for operator
No separate SIP stack required for the client
Faster Time to Market
Update : At the time of writing this post I did not anticipate the wave of change that bring focus on subjects like “net neutrality” , ” Save the internet” and “free internet” . However I see now that I had described this phenomenon way in advance for my time .
With the fast pace of telecom evolution both towards the access network front ( ie GSM , UMTS , 3G , 4G , LTE , VOLTE ) to core network side ( ie application servers , registrar , proxies , gateway , media server etc ) a CSP ( content service provider ) is trying hard to keep up with the user expectation . The user expects a plethora of services , reduced cost and high speed bandwidth . If this was not enough a CSP also has competition OTT ( Over The Top ) Players who provide communication and messaging for FREE .
You can read on how OTT’s players are disruption the revenue streams of traditional telecom operators and how can Telco’s develop their own OTT app , integrated with their backend system to answer to that challenge here – OTT ( Over the Top ) Communication applications
The following points outline the major business challenges faced by telecom operators today .
Technology Evolution Challenges
The increased data speeds and further more increasing hunger for the data overwhelms the existing network infrastructures.
Ensure uniform service experience across the network technologies to check the customer churn.
Access / Radio Technology independent delivery of services.
Enhance Reuse for exiting investments.
Multiple Service Platform Challenges
Typical network constitutes of Multiple Service Platforms increasing network complexity and integration challenges many fold.
Heterogeneous multiple SDP Solutions typically deployed to cater to Multiple Types of Networks/ Standards/Variants
Service Islands makes introduction of seamless services a challenging task for the CSP
Transport Upgrade and Convergence of Wireless Wireline
Retain investments in copper wire systems while migrating towards next generation Fiber Optic systems.
Severe competition among wire-line and wireless operators to provide latest services to retain subscriber base.
Fixed Mobile Convergence leading to a diminishing gap among the revenue shares of various operators in the space, and leading to losses for wire-line only players.
I have been contemplating points that make for a successful developer to develop solutions and services for a Telecom Application Server. The trend has shown many variations from pure IN programs like VPN , Prepaid billing logic to SIP servlets for call parking , call completion. From SIP servlets to JAISNLEE open standard based communication.
Most importatnl things for a OTT provider who acts as a service provider between the SME ( SMall and Medium Enterprises ) and Large scale telco carrier , is to buid Scalable and Flexible platform . Lets go in depth to discuss how can one go about schieving scalibility in SIP platforms .
Multi geography Scaled via Universal Router
A typical semi multi geography scaled , read replica based / data sharding based Distributed VoIP system which is controlled by a router that distributes the tarfffic to various regions based on destination number prefix matching looks like
Cluster SIP telephony Server for High Availiability
Clusters of SIP server are great at provding High availiability and resilience however they also add a factor of lantency and management issues .
considerations for a cluster
memory requirements to store the state for a given session and the increasing overhead of having more than two replicas within a partition.
Co-hosted virtual machine add resource contenstion and delay call established due to multi node traversal .
Additionally incase of node failures or reboots, the traffic redirection needs careful planning and can add complications in network.
System should be reliable to not let a let node failure propagate and become root cause for entire system failure due to corrupted data .
Failure Recovery
A Clustered SIp platform is quickly recoverble with Containerized applications
Clear separation between stateless engine layer and session management or Data layer is crtical to enable auto reboot of failed nodes in engine layer .
It should be noted that unlike HTTP based platforms , dialog and transaction state varaibles are crtical to SIP platfroms for exmaple , call duration for CDR entry . Therefore for a mid call failure and auto reboot
Multi-tier cluster architecture
Symmetrical Multi-Processing (SMP) architectures have
stateless “Engine Tier” processes all traffic and
distributes all transaction and session state to a “Data Tier.”
A very good exmaple of this is Oracle Communications Converged Application Server Cluster (OCCAS) which is composed of 3 tiers
Message dipatcher , Communication engine stateless and last Datastore which is in-memory session store for the dialogs and ongoing transactions
An advantage of having statless servers is that is the application server crashes or reboots , the session sattes is not lost as new server can pick up the session ifnromation from exgternal session store .
Role Abstraction / Micro-Service based architecture
The compoenets for a well performing highly scalable SIP arachitecture are abstracted in their role and reponsibilities . We can have catagories like
Load Balancer / Message Dispatcher
routes tarffic based on algorithm (round robin , hasing , prioroity based scejduling , weight based scheduling ) among active and ready servers
Backend Dynamic Routing and REST API services
Services which the Aplication server calls during its callflow excution which may include tasks like IP address associated with caller , screened numbers associated with destination etc such as XML Remote Procedure Call (XML-RPC) or AVAPI Service in kamailio
OSS/BSS layer
This layer is reponsible for jobs relation to operations and billing and should take place in indpendant system without affacting the session call flow or causing a high RTT .
POS CRM ,Order Management , Loyality , feedback , ticketing Post Paid Billing , Inter-carrier Billing BPM and EAI Provisioning & Mediation Number Management Inventory ERP, SCM Commissions Directory Enquiry Payments & Collections BI Fraud and RAS Pre-Paid Billing Document Management EBPP, Self Care
There are other componets ina typical VoIP micro services architecture such as Heartbeat service , backend accounting servuce , security check service, REST API service , synmaic routing service , event notofication service etc which should be decoupled from each other leading to high parallel programing approach.
Distributed Event management and Event Driven architecture
Distributed event management , monitoring and working on Data stream instead of stored Database
Distributed Messaging using Data streaming instead of static stored database data
Containerization
To improve Flexibility w.r.t Infrastructure binding ,, all server compoenets including edge compoenets , proxies , enginies , emdia server must be containerized in form of images or docker for easy deployment via an infracstructure tool like kubernetics , terraform , chef cookbooks and be efficently controleed with an Identify manage tool and CICD ( continous integartion and Delivery ) tool like Travis or jenkins
Autoscalling Cloud Servers
Autoscalled server are provided by majority of Cloud Infrastrcture provicderd such as AWS ( Amazon Web Services ), Google Cloud platform which scale the capacitty based on traffic in realtime also called elasticity. Any VoIP developer would notice patterns in voice traffic such as less during holidays/night hours where servers can be freeed, whereas taffic peaks during days where server capacity needs to scale up.
Additionally traffic may pike when the setup is under DDos attacks , not an uncommon thing for SIP server , then the server need to identify and block malacious source points and prevent unnecessary up scaling .
There are 2 approaches to scaling
Scale UP / Vertical Scaling
Resusing the existing server to upgrade performance to match the load requirnments
Scale OUT / Horizontal scaling
Increasing the number of servers and adding their IP to Load balancer to manage traffic .
It should be noted that scalling up or down shouel be carried out incrementally to have better control on resource to requirnment ratio.
Other points points here that make for a successful startup in logic building domain of telecom core network .
Security
It is crucial for any Voice traffic / media servcis provoder to have state of the art security in the content without disrupting data privacy norms.
SIP secure practises like Authentication , authorization ,Impersonating a Server , Temparing Message bodies , mid-session threats like tearing down session , Denial of Service and Amplification , Full encryption vs hop by hop encrption , Transport and Network Layer Security , HTTP Authentication , SIP URI, nonce and SIP over TLS flows , can be read at https://telecom.altanai.com/2020/04/12/sip-security/
While scaling out the infrastructure for extensing the Pop( point of presence ) accross the differnet geographies , define zones such as
red zone : public facing server like load balancers
dmz zone ( demilitarized zone ) interfacing servers betwee private and public network
green zone : provate and secure interal serer which communicate over private IPs snd should ne unrechable from outside .
To futher increase efficiency between communication and transmission between green zone server , setup private VPC ( Virtual provate cloud ) between them .
Follow Open standards and Data Privacy
To establish itself as a dependable Realtime communication provider , the product must follow stabdardised RFC’s and stacks such as SIP RFC 3261 and W3C drfat for Webrtc peer connection etc . It si also a good practise to be updated with all recommendation by ITU and IANA and keep with the implementation . For exmaple : STIR/SHAKEN –https://telecom.altanai.com/2020/01/08/cli-ncli-and-stir-shaken/
In a crowded market of many SIP Service providers and platforms
Envisions a multiple network technologies, that provides ability to build over new innovative cutting edge technologies in the market. It should deliver platform to launch newer services like WebRTC and RCS .
Innovation + Experiment + Oyt of Box Thinking
As a market differentiator following tools are advised
Easy to follow technical documentationand help and quick response to any technical question about platform posted on QnA sites (stackoverflow , Quora .. ) , tech forums ( Google groups , slack channels .. ) or else where ( facebook , twitter .. )
Graphical Event Timelines – time based events such as call setup , termination , codec negotiation , call rediection events
Drag and Drop Call Flow deisgner – As call routing logic beome more complicated with a large set of known and pre-defined operations ( parking , routing , voicemail , forking , rediercting etc) . The call routing can be easily composed from these preset operation as UI block attached to a call flow chain which results in calls being channels as predefined by this call flow logic . Leads to plenty of cutomaizibility and design flexibility to custoemrs to design their calls .
Competitive Pricing with Low or No Servicing cost
Cutting down the spiraling cost of Development of the new technologies platform with improvement in the usage of Data rather than voice by integrating new features like File sharing and MSRP messaging. An evolutionary architecture to reduce the effort and cost through high re-use of NGN Platform and Services.
Use Opensource Products
Introduce uniform service experience across different platforms which helps CSP’s to reduce Time Cycles and Costs for handling enhancements requests and the annual OPEX appreciably.
“Pay as you go ” Pricing model
Services which should be offered on a non chargable basis :
Round the clock technical support
Compensation for Downtime
CDRs per account
IP to IP calls
Security Certificates in TLS and SRTP calls
Autheticationa nd Authorization secure practises
Services that can be charges are Value added services
CarrierIntegration – trunk , PRI
Toll Free Numbers – DID numbers
Virtual Private Network (VPN) : An Intelligent Network (IN) service, which offers the functions of a private telephone network. The basic idea behind this service is that business customers are offered the benefits of a (physical) private network, but spared from owning and maintaining it
Access Screening(ASC): An IN service, which gives the operators the possibility to screen (allow/barring) the incoming traffic and decide the call routing, especially when the subscribers choose an alternate route/carrier/access network (also called Equal Access) for long distance calls on a call by call basis or pre-selected.
Number Portability(NP) : An IN service allows subscribers to retain their subscriber number while changing their service provider, location, equipment or type of subscribed telephony service. Both geographic numbers and non-geographic numbers are supported by the NP service.
Flexibility for inter-working
Interworking among the services from legacy IN solution and IMS /IT. Allow the Operators to extend their basic offering with added services via low cost software and increases the ARPU for subscribers.
Next Gen 911
911 like emrgency services afre moving from tradiotional TDM networks to IP networks . However this poses some challenges such as detecting callers geolocation and routing the call to his/her nearest servicing station pr Public safety Answering Point ( PSAP)
Backward compatibility with existing legacy networks
PSTN-SIP gateways to interface bwteen SIP platform and SS7 siganlling platform also convert the RTP stream to Analog waveforms required byb PSTN endpoints
Internetworking with IMS
IMS is a IP telephony service archietcture developed by 3rd Generation Partnership Project ( 3GPP) ,global cellular network standards organization that also standardized Third Generation (3G) services and Long Term Evolution (LTE) services
Develop on Interactive and populator frameworks like webRTC
Agile Development and Service Priented Architecture (SOA) are proven methods of delievry quality and updated products and releases which can cater to eveolcing market demands . In short “Be Future ready while protecting the existing investments”
Make a WebRTC solution that offers a plug in free, device agnostic, network agnostic web based communication tool along with the server side implementation.
Log aggregation and Analytics. PagerDuty Alerts Daily and Weekly backups and VM snapshots. Automated sanity Tests Centralized alert management, monitoring and admin dashboards . Deployment automation / CICD Tools and workflows for diagnostics, software upgrades, OS patches etc. Customer support portal , provisioning Web Application
Media Stats can help us collect the call qulaity metrics which determins the overall USer experience . Some frequently encountered issues include
Issue
Cause
Observance
High Packet Loss
250 ms of audio suration lost in 5 sec
broken audio
High Jitter
jitter >= 30 ms in 5 sec
robotic audio
Low Audio Level
audio level < -80dB
inaudible
High RTT
RTT > 300 ms in 5 sec
lags
Pro-active Call Analysis
Call details even during a setup phase , continuation or reinvite /update phase can suggest the probably outcomes based on previous results such as bad call quality from certain geographic areas due to their known network or firewall isseus or high packet loss from certain handset device types . We can deduce well in advance what call quality stats will be generated from such calls .
Contains which can be identfied from calls setup details itself include :
geography and number – Call was made from which orignating location to which destination
SIP devices – device related details , Version of device (browser version etc..,)
Chronological aspects of call – Initiation, ring start, pick up and end time.
call direction – inbound ( coming from carrier towards our VoIP platform ) or outbound ( call directed to carrier from out VoIP platform )
Network type – network ssues and quality score across network type
Contarins which can be identfied during a ongoing call itself include :
Participants and their local time – ongoing RTCP from Legs, probability of long Conferences is low in off hours
Call events – DTMF, XML, API calls , quality issues
The minor issues identified during an ongoing calls RTCP packets such as increasing jitter or packet loss can extrapolate to human perceivable bad audio quality of call after a while . Thus any suspected issues should be identified as early as traced and corrective action should be put in place .
Predicting Low Audio / Call quality
Having a predictive engine can forecast bad call Quality such as 408 timeouts , high RTT , low audio level , Audio lag , one way audio , MOS < 2.5 out of 5 etc .
The predictive engine can use targeted notifications pointing towards specific issues that can comeup in a call relatine and assign a technical rep to overlook or manually intervene . This can include scenario such as an agent warning a customer that his bad audio quality is due to him using an outdated SIP Device with slow codecs and suggest to upgrade it to lightweight codecs as per his bandwidth. This saves bad user experince of the customer and can happen without cusomer reporting the issues homself with feedback , RTP stats , PCAPS etc. Save a lot of trouble and effort in call debugging .
Social Media Platform Integration such as Skype for Business , Slack , WebEx
Integration of the services with social media/networking enables new monetizing benefits to CSPs especially in terms on advertising and gaining popularity , inviting new customers etc.
Enterprises are looking forward to reach customers with ennoblement of Telco in their present landscape which was impossible to reach before. Telco not only plays an instrumental role in increasing the customers base which results into increase in enterprise’s revenue but also offers the value addition in their present product/service delivery model. Hence it is high-time when developers can aggregate , use open-standard services / technologies ( GSMA , SIP , WebRTC ) and develop high end solutions for Telecom Domain .
Effienet Media Management – Media Streaming , conferencing , Recording and playback
CSP’s are looking into Long term growth and profitability from new online services media streaming services . Make use-cases around IPTV and VOD ( Video On Demand) . Also Voicemails , IVR , DTMF, TTS( text to speech ) , Speech recognition etc