VoIP manages Call setup and teardown using IP protocol. The APIs can be used to provide public or internal endpoinst to create mnage calls , conference addon services like recording , tgranscription or even do auth and heartbeat. This article lists some external programmable Call Control APIs, internal APIs for biling , health as well as Rate limitting.
get CDR ( filtered per cal or acc to specific date or account)
bulk export of CDR
Internal API gateways
API Rate Limiter
Noisy neighbour is when one of the clients monoplizes the bandwidth using most of the i/o or cpu or other resources which can negatively affect the performance for other users . Throttling is a good way to solve this problem by limit.
Auto scaling
Load balancer
Rate Limiter
horizotal or vertical scalling can countger incoming traffic
LB can limit number of simultaneous requests. It can reject or send to queue for later operation
Can intelligently understand the cost of each operation and perform throttling.
(-) takes time to scale out thus cannot solve noisy neighbour problem immediately
(-) but the LB’s behaviour is indiscriminate ( cannot distinguish between the cost of diff operations) (-) LB cannot ensure uniform distribution of distribution of operations among all servers.
A rate limiter should have low latency, accurate and scalable.
RateLimiter inside the serviceprocess
Rate Limiter as its own process outside as a daemon
(+) faster , no IPC (+) reisstnt to interprocess call failures
(+) programming langiage agnostic daemon (+) uses its own memory space, more predictable
(-) service meory needs to allocate space for rate limiters
widely used for auto discovery of service host
Token based Rate Limiting
provides admission contro
Token bucket filter
define a users quota in terms average rate and burst capacity
Hierarchical Token Bucket ( HTB)
uses the deficit round-robin algorithm for fair queuing
Fair Queing
give paying users a bandwidth fraction of 25%
priority queuing
decide 1 packet/ms for free or reduce rate user
distributes that sender’s bandwidth among the other senders
CBQ ( Class Based Queing)
Shaping is performed using link idle time calculations based on the timing of dequeue events and underlying link bandwidth. Input classes that tried to send too much were restricted, unless the node was permitted to “borrow” bandwidth from a sibling.
Modular QoS command-Line interface (MQC) Shaping
mplement traffic shaping for a specific type of traffic using a traffic policy
When the rate of packets matching the specified traffic classifier exceeds the rate limit, the device buffers the excess packets.
When there are sufficient tokens in the token bucket, the device forwards the buffered packets at an even rate.
When the buffer queue is full, the device discards the buffered packets.
Throttling
delay the packet until the bucket is ready / shaping
drop the packet / Policing
mark the packet as non-compliant
Failure management on Rate Limiter
Node Crash : just less requests trolled
Leaky bucket
tokens can go into -ve
System Design for API gateway
Important points for design API gateway
Serialize data in company binary format
allocate buffer in memory and build frequency count hash table and flash once full or based on time to calculate counters
aggregation on API gateway on the fly
Frontend Service
Partitioned Service
Backend Service
Lightweight web service Stateless Request Validation Auth / Authorization TLS(SSL ) termination Server sode encryption Caching Rate Limiting(throttling) Request deduplication
Caching layer between frontend and backend
Replication Leader Selection + Quorem
Distributed messaging system( fast and slow paths) for API
A distributed messahing system such as Apache kafka or AWs kinesis, internally splits a msg accross serveral partitions where each parition can be placed on a single shard in a seprate machine on a clustered system.
Load Balancer(LB) is the initial point of interaction between the client application and the core system. It is pivotal in the distribution of the load across multiple servers and ensuring the client is connected to the nearest VoIP/SIP application server to minimize latency. However, the load balancers are also susceptible to security breaches and DOS attacks as they have a public-facing interface. This section lists the protocols, types and algorithms used popularly in Load balancers of VoIP systems.
used by applications in cloud ADN (Application delivery network)
used by network address translators (NATs) DNS load balancing
examples and roles of software and hardware based load balancers
Load Balancers(LB) ping each server for health status and greylists servers that are unhealthy( respond late) as they may be overloaded or experiencing congestion. The LB monitors it rechecks after a while and if a server is healthy ( ie if a server responds with responds with status update) it can resume sending traffic to it. LB should also be distributed to different data centres in primary-secondary setup for HA.
Networking protocol
TCP Loadbalancer
HTTP load balancers
SIP based LB as Kamailio/ Opensips
can forwrad the packet without inspecting the content of the packet.
terminate the connection and look inside the request to make a load balcing decsiion for exmaple by using a cookie or a header.
domain specific to VoIP
(+) fast, can handle million of req per second
(+) handle SIP routing based on SIP headers and prevent flooding atacks and other malicious malformed packets from reaching application server
Load balancing algorithms
Weighted Scheduling Algorithm
Round Robin Algorithm
Least Connection First Scheduling
Lest response time algorithms
Hash based algorithm ( send req based on hashed value such as suing IP address of request URL)
Loadbalancer
Reverse Proxy
forward proxy server allows multiple clients to route traffic to an external server
accepts clients requestd for server and also returns the server’s response to the client ie routes traffic on behalf of multiple servers.
Balances load and incoming traffic endpoint
public facing endpoint for outgoing traffic additional level of abstraction and security, compression
used in SBC (session border controllers) and gateways
Client-side or even backend service discovery uses a broadcasting or heartbeat mechanism to keep track of active servers and deactivates unresponsive or failed servers. This process of maintaining active servers helps in faster connection time. Some approaches to Service Discovery
Mesh
(-) exponentially incresing network traffic
Gossip
Distributed cache
Coordination service with Service
(-) requesres coordination service for leader selection
Systems such as Consul, Etcd, and Zookeeper can help services find each other by keeping track of registered names, addresses, and ports. Health checks help verify service integrity and are often done using an HTTP endpoint.
Partition strategy can be based on various ways such as :-
Name based partition
geographic partition
names’ hashed value based on identifier
(-) can lead to hot partitions ( high density in areas of freq accessible identioers )
(-) high density spots for example all messages with a null key to go to the same partition
(-) doesnt scale
event time based hash
(+) data is spread evenly over time
To create a well distributed partition we could spread hot partition into 2 partitions or dedicate partitions for freq accessible items. An effective partitioning keys uses
Cardinality : total num of unique keys for a usecase. High cardinality leads to better distribution.
high cardinatility keys : names , email address , url since they have high variatioln
low cardinatlity keys : boolean flags such as gender M/F
Selectivity : number of message with each key. High selectivity leads to hotspots and hence low selectivity is better for even distribution.
Kubernetes is an open-source container-orchestration system for automating computer application deployment, scaling, and management used in DevOps. I have mentioned this in detail on the article on VoIP and DevOps below.
for a VOIP system catering to many clients accross the globe or accessing multiple carriers meant for different counteries based on Prefix matching , there should be alocal PoP in most used regions . typically these regions include – US east – west coasts, UK – germany of London , Asia Pacific – Mumbai ,Hong Kong and Australia.
Minimal Latency and lowest amount of tarffic via public internet
Creating multiple POPs and enabling private traffic via VPN in between them ensures that we use the backbone of our cloud provider such as AWS or datacentre instead of traversing via public internet which is slower and more insecure. Hopping on a private interface between the cloud server and maintaining a private connection and keepalive between them helps optimize the traffic flow while keeping the RTT and latency low.
A high-availability (HA) architecture implies Dependability.Usually via existence of redundant applications servers for backups: a primary and a standby. These applications are configured so that if primary fails, the other can take over its operations without significant loss of data or impact to business operations.
Downtime / SLA of 5 9’s in aggregate failures
4 9’s of availiability on each service components gives a downtime of 53 mins per service each year. However in aggregate failure this could amlount to (99.99)10 = 99.9 downtime which is 8-10 hours each year.
Thus, aggregate failure should be taken into consideration while designing reliable systems.
HA for Proxy / Load balancer (LB)
A LB is the first point of contact for outbound calls and usually does not save the dialogue information into memory or database but still contain the transaction information in memory. In case the LB crashes and has to restart, it should
have a quick uptime
be able to handle in dialogue requests
handle new incoming dialogue requests in a stateless manner
verify auth/authorization details from requests even after restart
HA for Call Control app server
App server is where all the business logic for call flow management resides and it maintains the dialog information in memory.
Issues with in-memory call states : If the VM or server hosting the call control app server is down or disconnected, then live calls are affected, this, in turn, causes revenue loss. Primarily since the state variable holding the call duration would be able to pass onto the CDR/ billing service upon the termination of the call. For long-distance, multi telco endpoint calls running hours this could be a significant loss.
Standby app server configurationand shared memory : If the primary app server crashes the standby app server should be ready to take its place and reads the dialog states from the shared memory.
Live load balanced secondary app server + external cache for state varaibles : External cache for state variables: a cluster of master-slave caches like Redis is a good way of maintaining the dialogue state and reading from it once the app server recovers from a failed state or when a secondary server figures it has a missing variable in local memory.
Media Server HA
Assuming the kamailio-RTPengine duo as App server and Media Server. These components can reside in same or different VMs. Incase of media server crash, during the process of restoring restarted RTpengine or assigning a secondary backup RTpengine , it should load the state of all live calls without dropping any and causing loss of revenue . This is achived by
external cache such as Redis ,
quick switchover from primary to secondary/fallback media server and
floating IPs for media servers that ensures call continuity inspite of failure on active media server.
Architecturally it looks the same as fig above on HA for the SIP app server.
Attacks and security compromisation pose a very signficant threat to a VoIP platform.
MITM attacks
Man in midddle attacks can be counetred by
End to end encryption of media using SRTP and signals using TLS
Strong SIP auth mechanism using challenges and creds where password is composed of mixed alphanumeric charecters and atleast 12 digits long
Authorization / whitelisting based on IP which adheres to CIDR notation
DDOS attacks
DDOS renders a particular network element unavailable, usually by directing an excessive amount of network traffic at its interfaces.
dDOS – multiple network hosts to flood a target host with a large amount of network traffic. Can be created by sending falsified sip requests to other parties such that numerous transactions originating in the backwards direction comes to the target server created congestion.
Can be counetred by
detect flooding and q in traffic and use Fail2ban to block
challenge questionable requests with only a single 401 (Unauthorized) or 407 (Proxy Authentication Required)
Raise Event notification alerts to designated developers for any anolous behavior. It could be call based or SMS basef alert based on the sevirity of the situtaion .
Sources for alert manager
Build failed ( code crashes, Jenkins error)
Deployment failed ( from Kubernetes , codechef, docker ..)
configuration errors ( setting VPN etc )
Server logs
Server health
homer alerts ( SIP calls responses 4xx,5xx,6xx)
PCAP alerts ( Malformed SIP SDP ..)
Internal Smoke test ( auto testing procedure done routinely to check live systems )
Support tickets from customer complaints ( treat these as high priority since they are directly impacting customers)
A QA framework basically validates the services and callflows on staging envrionment before pushing changes to production. Any architectural changes should especially be validated throughly on staginng QA framework befire making the cut. The qualities of an efficient QA platform are :
Genric nature – QA framework should be adatable to different envrionments such as dev , staging , prod
Containerized – it should be easy to spn the QA env to do large scale or small scale testing and hence it should be dockerized
CICD Integration and Automation – integrate the testcases tightly with gt post push and pull request creation . Minimal Latency and lowest amount of tarffic via public internet
Keep as less external dependecies as possible for exmaple a telecom carrier can be simulated by using an PBX like freeswitch or asterix
AsynchronousRun – Test cases should be able to run asynchronously. Such as seprate sipp xml script for reach usecase
Sample Testcases for VoIP
Authentication before establish a session
Balance and account check before establishing a session like whitelisting , blacklisting , restricted permission in a particular geography
Transport security and adaptibility checks , TLS , UDP , TCP
codec support validation
DTMF and detection
Cross checking CDR values with actual call initiator and terminator party
Inidividual Events ( like every click or every call metric)
Aggregate Data ( clicks per minute, outgoing calls per minute)
(+) fast write (+) can customize/ recalculate data from raw
(+) faster reads (+) data is fready for decision making / statistics
(-) slow reads (-) costlier for large scale implementations ( many events )
(-) can only query in the data as was aggregates ( no raw ) (-) requires data aggregation pipeline (-) hard to fix errors
suitable for realtime / data on fly low expected data delay ( minutes )
suitable for batch processing in background where delay is acceptable from mintes to hours
Push vs Pull Architecture
Push : A processing server manages state of varaible in memory and pushes them to data store.
(-) crashed processingserver means all data is lost
Pull : A temporary data strcyture such as a queue manages the stream of data and processing service pull from it to process before pusging to data stoore.
(+) a crashed server has to effect on temporarily queue held data and new server can simply take on where previous processing server left.
(+) can use checkpointing
Popular DB storage technologies
SQL
NoSQL
Structured and Strict schema Relational data with joins
Semi-structured data Dynamic or flexible schema
(+) faster lookup by index
(-) data intensive workload (+) high throughput for IOPS (Input/output operations per second )
used for Account information transactions
best suitable for Rapid ingest of clickstream and log data Leaderboard or scoring data Metadata/lookup tables
DynamoDB – Document-oriented database from Amazon MongoDB – Document-oriented database
A NoSQL databse can be of type
Quorem
Document
Key value
Graph
Cassandra is wide column supports asyn master less replication
Hinge base also a quorem based db also has master based preplication
MongoDB documente orientd DB used leacder based replication
SQL scaling patterns include:
Federation/ federated database system : transparently maps multiple autonomous database systems into a single virtual/federated database.
(-) slow since it access multiple data storages to get the value
Sharding / horizontal partition
Denormalization : Even though normalization is more memory efficient denormalization can enhance read performance by additing redundant pre computed data in db or grouping related data.
Normalizing data reduces data warehouse disk space by reducing data duplication and dimension cardinality. In its full definition, normalization is the process of discarding repeating groups, minimizing redundancy, eliminating composite keys for partial dependency and separating non-key attributes.
SQL Tuning : “iterative process of improving SQL statement performance to meet specific, measurable, and achievable goals”
Distributed event management, monitoring and working on incoming realtime data instead of stored Database is the preferred way to churn realtime analysis and updates. The multiple ways to handle incoming data are
Batch processing – has lags to produce results, not time crtical
Data stream – realtime response
Message Queues – ensures timely sequence and order
Buffering
Batching
Add events to buffer that can be read
Add events to batch and send when batch is full
(+) can handle each event
(+) cost effective (+) ensures throughput (-) if some events in batch fail should whole batch fail ? (-) not suited for real time processing
S3 like objects storage + Hadoop Mapreduce for processing
Timeout
Connection timeout : use latency percentiles to calculate this
Request timeout
Retries
exponential backoff : increase waiting time each try
jitter : adds rabdomness to retry intervals to spread out the load.
Grouping events into object storage and Message Brokers
slower than stream processing but faster than batch processing.
In event driven archietcture a produce components performs and action which creates an event thata consumer/listener would subscribes to consume.
(+) time sensitive
(+)Asynch
(+) Decoupled
(+) Easy scaling and Elasticity
(+) Heterogeneous
(+) contginious
Expanding the stream pipeline
Event Streams decouple the source and sink applications. The event source and event sinks (such as webhooks) can asynchronously communicate with each other through events.
Options for stream processing architectures
Apache Kafka
Apache Spark
Amazon kinesis
Google Cloud Data Flow
Spring Cloud Data Flow
Here is a post from earlier which discusses – Scalable and Flexible SIP platform building, Multi geography Scaled via Universal Router, Cluster SIP telephony Server for High Availability, Failure Recovery, Multi-tier cluster architecture, Role Abstraction / Micro-Service based architecture, Load Balancer / Message Dispatcher, Back end Dynamic Routing and REST API services, Containerization and Auto Deployment, Auto scaling CloudServersusing containerized images.
Stream processing on top of map reduce and stream processing engine. In lambda architecture we can send events to batch system and stream processing system in parallel. The results are stiched together at query time.
Lambda Archietcture : stream processing on top of map reduce and stream processing engine. Send events to batch system and stream processing system in parallel. The results are stiched together at query time.
Apache Kafka is used as source which is a framework implementation of a software bus using stream-processing. “.. high-throughput, low-latency platform for handling real-time data feeds”.
Apache Spark : Data partitioning and in memory aggregation.
Isolates cache fro service Cache and service do not share memory and CPU can scale independently can be used by many microservices flexibility in choosing hardware
doesnt require seprate hardware low operational and hardware cost scales together with the service
EEP duplicates and IP datagram and encapsulates and sends for remote relatime monitoring for SIP specific alerts and notifications . HEP is popular among many SIP servers including Freeswitch , Opensips, Kamailio, RTP engine as an external module .
intended for passive duplicated for remote collection
can be used for audit storage and analysis
does not alter the orignal datagram or headers
HOMER is Packet and Event capture system popular fpr VOIP/RTC Monitoring based on HEP/EEP (Extensible Encapsulation protocol)
SIP Server Integration
Homer and homer encapsulation protocl (HEP) integration with sip server brings the capabilities to SIP/SDP payload retention with precise timestamping better monitor and detect anomilies in call tarffic and events correlation of session ,logs , reports also the power to bring charts and statictics for SIP and RTP/RTCP packets etc. We read about sipcapture and sip trace modules in project sipcapture_siptrace_hep.
Both Kamailio and Opensips HEP Integration are structurally simmilar. In kamailio SIPCAPTURE [2] module enables support for –
● Monitoring/mirroring port ● IPIP encapsulation (ETHHDR+IPHDR+IPHDR+UDPHDR) ● HEP encapsulation protocol mode (HEP v1, v2, v3)
seting db_mode – synchronisation of dialog information from memory to an underlying database has following options 0 – NO_DB – the memory content is not flushed into DB; 1 – REALTIME – any dialog information changes will be reflected into the database immediately. 2 – DELAYED – the dialog information changes will be flushed into DB periodically, based on a timer routine. 3 – SHUTDOWN – the dialog information will be flushed into DB only at shutdown – no runtime updates.
note :
use the same hash_size while using diff kamailio to restore dialogs
database table for dialogue
install mysql
define root ( with db create permissions ) and user ( with database read wrote ) permission in kamctlrc
vi /usr/local/etc/kamailio/kamctlrc
Dialogue table schema *
name type size default null key extra attributes description id unsigned int 10 no primary autoincrement unique ID hash_entry unsigned int 10 no Number of the hash entry in the dialog hash table hash_id unsigned int 10 no The ID on the hash entry callid string 255 no Call-ID of the dialog from_uri string 128 no URI of the FROM header (as per INVITE) from_tag string 64 no identify a dialog, which is the combination of the Call-ID along with two tags, one from participant in the dialog. to_uri string 128 no URI of the TO header (as per INVITE) to_tag string 64 no identify a dialog, which is the combination of the Call-ID along with two tags, one from participant in the dialog. caller_cseq string 20 no Last Cseq number on the caller side. callee_cseq string 20 no Last Cseq number on the caller side. caller_route_set string 512 yes Route set on the caller side. callee_route_set string 512 yes Route set on on the caller side. caller_contact string 128 no Caller's contact uri. callee_contact string 128 no Callee's contact uri. caller_sock string 64 no Local socket used to communicate with caller callee_sock string 64 no Local socket used to communicate with callee state unsigned int 10 no The state of the dialog. start_time unsigned int 10 no The timestamp (unix time) when the dialog was confirmed. timeout unsigned int 10 0 no The timestamp (unix time) when the dialog will expire. sflags unsigned int 10 0 no The flags to set for dialog and accesible from config file. iflags unsigned int 10 0 no The internal flags for dialog. toroute_name string 32 yes The name of route to be executed at dialog timeout. req_uri string 128 no The URI of initial request in dialog xdata string 512 yes Extra data associated to the dialog (e.g., serialized profiles).
Siptrace module
SIPtrace module offer a possibility to store incoming and outgoing SIP messages in a database and/or duplicate to the capturing server (using HEP, the Homer encapsulation protocol, or plain SIP mode).
integrating iut with request route to start duplicating the sip messages
sip_trace();
setflag(22);
trace_mode * 1 – uses core events triggered when receiving or sending SIP traffic to mirror traffic to a SIP capture server using HEP 0 – no automatic mirroring of SIP traffic via HEP.
duplicate
address in form of a SIP URI where to send a duplicate of traced message. It uses UDP all the time.
New OSS Capture-Agent framework with capture suitable for SIP, XMPP and more. With internal method filtering , encryption and authetication this does look very promising howevr since I have perosnally not tried it yet , I will leave this space TBD for future
Multi-Protocol HEP Server & Switch in NodeJS. stand-alone HEP Capture Server designed for HOMER7 capable of emitting indexed datasets and tagged timeseries to multiple backends
PCAP monitoring -> Homer Server -> Notification and Fraud Prevention
A realtime monitoring and alerting setup fom homer can best safeguard on VoIP specific attacks and suspecious activity by early warning . Some list of attacks such as DDOS , SIP SQL injections , parser , remote manipulation hijacking as cell as resource enumeration are common ifor a cloud telephony provider.
Adiitionally homer provide session quality using varables that include [1]
SD = Session Defects [SUM(500,503,504)]
ISA = Ineffective Session Attempts [SUM(408,500,503)]
AHR = Average HOP Requests
ASR = Answer Seizure Ratio [(‘200’ / (INVITES – AUTH – SUM(3XX))) * 100]
NER = Network Efficiency Ratio [(‘200’ + (‘486′,’487′,’603’) / (INVITES -AUTH-(SUM(30x)) * 100]
HOMER Web Interface or Custom Dashboard
Some more visualization for inter team communication such as NOC team can include
With the dawn of IP telephony service and cloud communication platforms in recent years, the SIP has caught the attention of many application developers. while SIP is essentially a session management multimedia signalling protocol its generic stack can be used for various use cases from IoT camera streaming sessions to call centres even auto calling for purpose of sharing OTP(one-time password) etc. In this I will highlight the usecase of large calltraffic and the use of SIP trunks.
SIP based trunking can provide significant cost savings and business process improvements by supporting the native SIP protocol that controls the VoIP systems used in call centres and business communication platforms.
(+) unified communication
(+) lower telco network
(+) streamline operations for multicountry/ geography
In the past, telephone systems used trunk lines to connect different parts of the network. Trunk lines were long-distance communication lines that connected telephone exchanges in different locations. Trunk calls were calls made over these trunk lines. They were typically used for long-distance communication, as they allowed calls to be made between exchanges that were geographically far apart. Trunk calls were generally more expensive than local calls, as they involved the use of long-distance communication lines.
Traditional trunk calls operated like a circuit with local loops , trunk lines and switching offices. The telco acted as carriers that sell of lease communication lines to facilitate communication over long distances using local exchanges and interexchange carriers.
In the early days of telephone systems, trunk lines were typically made of copper wires or cables. Later, trunk lines were replaced with satellite links and fiber optic cables, which provided higher capacity and faster transmission speeds. Today, with the widespread adoption of VoIP (Voice over Internet Protocol) technology, many telephone systems no longer use trunk lines in the traditional sense. Instead, they use virtual connections, such as SIP trunks (Session Initiation Protocol trunks), which allow organizations to make and receive phone calls over the internet. SIP trunks are generally more flexible and cost-effective than traditional trunk lines, and do not require the installation of additional hardware.
Voice trunk Lines in SS7 based Next Generations IN networks used media gate ways and MGCP, H323 protocols
SIP is a protocol that is commonly used in VoIP (Voice over Internet Protocol) systems to set up, modify, and terminate sessions that involve the exchange of audio, video, and other media. SIP Trunks are virtual voice channels (or paths) which deliver media (voice, video, IM) over an IP network to a designated endpoint. SIP Trunks can be thought of as a virtual line or concurrent call path. SIP Trunks are delivered over an IP connection like Tier One Carrier or Voice Optimized Recommended or UDP. SIP Trunk may be over-subscribed ie can have more numbers than trunks for example G.711 – 17 calls over T1 or G.729a – 45 calls over T1. SIP Trunking can be provided as one-way or two-way lines. Direct Inward Dialing (DIDs) can be used for toll-free number service.
Centralized SIP Trunk Model is designed to aggregate all calls from all sites and funnels them into a single entry point. Each site has its own SIP trunk termination of the appropriate capacity for calls to and from that site.
Such SIP trunks models offer benefits in three significant areas:
Cost savings, arising from many factors including reduced telecommunications network charges and streamlined operations.
Unified communications, where voice, video, email, text and other messaging technologies are combined to provide greater flexibility for users by enabling new ways to transfer information and manage connectivity. Many SIP trunk providers offer advanced features such as call forwarding, call waiting, and voicemail, which can improve the overall communication experience for employees.
Business Continuity and Disaster Recovery, where the right physical configuration in conjunction with intelligence in the network can be leveraged to provide uninterrupted communications and alternative means to stay connected for employees in the event of system bottlenecks or failures.
SIP Trunking is a low-cost IP-based alternative to ISDN offering for medium to large businesses needing upwards of several tens of channels in a trunk, often across multiple sites, with IP VPN access.
(+) Optimal utilization of bandwidth by delivering both data and voice in the same bandwidth
A telephony company such as a telecom service provider may expose SIP trunks as a means of connecting inbound or outbound calls through its telecom network. For the integrator ( or the service provider managing the other enedpoint of the call leg ) it can be no different that a traditional phone call.The SIP signalling however is useful for enabling better session understaning using standard SIP requests and responses as compared to SS7 or PRI lines.
•Cost analysis •Assess traffic volumes and patterns •Assess network design implications •Emergency call policy •Define production user community phases •Define user community to pilot •Evaluate future new services •Assess security precautions
The steps to set up a SIP trunk connection may vary depending on the specific provider and the equipment being used. However, here are some general steps that are often involved in the process:
Choose a SIP trunk provider: Research and compare different SIP trunk providers to find one that meets your organization’s needs and budget.
Sign up for a SIP trunk account: Follow the provider’s instructions to sign up for a SIP trunk account. This may involve completing an online form, providing contact information and payment details, and selecting the desired features and services.
Configure your VoIP phone system: Consult your VoIP phone system’s documentation to learn how to configure it to work with a SIP trunk. This may involve specifying the SIP trunk’s IP address and port number, as well as any authentication credentials that are required.
Test the connection: Once the SIP trunk is set up, it is a good idea to test the connection to ensure that it is working properly. Make a few test calls to verify that the connection is functioning as expected.
Use the SIP trunk: Once the SIP trunk is set up and tested, it can be used to make and receive calls using your VoIP phone system.
SIP Trunking platform has to integrate with multiple networks seamlessly. Components for setting up a SIP trunking system requires atleast these
Compliance with standrad signalling protol, like SIP.
SBC( Session Border Controller ) facing the private PBX
Gateway for specific endpoints such as PSTN gateway , public internet gateway etc
L3/L4 Layer switches
Telco operator lines
Codec support
Kamailio is an open-source SIP (Session Initiation Protocol) server that can be used to create a SIP trunk. Kamailio can be PBX used to connect different locations within an organization, enabling employees to communicate with each other using their VoIP phones. Kamailio can also be used to set up a SIP trunk in a number of ways. For example, it can be used to connect an organization’s VoIP phone system to the public telephone network, allowing employees to make and receive calls from outside the organization.
Kamailio is a highly flexible and customizable SIP server that can be configured to meet the specific needs of an organization. It offers a range of features and functionality, including call routing, load balancing, and security. Kamailio is a popular choice for organizations that want to set up a SIP trunk because it is open-source and can be customized to meet their specific needs.
SIP trunk with VoIP phone systems are often preferred over traditional phone systems because they are generally more flexible and cost-effective. They allow employees to make and receive calls from any device with an internet connection, including desk phones, smartphones, and laptops. They can be easily scaled up or down to meet changing communication needs and do not require the installation of additional physical hardware. Some factors to consider when evaluating SIP trunks include:
Cost: It is important to compare the costs of different SIP trunk providers and consider factors such as monthly fees, per-minute charges, and any additional fees for features or services.
Coverage: Make sure that the SIP trunk provider has coverage in the areas where your organization needs to make and receive calls.
Quality: The quality of a SIP trunk can vary greatly depending on the provider and the connection. Be sure to research the provider’s reputation for call quality and reliability.
Features: Different SIP trunk providers may offer different features, such as call forwarding, call waiting, and voicemail. Consider which features are important to your organization and make sure that the SIP trunk provider offers them.
Customer support: It is important to choose a SIP trunk provider that offers reliable customer support in case you experience any issues with your service.
Other features that are good to have is integration to existing backend for OSS/BSS stack. Some of the feature set for a carrier grade SIP trunking solution are listed here
Inbound and outbound trunks
Number Import/Export
Security
Dynamic registeration of users
Authentication and Authorization
Security (SRTP)
Cost Savings
Low cost for large traffic volumes instead of charges of call per second
CDR for tracing and monitoring call failures
Clear media stream ( no robotic or choopy audio). Good MOS score
realtime traffic monitoring to rule out bad players.
Inbound and Outbound call – Call Establishment, Rejection, Termination
DDI: Direct Dialling-In ranges can be provided on the SIP Trunk
CLIP(Calling Line Identification Presentation )/CLIR Calling Line Identification Presentation Restriction)for Inbound and Outbound
Incoming Call Barring: bar receiving of calls to certain extensions
Outgoing Call Barring: Restrict calls to certain numbers
Incoming Call Diversion – unconditional, busy, and unreachable
Call Admission Control: Call Admission Control (CAC) is a mechanism to restrict the number of simultaneous sessions (calls)
Incoming Call Diversion (DestNo not reachable, CAC exceeded, unconditional)
Geographic and Non-Geographic Number Support
Multiple Codec Support
Emergency Calling: Emergency Calls are routed on a priority basis irrespective of the customer’s available channel
Trunking inbound services voice can be used to support contact centres, conferencing, number translation services etc. Regulatory requirements for the operation of the customer in the PSTN of respective countries must be met with Country Specific Emergency Calling support Enhanced feature set for SIP trunking should include the features of the SIP Trunking with Multicountry support
Proactive MCID (Malicious CallerId) Identification and tracing
Call Distribution(CD)
Intelligent Routing involving machine learning and constant feedback
Origin Based Routing
Menu Routing
Origin Dependent Routing (ODR)
PIN Routing
Dynamic Route Select
Time-Dependent Routing (TDR)
Uniform Load Distribution(ULD)
International Routing
Mobile Routing
Payphone Routing
Product Association
Ultimately, the most useful SIP trunk for your organization will depend on your specific needs and budget. It is a good idea to research and compare different SIP trunk providers to find the one that best meets your organization’s needs.
SIP trunking systems are likely to continue to be an important part of the telecommunications landscape in the future. As more and more organizations adopt WebRTC or SRT based VoIP (Voice over Internet Protocol) technology for their phone systems, the demand for SIP trunks is likely to continue to grow. One trend that is expected to shape the future of SIP trunking is the increasing adoption of cloud-based communication systems. As more organizations move their communication systems to the cloud, they are likely to turn to SIP trunks as a way to connect their phone systems to the public telephone network and enable remote communication. Another trend that is expected to impact the future of SIP trunking is the increasing adoption of 5G technology. 5G networks offer faster speeds and lower latency, which may make it possible to use SIP trunks for real-time communication applications such as interactive and/or immersive video conferencing.
GDPR, Europe’s digital privacy legislation passed in 2018, replaces the 1995 EU Data Protection Directive. It is rules designed to give EU citizens more control over their personal data & strengthen privacy rights. It aims to simplify the regulatory environment for business and citizens.
GDPR (General Data Protection Regulation) in European Union 2018,
California Consumer Privacy Act (CCPA) 2019,
Personal Data Protection Bill (PDP) – India 2018 and
also specifications against Robocalls and SPIT ( SPAM over Internet Telephony) among others
Multinational companies will predominantly be regulated by the supervisory authority where they have their “main establishment” or headquarter. However, the issue concerning GDPR is that it not only applies to any organisation operating within the EU, but also to any organisations outside of the EU which offer goods or services to customers or businesses in the EU.
Key Principles of GDPR are
Lawfulness, fairness and transparency
Purpose limitation
Data minimisation
Accuracy
Storage limitation
Integrity and confidentiality (security)
Accountability
GDPR consists of 7 projects (DPO, Impact assessment, Portability, Notification of violations, Consent, Profiling, Certification and Lead authority) that will strengthen the control of personal data throughout the European Union.
Stakeholders
stakeholders of data protection regulation are Data Subject – an individual, a resident of the European Union, whose personal data are to be protected
Data Controller – an institution, business or a person processing the personal data e.g. e-commerce website.
Data Protection Officer – a person appointed by the Data Controller responsible for overseeing data protection practices.
Data Processor – a subject (company, institution) processing a data on behalf of the controller. It can be an online CRM app or company storing data in the cloud.
Data Authority – a public institution monitoring implementation of the regulations in the specific EU member country.
Extra-Territorial Scope
Any VoIP service provider may feel that since they are not based out of EU such as officially headquartered in the Asia Pacific or US region they may not be legally binding to GDPR. However, GDPR expands the territorial and material scope of EU data protection law. It applies to both controllers and processors established in the EU, and those outside the EU, who offer goods or services to or monitor EU data subject.
VoIP service providers as Data Processors
A processor is a “person, public authority, agency or other body which processes personal data on behalf of the controller”. Most VoIP service providers are multinational in nature with services offered directly or indirectly to all regions. The GDPR imposes direct statutory obligations on data processors, which means they will be subject to direct enforcement by supervisory authorities, fines, and compensation claims by data subjects. However, a processor’s liability will be limited to the extent that it has not complied with it’s statutory and contractual obligations.
Data minimization – It is now a good practise to store and process as less user’s personal data as necessary to render our services effectively. Also to maintain data for only a stipulated time ( approx 90 days of CDR for call details and logs )
Record Keeping, Accountability and governance
To show compliance with GDPR, a service provider maintain detailed records of processing activities. Also, they must implement technological and organisational measures to ensure, and be able to demonstrate, that processing is performed in accordance with the GDPR. Some ways to apply these are :
Contracts: putting written contracts in place with organisations that process personal data on your behalf
maintaining documentation of your processing activities
Organisational policies focus on Data protection by design and default – two-factor auth, strong passwords to guard against brute-force, encryption, focus on security in architecture
Risk analysis and impact assessments: for uses of personal data that are likely to result in a high risk to individuals’ interests
Audit by Data protection officer
Clear Codes of conduct
Certifications
As for a VOIP landscape thankfully every call or message session is followed by a CDR ( Calld Detail Record ) or MDR ( Message Detail Record).
Additionally, assign a unique signature to every data-access client the VoIP system and log every read/write operation carried out on data stores whether persistent datastores or system caches.
Privacy Notices to Subjects
User profile data such as :
Basic identity information, name, address and ID numbers
Web data such as location, IP address, cookie data and RFID tags
Health and genetic data
Bio-metric data
Racial or ethnic data
Political opinions
Sexual orientation
is protected strictly under GDPR rules
A service provider should provide indepth information to data subjects when collecting their personal data, to ensure fairness and transparency. They must provide the information in an easily accessible form, using clear and plain language.
Consent
The GDPR introduces a higher bar for relying on consent , requiring clear affirmative action. Silence, pre ticked boxes or inactivity will not be sufficient to constitute consent. Data subjects can withdraw their consent at any time, and it must be easy for them to do so.
Lawful basis for processing Data now include
In Article 6 of the GDPR , there are six available lawful bases for processing.
(a) Consent: the individual has given clear consent for you to process their personal data for a specific purpose.
(b) Contract: the processing is necessary for a contract you have with the individual, or because they have asked you to take specific steps before entering into a contract.
(c) Legal obligation: the processing is necessary for you to comply with the law (not including contractual obligations).
(d) Vital interests: the processing is necessary to protect someone’s life.
(e) Public task: the processing is necessary for you to perform a task in the public interest or for your official functions, and the task or function has a clear basis in law.
(f) Legitimate interests: the processing is necessary for your legitimate interests or the legitimate interests of a third party, unless there is a good reason to protect the individual’s personal data which overrides those legitimate interests.
File such as PCAPS , Recordings and transcripts of calls hold sensitive information from end users , these should be encryoted and inaccssible to even the dev teams within the org without explicit consent of end user .
Individuals’ Rights
The GDPR provides individuals with new and enhanced rights to Data subjects who will have more control over the processing of their personal data. A data subject access request can only be refused if it is manifestly unfounded or excessive, in particular because of its repetitive character.
Rights of Data Subjets include
Right of Access
Right to Rectification
Right to Be Forgotten
Right to Restriction of Processing
Right to Data Portability
Right to Object
Right to Object to Automated Decisionmaking
For a VoIP service provider if a user opts for redaction then none of his calls or messages should be traced in logs . Also replace distinguishable end user identifier such as phone number and sip uri with *** charecters
Provide option for “Account Deletion” and purge account – If a user wished to close his/her account , his/her detaisl should be deleted form the sustem except for the bare bones detaisl which are otherwise required for legal , taxation and accounting requirnments
Breach Notification
A controller is a “person, public authority, agency or other body which, alone or jointly with others, determines the purposes and means of processing of personal data”,
A controller will have a mandatory obligation to notify his supervisory authority of a data breach within 72 hours unless the breach is unlikely to result in a risk to the rights of data subjects. Will also have to notify affected data subjects where the breach is likely to result in a “high risk” to their rights. A processor, however, will only be obliged to report data breaches to controllers
International Data Transfers
Data transfers to countries outside the EEA(European Economic Area) continue to be prohibited unless that country ensures an adequate level of protection. The GDPR retains existing transfer mechanisms and provides for additional mechanisms, including approved codes of conduct and certification schemes.
The GDPR prohibits any non-EU court, tribunal or regulator from ordering the disclosure of personal data from EU companies unless it requests such disclosure under an international agreement, such as a mutual legal assistance treaty.
One of the biggest challenges for a service provider is the identification & categorization of GDPR impacted data sets in disparate locations across the enterprise. A dev team must flag tables, attributes and other data objects that are categorically covered under GDPR regulations and then ensure that they are not transferred to a server outside of EU.
In the present age of Virtual shared server instance, cloud computing and VoIP protocol it is operational a very tough task for a communication service provider to ensure that data is not transferred outside of EU such as a VoIP call from origination in US and destination in EU will require information exchanges via SDP, vcard , RTP stream via media proxies etc.
Sanctions
The GDPR provides supervisory authorities with wide-ranging powers to enforce compliance, including the power to impose significant fines. You will face fines of up to €20m or 4% of your total worldwide annual turnover of the preceding financial year. In addition, data subjects can sue you for pecuniary or non-pecuniary damages (i.e. distress). Supervisory authorities will have a discretion as to whether to impose a fine and the level of that fine.
Data Protection officer (DPO)
Under the terms of GDPR, an organisation must appoint a Data Protection Officer (DPO) if it carries out large-scale processing of special categories of data, carries out large scale monitoring of individuals such as behaviour tracking or is a public authority.
Major standards bodies including 3GPP, ITU-T, and ETSI have all adopted SIP as the core signalling proMajor standards bodies including 3GPP, ITU-T, and ETSI have all adopted SIP as the core signalling protocol for services such as LTE, VoIP, conferencing, Video on Demand (VoD), IPTV (Internet Television), presence, and Instant Messaging (IM) etc. With the continuous evolution of SIP as the defacto VoIP protocol, we need to understand the risk mitigation practices around it.
malicious registrations on registrar by a third party who modifies From header field of a SIP request.
exmaple implementation : attacker de-registers all existing contacts for a URI attacker can also register their own device as the appropriate contact address, thereby directing all requests for the affected user to him
attacker impersonates the remote server user’s request can now be intercepted by some other party user’s request may be forwarded to insecure locations
Solution : confidentiality, integrity, and authentication of proxy servers
Proxy/redirect sever, and registrars SHOULD possess a site certificate issued by CA which could be validated by UA
If users are relying on SIP message bodies to communicate either of
session encryption keys for a media session
MIME bodies
SDP
encapsulated telephony signals Then the atackers on proxy server can modify the session key or can act as a man-in-the-middle and do eaves droppng
exmaple implementation : attacker can point RTP media streams to a wiretapping device can changes Subject header field to appear to users as spam
solution – end to end ecryption over TLS + Digest Authorization
Mid-session threats like tearing down session
Request forging– attacker learns the params of the session like To , From tags etc then he can alter ongoing session parameters and even bring it down
example implementation : attacker inserts a BYE in a ongoing session thereby tearing it down can insert re INVITE and redierct the stream to wiretaping device
solution – authetication on every request signing and encrypting of MIME bodies, and transference of credentials with S/MIME
DOS – rendering a particular network element unavailable, usually by directing an excessive amount of network traffic at its interfaces. dDOS – multiple network hosts to flood a target host with a large amount of network traffic.
Can be created by sending falsified sip requests to other parties such that numerous transactions originating in the backwards direction comes to the target server created congestion. Some examples of implementation of DOS attacks :
Attackers creates a falsified source IP address and a corresponding Via header field that identify a targeted host as the originator of the request. Then send this to large number of SIP network element. This geneerates DOS aimed at target.
Attackers uses falsified Route header field values in a request that identify the target host and then send such messages to forking proxies that will amplify messaging sent to the target.
Flooding with register attacks can deplete available memory and disk resources of a registrar by registering huge numbers of bindings.
Flooding a stateful proxy server causes it to consume computational expense associated with processing a SIP transaction
Solution – detect flooding and pike in traffic and use ipban to block challenge questionable requests with only a single 401 (Unauthorized) or 407 (Proxy Authentication Required), forgoing the normal response retransmission algorithm, and thus behaving statelessly towards unauthenticated requests.
Security mechanisms
Full encryption vs hop by hop encrption
SIP mssages cannot be encrypted end-to-end in their entirety since message fields such as the Request-URI, Route, and Via need to be visible to proxies in most network architectures so that SIP requests are routed correctly. proxy servers need to also update the message with via headers
Thus SIP uses low level security along with hop by hop encrption and auth headers to verify the identity of proxy servers
Transport and Network Layer Security
IPsec – used where set of hosts or administrative domains have an existing trust relationship with one another.
TLS – used where hop-by-hop security is required between hosts with no pre-existing trust association.
SIPS URI Scheme
Used as an address-of-record for a particular user, signifies that each hop over which the request is forwarded, must be secured with TLS
HTTP Authentication
Reuse of the HTTP Digest authentication via 401 and 407 response codes that implement challenge for autehtication provides replay protection and one-way authentication.
S/MIME
allows SIP UAs to encrypt MIME bodies within SIP, securing these bodies end-to-end without affecting message headers. provides end-to-end confidentiality and integrity for message bodies
nonce-count
provides replay protection
SIP over TLS
SIP messages can be secured using TLS. There is also TLS for Datagrams called DTLS.
Security of SIP signalling is different from security of protocols used in concert with SIP like RTP , RTCP. and that will be covered in later topics of this article.
TLS operation consists of two phases: handshake phase and bulk data encryption phase
Handshake phase
Prepare algorithm to be used during TLS session
Server Authentication
server sends its certificate to the client, which then verifies the certificate using a certificate authority’s (CA’s) public key.
Client Authentication
Server sends an additional CertificateRequest message to request the client’s certificate. The client responds with
Certificate message containing the client certificate with the client public key and
CertificateVerify message containing a digest signature of the handshake messages signed by clients private key
Server authenticates client by client’s public key , since only client holding correct private key can sign the message.
Prepare the shared secret for bulk data encryption
client generate a pre_master_secret, and encrypt it using the server’s public key obtained from the server’s certificate. The server decrypts the pre_master_secret using its own private key. Both the server and client then compute a master_secret they share based on the same pre_master_secret. The master_secret is further used to generate the shared symmetric keys for bulk data encryption and message authentication
Public key cryptographic operations such as RSA are much more expensive than shared key cryptography. This is why TLS uses public key cryptography to establish the shared secret key in the handshake phase, and then uses symmetric key cryptography with the negotiated shared secret as the data encryption key.
Stateless proxy servers do not maintain state information about the SIP session and therefore tend to be more scalable. However, many standard application functionalities, such as authentication, authorization, accounting, and call forking require the proxy server to operate in a stateful mode by keeping different levels of session state information.
Steps :
The SIP proxy server enforces proxy authentication with 407 Proxy Authentication Required challenge.
UAC provides credentials that verify its claimed identity (e.g., based on MD5 [34] digest algorithm) and retransmits in authorization header
Security of RTP
confidentiality protection of the RTP session and integrity protection of the RTP/RTCP packets requires source authentication of all the packets to ensure no man-in-the-middle (MITM) attack is taking place.
end to end media encryption – SRTP ( Secure RTP )
encodes the voice into encrypted IP packages and transport those via the internet from the transmitter to receive
References
The Impact of TLS on SIP Server Performance – Charles Shen† Erich Nahum‡ Henning Schulzrinne† Charles Wright , Department of Computer Science, Columbia University,IBM T.J. Watson Research Center
I have written about VoIP and security in these blogs before
For security around web browser-based calling via webrtc, the articles below discuss security practices in general
Webrtc Security , which describes browser threat modal , access to local resource , Same Orogin Policy (SOP) and Cross Resource Sharing ( CORS) as well as Location sharing , ICE , TUEN and threats to privacy with screen sharing , microgone camera long term access and probable mid call attacks .
Genric secrutity of web Application build around hosting platform of webrtc. Includs concepts like Identity management , browser security – cross site security amd clickjacking , Authetication of devices and applications , Media Encryption and regex checking.
Secure Communication – https://telecom.altanai.com/2018/03/16/secure-communication-with-rtp-srtp-zrtp-and-dtls/ which discusses the Key management protocols used for establishing end to end encryption in VoIP media streams. It describes Sanity checks , ACL lists with permissions , hiding topology details , countering Flood using pike and Fail2Ban as well as Traffic monitoring and detection .
This article describes various Certificates and compliances, Bill and Acts on data privacy, Security and prevention of Robocalls as adopted by countries around the world pertaining to Interconnected VoIP providers, telecommunications services, wireless telephone companies etc
Compliance certificates by Industry types
HIPAA (Health Insurance Portability and Accountability Act)
Deals with privacy and security of personal medical records and electronic health care transaction
Applicability : If voip company handles medical information
Includes :
Not allowed Voice mail transcription
Should have End-to-End Encryption
Restrict using unsecured WiFi networks to prevent Snooping
User security , strong password rules and mandatory monthly change
Secure Firmware on VoIP phones
Maintaining Call and Access Logs
SOX( Sarbanes Oxley Act of 2002)
Also known as SOX, SarbOX or Public Company Accounting Reform and Investor Protection Act
Applicability : if managing the communications operations of a regulated, publicly traded company
Includes :
Retain records which include financial and other sensitive data
ways employees are provided or denied access to records or data based on their roles and responsibilities
do information audit by a trusted third party.
Retention and deletion of files such as audio files like voicemails, text messages, video clips, declared paper records, storage, and logs of communications activities
Physical and digital security controls around cloud-based VoIP applications and the networks
Privacy Related Compliance certificates
COPPA (Children’s Online Privacy Protection Act ) of 1998
prohibits deceptive marketing to children under the age of 13, or collecting personal information without disclosure to their parents.
any information is to be passed on to a third party, must be easy for the child’s guardian to review and/or protect
2011 amendment requires that the data collected was erased after a period of time,
2014 FTC issued guidelines that apps and app stores require “verifiable parental consent.”
CPNI (Customer Proprietary Network Information) in united states is the information that communication providers acquire about their subscribers. This Individually identifiable information that is created by a customer’s relationship with a provider, such as data about the frequency, duration, and timing of calls, the information on a customer’s bill, and call identifying information. This processing information is governed strictly by FCC and certification should be renewed on an annual basis
Provider can pass along that information to marketers to sell other services, as long as the customer is notified
In 2007, the FCC explicitly extended the application of the Commission’s CPNI rules of the Telecommunications Act of 1996 to providers of interconnected VoIP service.
CALEA
Communications Assistance for Law Enforcement Act (CALEA) conduct electronic surveillance by imposing specific obligations on “telecommunications carriers” for assisting law enforcement, including delivering call interception and call identification functionality to the government with a minimum of interference to customer service and privacy.
Establishes requirements of organizations that process data, defines the rights of individuals to manage their data, and outlines penalties for those who violate these rights.
No personal data may be processed unless this processing is done under one of six lawful bases specified by the regulation (consent, contract, public task, vital interest, legitimate interest or legal requirement). When the processing is based on consent the data subject has the right to revoke it at any time.
Controllers must notify Supervising Authorities (SA)s of a personal data breach within 72 hours of learning of the breach.
California Consumer Privacy Act (CCPA) 2019
consumer rights relating to the access to, deletion of, and sharing of personal information that is collected by businesses.
Allows consumers to know whether their personal data is sold or disclosed , to whom .
Allows opt-out right for sales of personal information
Right to deletion – to request a business to delete any personal information about a consumer collected from that consumer
Personal Data Protection Bill (PDP) – India 2018
This bill introduces various private and sensitive protection frameworks like restriction on retention of personal data, Right to correction and erasure (such as right to be forgotten) , Prohibition and transparency of processing of personal data. It also classifies data fiduciaries including certain social media intermediaries.
The Bill amends the Information Technology Act, 2000 to delete the provisions related to compensation payable by companies for failure to protect personal data.
Other data privacy acts similar to GDPR
South Korea’s Personal Information Protection Act 2011
Brazil’s Lei Geral de Proteçao de Dados (LGPD) 2020
Privacy Amendment (Notifiable Data Breaches) to Australia’s Privacy Act 2018
Japan’s Act on Protection of Personal Information 2017
Thailand Personal Data Protection Act (PDPA) 2020
Features offered by VOIP companies for Data privacy
Access Control & Logging
Auto Data Redaction / Account Deletion policy
SIEM (Security information and event management) alerts
Information security , Encrypted Storage For Recordings & Transcripts
Disclosing all third party services that are involved in data processing too
Role Based Access Control and 2 Factor Authentication
Data Security Audits and appointing data protection officer to oversee GDPR compliance
Against Robocalls and SPIT ( SPAM over Internet Telephony)
2009 Truth in Caller ID Act
Telephone Consumer Protection Act of 1991
Implementation of Do not call registry against use of robocalls, automatic dialers, and other methods of communication
Do-Not-Call Implementation Act of 2003
if a business has an established relationship with a customer, it can continue to call them for up to 18 months. If a consumer calls the company, say, to ask for information about the product or service, the company has three months to get back to him.
if the customer asks to not receive calls, the company must stop calling, or be subject to fines.
Exemptions – Calls from a not-for-profit B organisation , informational messages as flight cancellations , Calls from sales and debt collectors etc
Personal Data Privacy and Security Act 2009
Implemented to curb identity theft and computer hacking. Sensitive personal identifiable information includes : victim’s name, social security number, home address, fingerprint/biometrics data, date of birth, and bank account numbers.
Any company that is breached must notify the affected individuals by mail, telephone, or email, and the message must include information on the company and how to get in touch with credit reporting agencies
If the breach involves government or national security , company must also contact the Secret Service within fourteen days
TRACED Act (Telephone Robocall Abuse Criminal Enforcement and Deterrence) 2019
Canadian Radio-television and Telecommunications Commission (CRTC) 2018 -32
Unlike traditional telephone connections, which are tied to a physical location, VOIP’s packet switched technology allows a particular number to be anywhere making it more difficult for it to reach localised services like emergency numbers of Public Safety Answering Points (PSAPs) . Thus FCC regulations as well as the New and Emerging Technologies 911 Improvement Act of 2008 (NET 911 Act), interconnected VoIP providers are required to provide 911 and E911 service.
To understand the need for implementing an identification verification technique in Internet protocol based network to network communication system , we need to evaluate the existing problem plaguing the VoIP setup .
What is Call ID spoofing ?
Vulnerability of existing interconnection phone system which is used by robo-callers to mask their identity or to make it appear the call is from a legitimate source, usually originates from voice-over-IP (VOIP) systems.
In this context understand the Caller Line identification CLI/ NCLI techniques used by VoIP and OTT( over the top) providers today.
CLI (Caller Line Identification)
If call goes out on a CLI route ( White Route ) the received party will likely see your callerID information
Lawful – Termination is legal on the remote end ie abiding country’s telco infrastructure and stable
Expensive – usually with direct or via leased line (TDM) interconnections with the tier-1 carriers.
Non-CLI (Non-Caller Line Identification)
The Caller ID is not visible at the call If call goes out on a Non-CLI route (Grey Route) goes out on a non-CLI routes they will see either a blocked call or some generic number.
Unlawful – questionable legality or maybe violating some providers AUP(Acceptable Use Policy ) on the remote end.
Cheaper – low quality , usually via VoIP-GSM gateways
Example include robocalls , tele-marketting / spam etc which are unwilling to share their Caller Id for call receiver, to not be blocked or cancelled.
To overcome the problem of non-verifiable spam , robocalls a suite of protocols and procedures are proposed that can combat caller ID spoofing on VOIP and connected public telephone networks.
STIR/SHAKEN
Secure Telephony Identity Revisited (STIR) / Signature-based Handling of Asserted information using toKENs (SHAKEN)
Used by robocallers to mask their identity or to make it appear the call is from a legitimate source usually orignates from voice-over-IP (VOIP) systems
STIR
Standards developed by the Internet Engineering Task Force (IETF)
For telecommunication service providers implement certificate management system to create and manage the public and private keys, digital certificates used to sign and verify Caller ID details.
Adds information to the SIP headers that allow the endpoints along the system to positively identify the origin of the data , such as JSON web tokens encrypted with the provider’s private key, encoded using Base64,
There are three levels of verification, or “attestation”
A : Full Attestation indicates that the provider recognizes the entire phone number as being registered with the originating subscriber.
B : Partial Attestation call originated with a known customer but the entire number cannot be verified,
C : Gateway Attestation call can only be verified as coming from a known gateway
How can the Public Key Infrastructure be used ?
In an interconnection network , each telephone service provider will obtain its digital certificate from a certificate authority (CA) that is trusted by other telephone service providers. Calling party signs the SIP Header caller ID as legitimate . The called party verifies that the calling number is authentic
STIR
Originating service provider’s encrypted SIP Identity Header includes the following data:
Attestation level
Date and Time
Calling and Called Numbers
Orig ID for analytics and/or traceback purposes among others
Location of certificate repository
Signature
Encryption algorithm
FCC has also assigned the role of a Secure Telephone Identity Policy Administrator (STI-PA) which oversees that CAs do not provide certificate to spoofing robocallers and enforce the framework for STIR /SHAKEN .
STIR is based on the SIP protocol and is designed to work with calls being routed through a VOIP network. Since traditional endpoints like POTS and SS7 networks also should be covered under this call authenticity framework , SHAKEN was developed to manage call via IP-to-telephone gateways .
Developed by the Alliance of Telecommunications Industry Solutions (ATIS)
Working Steps :
When a call is initiated, a SIP INVITE is received by the originating service provider.
Originating service provider verifies the call source and number to determine how to confirm validity.
Full Attestation (A) — The service provider authenticates the calling party AND confirms they are authorized to use this number. An example would be a registered subscriber.
Partial Attestation (B) — The service provider verifies the call origination but cannot confirm that the call source is authorized to use the calling number. An example would be a calling number from behind an enterprise PBX.
Gateway Attestation (C) — The service provider authenticates the call’s origin but cannot verify the source. An example would be a call received from an international gateway.
Create a SIP Identity header that contains information on the calling number, called number, attestation level, and call origination, along with the certificate thus caller ID “signed” as legitimate
SIP INVITE with the SIP Identity header with the certificate is sent to the destination service provider.
Destination service provider verifies the identity of the header and certificate.
Diagrammatic depiction of flow of how Telecom carriers to digitally validates authenticity before receiving or handoff through their network
SHAKEN
References
RFC 8224 – Authenticated Identity Management in the Session Initiation Protocol (SIP)
RFC 8226 – Secure Telephone Identity Credentials: CertificatesRFC 8588 – Personal Assertion Token (PaSSporT) Extension for Signature-based Handling of Asserted information using toKENs (SHAKEN)
SBC ( Session Borde Controllers ) are basically gateways that provide interconnectivity between the hosted IP-PBX of the enterprise to the outside world endpoints such as telco service provider, PSTN/ TDM , SIP trunking providers or even third party OTT provider apps like skype for business etc. If you have a hosted IPPBX or PBX in your data-centre or on premise and you need controlled but heavy outflowing traffic, it is a good idea to integrate a resilient and efficient SBC to provide seamless interconnectivity.
For an enterprises such as an Trading floor or warehouse with multiple phone types , softphones , hardphones , turrets etc distributed across various geographies and zones a device agnostic architectural setup is prime . Listing the essentials for setting up such a system. Note supplementary services are data-services , logging , licensing etc are important but kept out of scope to keep focus on functional aspects .
An enterprise application usually is structured in tiers or layers
Client tier – the networks clients communication to the central java programs . Runs on client machines
web tier – state full communication between client and business tier . Runs in server machine.
business tier- handles the logic of the application. The business tier uses the Enterprise Java Bean (EJB) container, which manages the execution of the beans
data tier – encompasses DB drivers . Runs on separate machines for database storage
Event services for Line status notifications
providers lines status notification across enterprise for inter zone and softphone to hardphone .
Routing services
routing calls within enterprise and hardphone sites read more about resource zones later in the article
Consolidated set of all service and component that make up the VOIP platform besides media handlers. It includes SIP adapters, bridge managers, call processing frameworks, API frameworks, healthchecks etc.
Call processing framework ( CPF)
Signalling and call routing logic, mostly in SIP and trunks. Manages identities such as Call Line information, Called Party Information, line status etc in shared memory.
Multiple shared Lines and their statuses
Incases where there is a need to process multiple calls from a single User agent device such as a softphone or hardphone ( common scenario for a turret phone) , the design involves assigning it multiple sip uris and each sip uri will establish a line. When caller calls callee , the line is said to be BUSY , otherwise said to be IDLE. Transition of a shared sip line from IDLE to BUSY is transmitted to others via SIP PUBLISH as other UAs holding the same sip Similarly any other event like transfer is propagated to other via SIP UPDATE
Clustering Call control managers
A Call Communication manager (CCM) from various zones should be able to cowork on call and session management and advanced features such as routing from home guest zone to home zone , call transfer , refer , barge etc. Designing a clustered setup will also provide elasticity , fail-over and high availability. Can use clustered , HA compliant framework such as Oracle Communication Application Server , suited for enterprise level deployments.
Call Replication and distributed memory management
A node will store two types of data: active sessions and passive sessions. The active sessions are used by the node and stored in cache. The passive sessions are the replicas from the other nodes’ active sessions. The passives sessions are stored on a persistent storage.
When dealing with many SIP endpoints , now referred to as resource, it is best to assign the resources to their respective zones. Thus a resource’s status updates will be only updated by its active resource zone while can be read by any resource zone.
Incoming request Zone vs Active Resource Zone
For an Incoming request such a INVITE , check whether the zone sending the request is its active resource zone or not .If the Active Resource Zone is the same zone on which the INVITE came in, then the call is handled by that zone. If the Active Resource Zone is a different zone, then the call needs to be forwarded to the Active Resource Zone.
Bridges for Local Media connections
Although call signalling is handled by a resources active resource zone only, we can still create media bridges in local zone of the resource .
Local MM bridges are used to auto answer an incoming sip line call and create trunk , especially from hardphones which do not support provisional responses.
Interzone proxy Handler
proxies call control messages between active and non active resource zones. Primarily mapping the sip messages with all custom headers inbetween the communication device interfaces.
Dial Trunk using multiple dedicated SIP lines and connect via Media Bridge
To save up on call routing /connection time and to support te ability to add as many users on call at runtime , a dedicated media bridge is established for every call.
A sip line activated is auto-answered by MM , creates a trunk and waits for other endpoint to join the bridge. The flow is as follows :
As INVITE arrives for an IDLE sip line , it is connected to a trunk and auto answered by a local MM bridge .
Since the call is already answered , when caller dials number for callee , collect the DTMF digits over RTP using RFC 2833 DTMF events.
Run inter-digit timer for digit collection and detect end of dialing on timeout.
The dialed trunk connection is made and call is added to media bridge
When provisional responses are received on the trunk connection, generate in-band call progress tones (ringing, proceeding etc) via the MM
When the line answers, the progress tones have to be stopped and the called party gets bridged to the calling party via the media bridge.
Call Diversion involves forwarding calls from zone to another zone. joinjed parties get call UPDATE status and forward response.
Call barge is the processing of joining an ongoing call . The barge event is usually propagated to joined parities via SIP INFO. Private lines do not allow barge in and are exclusively reserved for only few users.
Interconnectivity provided by an SBC ( Session Border Controller)
Hold-Resume and Music on Hold in multi-line evironment
While a regular p2p call involves simple reinvite based hold and resume with varrying SDP, the scenario is slightly more detailed for hold resume on bridged trunk connection , as explained below.
As the calls made are on bridge , a hold signal involves a RE-INIVITE with held-SDP to media manager (MM). If hold status on trunk is 200 OK the hold status will be sent to other call interfaces connected on the trunk. Else if hold is denied, 403 is sent back to hold-initiates.
Music on hold is an one way RTP mostly from media server.
For a bridged scenarios , separate Music on hold bridges are kept on Media Managers. When an UA has to hold , it is removed from original bridge and place on music on hold bridge. To be unhold/ resume it is placed back into the orignal bridge from music on hold bridge.
Conference
user initiates conference, the conference feature can execute on the zone where the user was logged on, irrespective of zones where the other conference attendees join from . The Call processing framework of originators zone completes the SDP exchange to establish two-way speech path among all the parties.
Incases there are multiple connections from a zone , a local MM conference bridge can be created for them which would connect back to originators MM conf bridge . this two part conf bridge will be transparent to the sip line sand users .
For provisioning inputs and settings setup a Diagnostics , Administration and Configuration platform which can process APIs for data services , licences , alarms or do remote device control such as using SNMP.
bridging multiple interfaces in different networks even between the IPv4 and IPv6 networks
auto NAT discovery and STUN
protocol conversion such as TLS to UDP etc
Flood detection and IP filtering
For SIP specific functionalities, SBC does
SIP validation involving checks on syntax and message contents also consistency checks are performed.
stateful and call aware. tracing, monitoring and checking for validitya and health of all the SIP messages
Topology hiding
Traffic filtering
Codec filtering , reordering , media pinning, transcoding, or call recording
Data replication brings High Availability (HA) with hot backups or even Active-Active solutions.
Traffic sharing and routing roles of SBC can include
IP-based and Digest-based authentication
limiting traffic by number of concurrent calls or calling rate.
Dialplan and/or Custom routing
Dispatching/Load-balancing to a backend cluster of servers
SBC’s can be physical hardware boxes or software based applications, as the name suggests their purpose is to control the session at border between the enterprise and external service provider. They can be used for various roles such as
SIP to PSTN – SIP is an IP protocol whereas PSTN is a TDM one , achieving interoperability is also the KRA of an SBC
SIP trunking – SBC provide a secure sip connectivity to connect calls to sip trunks which provide bulk calls functionality at a flat pricing.
support for various fixed or mobile endpoints – SBC ensure they are RFC compliant and can extend SIP to any kind of telecom endpoint like PSTN , GSM, fax , Skype , sipphone , IP phones etc.
NAT (Network address translator) – To meet the packet routing challenges across a firewall or even during private -public mapping. A combo of DHCP servers and NAT provider comes very handy to reroute or perform hole punching such that signalling and media packets are not dropped and meet the required endpoint. More about NAT here – NAT traversal using STUN and TURN.
Load balancing – Reverse proxies and Load balancers is a much adopted industry practise to mask the inner IPs of the VoIP platform and also route traffic appropriately between control and media server .
Security, QoS and Regulatory compliance – since SBCs are required to typically support a large array of clients they adhere to regulatory and industry accepted standards ,which also involves security features like AAA, TLS/SSL and other means for quality of assurance like logging and fault detection, preventing DDoS etc . In many cases SBC can also encrypt / decrypt RTP streams for probing , tapping or lawful inspection .
There are 2 ways to integrate IP calls to telecom provider endpoints such as GSM or LTE phones.
PRI lines
SIP trunks
Additional SBC features
Inaddition to above it is good to have if an SBC provides extra features like forking , emergency number dialing ( 911 ) or active directory integration . Real Time Analysis and monitoring of call and metrics are also expected from a SBC since they reside on edge of the network and are more vulnerable to threats . For example Dialogic Mediant SBC’s and gateways , Audio Codes SBCs
With the shift from on premise PBXs to cloud based VM or microservice architecture , SBC vendors adopt a lager umbrella of services also including automation scripts for checks , reporting tools / consoles , developer friendly APIs to manage sessions via SBC and even WebRTC gateways to connect browser endpoints.
A basic enterprise VoIP/SIP solution is illustrated in Figure. The key element is a soft switch (SIP PBX) which might be implemented as a combination of several SIP entities, such as SIP registrar, proxy server, redirect server, forking server, Back-To-Back User Agent (B2BUA) etc. SIP clients can be SIP hard-phones or soft-phones on PCs, PDAs etc. A PSTN gateway links the enterprise SIP PBX to the public PSTN. Enterprise applications, media servers, presence servers, and the VoIP/SIP PBX are interconnected through a company intranet.
VoIP System with IMS : With IMS, applications will be able to establish sessions across different access networks, with guaranteed QoS, flexible charging & AAA support. Call control, user’s database and services, which are the typical functions of softswitch, are controlled by separate units in IMS. CSCF (Call Session Control Function) handles session establishment, modification and release of IP multimedia sessions using the SIP/SDP protocol suite. Services features are separated from call control and handled by application servers. Subscriber’s database function is separated from service logic function and handled by HSS using open subscriber directory interface.
Link registration using subscribe-notify can be handled via Enterprise App server in PBX.
Forking proxy Setup of PBX : The enterprises SIP PBX can work as a forking proxy during call setup to redirect the calls.
Other usecases can involve presence sharing between different enterprise PBX with both domains interconnect their presence servers.
Any VOIP dependant system which deals with bulksome voice / video traffic from external endpoints is a usages scenarios. Listing few
provision of pre-defined enterprise based SIP URI.
Contact Call centres
Remote work / offsite monitoring
CRM solution for sales/marketing
Connecting webrtc click to dial from webpage to enterprise representatives
connecting enterprise UCC clients to PSTN endpoints
The There are many more features and usecases for an IP-PBX solution for an enterprise. The features of modern IP PBX systems are a big addon to internal secure telecom channel in an company and accross its various office.
There has been a significant shift in replacing hard PBX systems with software-based IP PBX such as using Freeswitch, Asterisk or other commercial-grade SIP servers which seamlessly integrate into other business software such as CRM systems, task force management systems. In recent times cloud telephony providers, particularly CPaaS platforms have revolutionized the IP telecommunication landscape with lightweight and feature-rich communication agents( web, native platform) and services such as programmable API to control call logic and services such as recording, IVR announcements, call parking, Automatic Queueing so on.
Elevated Call failure SIP 503 or Call timeout SIP 408
cron service or processed alerts
DB connections / connection pool process alerts
port check, unexpected result alert
cron zombie process checks alerts
Bulk calls checks
Process control supervisor or pm2 checks
Health and load on the reverse proxy, load balancer as Nginx alerts
VPN checks
SSL cert expiry checks
Health of Task scheduling services such as RabbitMQ, Celery Distributed Task Queue
Cluster status
Status of Crticial Application Server
Programming or Syntax error in the production environment
Distributed memory caching – redis , memcahe
SMS service using smsc on Kannel
Overview of VoIP platform DevOPS tools
This article is focussed around various tools required to operate and maintain a growing large scale VoIP Platform, which are mostly classified under following roles:
PCAP Collections
CICD on Jenkins pipeline
Configuration management using chef cookbooks
virtualization and containerization using Docker
Infrastructure management using terraform / Kubernetes
Packet Capture (PCAP) is an API that captures live network packets. Besides tracking, audit and RTC visualizers, PCAP is widely used for debugging faults such as during production alarm on high failure occurrences.
Example usecase: Production alert on 503 SIP response or log entry from a gateway is not as helpful as PCAP tracking of the session ID of call across various endpoints in and out of the network to determine the point of failure.Debugging involves :
Pre-specified SIP / RTP and related protocols capture
Docker containers can be used instead of virtual machines such as VirtualBox , to isolates applications and be OS and platform independent
Makes distributed development possible and automates the deployment possible
unpause Unpause all processes within one or more containers
update Update configuration of one or more containers
wait Block until one or more containers stop, then print their exit codes
see all iamges
> docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
sipcapture/homer-cron latest fb2243f90cde 3 hours ago 476MB
sipcapture/homer-kamailio latest f159d46a22f3 3 hours ago 338MB
sipcapture/heplify latest 9f5280306809 21 hours ago 9.61MB
<none> <none> edaa5c708b3a
See all stats
> docker stats
CONTAINER ID NAME CPU % MEM USAGE / LIMIT MEM % NET I/O BLOCK I/O PIDS
f42c71741107 homer-cron 0.00% 52KiB / 994.6MiB 0.01% 2.3kB / 0B 602MB / 0B 0
0111765091ae mysql 0.04% 452.2MiB / 994.6MiB 45.46% 1.35kB / 0B 2.06GB / 49.2kB 22
Run command from within a docker instnace
docker exec -it bash
First see all processes
docker ps
select a process and enter its bash
docker exec -it 0472a5127fff bash
to edit or update a file inside docker either install vim everytime u login in resh docker conainer like
apt-get update
apt-get install vim
or add this to dockerfile
RUN [“apt-get”, “update”] RUN [“apt-get”, “install”, “-y”, “vim”]
see if ngrep is install , if not then install and run ngrep to get sip logs isnode that docker container
apt update
apt install ngrep
ngrep -p "14795778704" -W byline -d any port 5060
docker volume – Volumes are used for persisting data generated by and used by Docker containers. docker volumes have advantages over blind mounts such as easier to backup or migrate , managed by docker APIs, can be safely shared among multiple containers etc
docker stack – Lets to manager a cluster of docker containers thorugh docker swarm can be defined via docker-compose.yml file
docker service
create Create a new service
inspect Display detailed information on one or more services
logs Fetch the logs of a service or task
ls List services
ps List the tasks of one or more services
rm Remove one or more services
rollback Revert changes to a service’s configuration
scale Scale one or multiple replicated services
update Update a service
Run docker containers
sample run command
docker run -it -d --name opensips -e ENV=dev imagename:2.2
-it flags attaches to an interactive tty in the container.
-e gives envrionment variables
-d runs it in background and prints container id
Remove docker entities
To remove all stopped containers, all dangling images, and all unused networks:
docker system prune -a
To remove all unused volumes
docker system prune --volumes
To remove all stopped containers
docker container prune
sometimes docker images keep piling with stopped congainer such as
REPOSITORY TAG IMAGE ID CREATED SIZE d1dcfe2438ae 15 minutes ago 753MB 2d353828889b 16 hours ago 910MB ...
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
0dd6698a7517 2d353828889b "/entrypoint.sh" 13 minutes ago Exited (137) 13 minutes ago hardcore_wozniak
to remove such images and their conainer , first stop and remove confainers
docker stop $(docker ps -a -q)
docker rm $(docker ps -a -q)
Terraform is used for building, changing and versioning infrastructure. Infra as Code – can run single application to datacentres via configuration files which create execution plan. It can manage low-level components such as compute instances, storage, and networking, as well as high-level components such as DNS entries, SaaS features, etc. Resource Graph – builds a graph of all your resources
tfenv can be used to manage terraform versions
> brew unlink terraform
tfenv install 0.11.14
tfenv list
This is used for declaring resources and descriptions of infrastructure and associated files have a .tf or .tf.json file extension Group of resources can be gathered into a module. Terraform configuration consists of a root module, where evaluation begins, along with a tree of child modules created when one module calls another.
Example : launch a single AWS EC2 instance , fle server1.tf
container orchestration platform , automating deployment, scaling, and management of containerized applications. Can deploy to cluster of computers, automating the distribution and scheduling as well
Service discovery and load balancing – gives Pods their own IP addresses and a single DNS name for a set of Pods, and can load-balance across them.
Automatic bin packing – Automatically places containers based on their resource requirements and other constraints, while not sacrificing availability. Mix critical and best-effort workloads in order to drive up utilization and save even more resources.
Storage orchestration – Automatically mount the storage system of your choice, whether from local storage, a public cloud provider such as GCP or AWS, or a network storage system such as NFS, iSCSI, Gluster, Ceph, Cinder, or Flocker.
Self-healing – Restarts containers that fail, replaces and reschedules containers when nodes die, kills containers that don’t respond to your user-defined health check, and doesn’t advertise them to clients until they are ready to serve.
Automated rollouts and rollbacks – progressively rolls out changes to your application or its configuration, while monitoring application health to ensure it doesn’t kill all your instances at the same time.
Secret and configuration management – Deploy and update secrets and application configuration without rebuilding your image and without exposing secrets in your stack configuration.
Batch execution– manage batch and CI workloads, replacing containers that fail, if desired.
Horizontal scaling – Scale application up and down with a simple command, with a UI, or automatically based on CPU usage.
Starting Kubernetes…minikube version: v1.3.0
commit: 43969594266d77b555a207b0f3e9b3fa1dc92b1f
minikube v1.3.0 on Ubuntu 18.04
Running on localhost (CPUs=2, Memory=2461MB, Disk=47990MB) …
OS release is Ubuntu 18.04.2 LTS
Preparing Kubernetes v1.15.0 on Docker 18.09.5 …
kubelet.resolv-conf=/run/systemd/resolve/resolv.conf
Pulling images …
Launching Kubernetes …
Done! kubectl is now configured to use "minikube"
dashboard was successfully enabled
Kubernetes Started
Basic Commands
start Starts a local kubernetes cluster
status Gets the status of a local kubernetes cluster
stop Stops a running local kubernetes cluster
delete Deletes a local kubernetes cluster
dashboard Access the kubernetes dashboard running within the minikube cluster
Images Commands:
docker-env Sets up docker env variables; similar to ‘$(docker-machine env)’
cache Add or delete an image from the local cache.
Configuration and Management Commands:
addons Modify minikube’s kubernetes addons
config Modify minikube config
profile Profile gets or sets the current minikube profile
update-context Verify the IP address of the running cluster in kubeconfig.
Networking and Connectivity Commands:
service Gets the kubernetes URL(s) for the specified service in your local cluster
tunnel tunnel makes services of type LoadBalancer accessible on localhost
Advanced Commands:
mount Mounts the specified directory into minikube
ssh Log into or run a command on a machine with SSH; similar to ‘docker-machine ssh’
kubectl Run kubectl
Troubleshooting Commands:
ssh-key Retrieve the ssh identity key path of the specified cluster
ip Retrieves the IP address of the running cluster
logs Gets the logs of the running instance, used for debugging minikube, not user code.
update-check Print current and latest version number
kubectl
controls the Kubernetes cluster manager.
Basic Commands (Beginner):
create Create a resource from a file or from stdin.
expose Take a replication controller, service, deployment or pod and expose it as a new Kubernetes Service
run Run a particular image on the cluster
set Set specific features on objects
explain Documentation of resources
get Display one or many resources
edit Edit a resource on the server
delete Delete resources by filenames, stdin, resources and names, or by resources and label selector
Deploy Commands:
rollout Manage the rollout of a resource
scale Set a new size for a Deployment, ReplicaSet, Replication Controller, or Job
autoscale Auto-scale a Deployment, ReplicaSet, or ReplicationController
Cluster Management Commands:
certificate Modify certificate resources.
cluster-info Display cluster info
top Display Resource (CPU/Memory/Storage) usage.
cordon Mark node as unschedulable
uncordon Mark node as schedulable
drain Drain node in preparation for maintenance
taint Update the taints on one or more nodes
Troubleshooting and Debugging Commands:
describe Show details of a specific resource or group of resources
logs Print the logs for a container in a pod
attach Attach to a running container
exec Execute a command in a container
port-forward Forward one or more local ports to a pod
proxy Run a proxy to the Kubernetes API server
cp Copy files and directories to and from containers.
auth Inspect authorization
Advanced Commands:
diff Diff live version against would-be applied version
apply Apply a configuration to a resource by filename or stdin
patch Update field(s) of a resource using strategic merge patch
replace Replace a resource by filename or stdin
wait Experimental: Wait for a specific condition on one or many resources.
convert Convert config files between different API versions
kustomize Build a kustomization target from a directory or a remote url.
Settings Commands:
label Update the labels on a resource
annotate Update the annotations on a resource
completion Output shell completion code for the specified shell (bash or zsh)
Other Commands:
api-resources Print the supported API resources on the server
api-versions Print the supported API versions on the server, in the form of “group/version”
config Modify kubeconfig files
plugin Provides utilities for interacting with plugins.
version Print the client and server version information
DevOps monitoring tools nagios
Manage Docker configs
create Create a config from a file or STDIN
inspect Display detailed information on one or more configs
ls List configs
rm Remove one or more configs
Manage containers
attach Attach local standard input, output, and error streams to a running container
commit Create a new image from a container’s changes
cp Copy files/folders between a container and the local filesystem
create Create a new container
diff Inspect changes to files or directories on a container’s filesystem
exec Run a command in a running container
export Export a container’s filesystem as a tar archive
inspect Display detailed information on one or more containers
kill Kill one or more running containers
logs Fetch the logs of a container
ls List containers
pause Pause all processes within one or more containers
port List port mappings or a specific mapping for the container
prune Remove all stopped containers
rename Rename a container
restart Restart one or more containers
rm Remove one or more containers
run Run a command in a new container
start Start one or more stopped containers
stats Display a live stream of container(s) resource usage statistics
stop Stop one or more running containers
top Display the running processes of a container
unpause Unpause all processes within one or more containers
update Update configuration of one or more containers
wait Block until one or more containers stop, then print their exit codes
Alternatives, Senu multi-cloud monitoring or Raygun
Aggregate logs into logstash and provide search and filtering via Elastic Search and Kibana. Can also trigger alerts or notifications on specific keyword searches in logs such as WARNING or ERRRO or call_failed. Some common alert scenarios include :
SBC and proxy gateways failures – check states of VM instance
DNS caching alerts – Domain Name System (DNS) caching, a Dynamic Host Configuration Protocol (DHCP) server, router advertisement and network boot alerts from service such as dnsmasq
Disk usage alert – setup alerts for 80% usage and trigger an alarm to either manually prune or create automatic timely archive backups. check the percentage of DISK USAGE
df -h
Mostly it is either the logs file or pcap recorder which need to be archieved in external storage.
Use logrotate – it can rotates, compresses, and mails system logs
config file for logrorate – logrotate -vf /etc/logrotate.conf
Elevated Call failure SIP 503 or Call timeout SIP 408 – high frequency of failed calls indicate an internal issue and must be followed up by smoke testing the entire system to identify any probable issue such as undetected frequent crashes of any individual component or any blacklisting by a destination endpoint etc
sudo tail -f sip.log | grep 503
or
sudo tail -f sip.log | grep WARNING
cron service or processed alerts –
ps axf
PID TTY STAT TIME COMMAND
2 ? S 0:00 [kthreadd]
3 ? I< 0:00 \_ [rcu_gp]
4 ? I< 0:00 \_ [rcu_par_gp]
5 ? I 0:00 \_ [kworker/0:0-eve]
6 ? I< 0:00 \_ [kworker/0:0H-kb]
7 ? I 0:00 \_ [kworker/0:1-eve]
8 ? I 0:00 \_ [kworker/u4:0-nv]
9 ? I< 0:00 \_ [mm_percpu_wq]
10 ? S 0:00 \_ [ksoftirqd/0]
11 ? I 0:00 \_ [rcu_sched]
12 ? S 0:00 \_ [migration/0]
13 ? S 0:00 \_ [cpuhp/0]
14 ? S 0:00 \_ [cpuhp/1]
15 ? S 0:00 \_ [migration/1]
16 ? S 0:00 \_ [ksoftirqd/1]
17 ? I 0:00 \_ [kworker/1:0-eve]
18 ? I< 0:00 \_ [kworker/1:0H-kb]
or checks cron status
service cron status
● cron.service - Regular background program processing daemon
Loaded: loaded (/lib/systemd/system/cron.service; enabled; vendor preset: enabled)
Active: active (running) since Fri 2016-06-26 03:00:37 UTC; 1min 17s ago
Docs: man:cron(8)
Main PID: 845 (cron)
Tasks: 1 (limit: 4383)
CGroup: /system.slice/cron.service
└─845 /usr/sbin/cron -f
Jun 26 03:00:37 ip-172-31-45-21 systemd[1]: Started Regular background program processing daemon.
Jun 26 03:00:37 ip-172-31-45-21 cron[845]: (CRON) INFO (pidfile fd = 3)
Jun 26 03:00:37 ip-172-31-45-21 cron[845]: (CRON) INFO (Running @reboot jobs)
restart or start cron service if required
DB connections / connection pool process – keep listening for any alerts on DB connections failure or even warnings as this can be due to too many read operations such as in DDOS and can escalate very quickly
cron zombie process checks – zombie process or defunct process is a process that has completed execution (via the exit system call) but still has an entry in the process table: it is a process in the “Terminated state”. List xombie process and kill them with pid to free up .
kill -9 <PID1>
Bulk calls checks – consult ongoing call cmd commands for application server such as For Freeswitch use
Incase of DDOS or other macious attacker IP identification block the IP
iptables -I INPUT -s y.y.y.y -j DROP
Can also use fail2ban
>apt-get update && apt-get installfail2ban
Additionally check how many dispatchers are responding on outbound gateway
opensipsctl dispatcher dump
Process control supervisor or pm2 checks – supervisor is a Linux Process Control System that allows its users to monitor and control a number of processes
ps axf | grep supervisor
for pm2
> pm2 status
[PM2] Spawning PM2 daemon with pm2_home=/Users/altanai/.pm2
[PM2] PM2 Successfully daemonized
┌─────┬───────────┬─────────────┬─────────┬─────────┬──────────┬────────┬──────┬───────────┬──────────┬──────────┬──────────┬──────────┐
│ id │ name │ namespace │ version │ mode │ pid │ uptime │ ↺ │ status │ cpu │ mem │ user │ watching │
htop to check memeory and CPU
Health and load on the reverse proxy, load balancer as Nginx – perform a direct curl request to host to check if Nginx responds with a non 4xx / 5xx response or not
curl -v <public-fqdn-of-server>
Incase of error response , restart
/etc/init.d/nginx start
Incase of updates restart ngnix config
nginx -s reload
For HTTP/SSL proxy daemon such as tiny proxy which are used for fast resposne , set the MinSpareServers, MaxSpareServers , MaxClients , MaxRequestsPerChild etc appropriately
VPN checks – restart fireealls or IPsec incase of ssues
/etc/init.d/ipsec restart
Additionally also check ssh service
ps axf | grep sshd
restart sshd if required
SSL cert expiry checks – to keep the operations running securely and prevent and abrupt termination it is a good practise to run regular certificate expiry checks for SSL certs especially on secure HTTP endpoint like APIs , web server and also on SIP applications servers for TLS. If any expiry is due in < 10 days to trigger an alert to renew the certs
Health of Task scheduling services such as RabbitMQ, Celery Distributed Task Queue – remote debugging of these can be set up via pdb which supports setting (conditional) breakpoints and single stepping at the source line level, inspection of stack frames, source code listing, and evaluation of arbitrary Python code in the context of any stack frame.
It can also be set up via using the client libraries provided by these Queue services themselves
Cluster status – setup an efficient health check service which monitors the cluster status for High Availability. JSON object depicting the status of cluster shards
fscli > show status UP 0 years, 0 days, 0 hours, 58 minutes, 33 seconds, 15 milliseconds, 58 microseconds FreeSWITCH (Version 1.6.20 git 987c9b9 2018-01-23 21:49:09Z 64bit) is ready 3 session(s) since startup 0 session(s) - peak 1, last 5min 1 0 session(s) per Sec out of max 30, peak 1, last 5min 1 1000 session(s) max min idle cpu 0.00/80.83 Current Stack Size/Max 240K/8192K
Programming or Syntax error in the production environment – mostly arising due to incomplete QA/testing before pushing new changes to production. Should trigger alerts for dev teams and meet with hot patches.
Many programing application development frameworks have inbuild libs for debugging , exceotion handling and reporting such as
backend service in Django
API service in Go
Distributed memory caching – redis , memcahe : Redis info shows the master -salve configuration for all the instances as well as their memeory and cpu status.
With advent of Voice over IP, the real time streaming of data/audio/video also became critically important to be protected from eavesdropping or modification over the open internet. While Secure Real-time Transport Protocol (SRTP) is a profile of the Real-time Transport Protocol (RTP), which can provide confidentiality, message authentication, and replay protection to the RTP traffic and to the control traffic for RTP, the Real-time Transport Control Protocol (RTCP). ZRTP is a protocol that negotiates the keys and other information required to setup a SRTP audio and video session. To read about RealTime Transport protocol (RTP) , RTP control protocol (RTCP ), before reading about adding security to RTP , RTCP and its feedback use the article link below.
SRTP provides a framework for encryption and message authentication of RTP and RTCP streams by negotiating keys.
It is not a transport but a profile of the Real-time Transport Protocol (RTP) for securing RTP streams in addition to providing confidentiality, integrity protection, source authentication, and replay protection.
The SRTP specification also defines how to setup and maintain a cryptographic context. This context holds all necessary data to perform the security operations, for example the SRTP encryption keys, the packet sequence counters, authentication keys, and so on. Each SRTP session, which is the same as a RTP session, has its own context. Thus a bidirectional SRTP communication requires two different SRTP cryptographic contexts.
It is a framework for encryption and message authentication of RTP and RTCP streams. Offers confidentiality and integrity of the entire RTP and RTCP packets, together with protection against replayed packets. – secure for unicast and multicast RTP applications – low computational cost and small footprint – high throughput and low packet expansion to support bandwidth economy. – permits upgrading with new cryptographic transforms, – protection for heterogeneous environments (mix of wired and wireless networks)
Independant from the underlying transport, network, and physical layers used by RTP, in particular high tolerance to packet loss and re-ordering.
Secure RTCP (SRTCP) is similar to the SRTP format of the SRTCP packet which has the authentication tag and MKI headers, including two additional headers:
SRTCP index
Encrypt-flag
Key management protocols for SRTP
Since SRTP does not contain an integrated key management solution, one can employ any of the following key management protocols
It is a way to negotiate the key/cryptographic parameters for SRTP. Keys are transported in the SDP attachment of a SIP message using TLS transport layer (SSLv3/TLSv1) or other methods like S/MIME.
media attribute defined by SDES is “crypto” a=crypto: inline: [session-parms]
For Secure Communication kamailio supports – Digest SIP User authentication , Authorization via ACL or group membership , IP and Network authentication , TLS support for SIP signaling , transparent handling of SRTP for secure audio , TLS domain name extension support ,authentication and authorization against database (MySQL, PostgreSQL, UnixODBC, BerkeleyDB, Oracle, text files), RADIUS and DIAMETER.
Code to set flag rtp_secure_media to true if both TLS and SRTP are active
Invite from Jitsi client alternatively offering 3 different types of audio SDP’s – RTP/SAVPF , RTP/SAVP and RTP/AVP. Which ever will be accepted by the other endpoint will be communicated back using SDP in 200 OK.
When SRTP it’s critical to not offer or accept variable bit rate codecs, doing so would leak information and possibly compromising SRTP stream. (FS-6404).
Supported SRTP Crypto Suites:
AEAD_AES_256_GCM_8
This algorithm is identical to AEAD_AES_256_GCM (see Section 5.2 of [RFC5116]), except that the tag length, t, is 8, and an authentication tag with a length of 8 octets (64 bits) is used. An AEAD_AES_256_GCM_8 ciphertext is exactly 8 octets longer than its corresponding plaintext.
AEAD_AES_128_GCM_8
This algorithm is identical to AEAD_AES_128_GCM (see Section 5.1 of [RFC5116]), except that the tag length, t, is 8, and an authentication tag with a length of 8 octets (64 bits) is used. An AEAD_AES_128_GCM_8 ciphertext is exactly 8 octets longer than its corresponding plaintext.
AES_CM_128_HMAC_SHA1_80 is the SRTP default AES Counter Mode cipher and HMAC-SHA1 message authentication with an 80-bit authentication tag. The master-key length is 128 bits and has a default lifetime of a maximum of 2^48 SRTP packets or 2^31 SRTCP packets, whichever comes first.
This crypto-suite is identical to AES_CM_128_HMAC_SHA1_80 except that the authentication tag is 32 bits. The length of the base64-decoded key and salt value for this crypto-suite MUST be 30 octets i.e., 240 bits; otherwise, the crypto attribute is considered invalid.
AES_CM_128_NULL_AUTH
The SRTP default cipher (AES-128 Counter Mode), but to use no authentication method. This policy is NOT RECOMMENDED unless it is unavoidable; see Section 7.5 of [RFC3711].
SRTP variables that modify behaviors based on direction/leg:
rtp_secure_media
possible values:
mandatory – Accept/Offer SAVP negotiation ONLY
optional – Accept/Offer SAVP/AVP with SAVP preferred
forbidden – More useful for inbound to deny SAVP negotiation
false – implies forbidden
true – implies mandatory
default if not set is accept SAVP inbound if offered.
This is the same as rtp_secure_media, but would apply to either inbound or outbound offers specifically.
How to specify crypto suites: By default without specifying any crypto suites FreeSWITCH will offer crypto suites from strongest to weakest accepting the strongest each endpoint has in common. If you wish to force specific crypto suites you can do so by appending the suites in a comma separated list in the order that you wish to offer them in.
Optionaly you can use rtp_secure_media_suites to dictate the suite list and only use rtp_secure_media=[optional|mandatory|false|true] without having to dictate the suite list with the rtp_secure_media* variables.
In vars.xml SIP and TLS settings valid options: sslv2,sslv3,sslv23,tlsv1,tlsv1.1,tlsv1.2 default: tlsv1,tlsv1.1,tlsv1.2 . http://wiki.freeswitch.org/wiki/Tls
TLS cipher suite: default ALL:!ADH:!LOW:!EXP:!MD5:@STRENGTH The actual ciphers supported will change per platform. openssl ciphers -v ‘ALL:!ADH:!LOW:!EXP:!MD5:@STRENGTH’ Will show you what is available in your verion of openssl.
A client at 7777777777@ is trying to call 9999999999@ , which freeswitch has to proxy and convert from RTP to SRTP. The following debug logs form sofia external show this process.
[DEBUG] sofia.c:7693 (sofia/external/7777777777@:5080) State Change CS_NEW -> CS_INIT
State NEW
Running State Change CS_INIT (Cur 1 Tot 33)
Standard INIT
State Change CS_INIT -> CS_ROUTING
State INIT going to sleep
Running State Change CS_ROUTING (Cur 1 Tot 33)
Callstate Change DOWN -> RINGING
State ROUTING
send 389 bytes to udp/[]:4642 at 07:08:27.376085:
After the INVITE is received and processed with 100 trying reply, the routing and RTP secure tranformation begins by adding crypto keys and forwarding to destination
Standard EXECUTE
ed5e07ee EXECUTE sofia/external/7777777777@:5080 set(rtp_secure_media=optional)
[rtp_secure_media]=[optional]
ed5e07ee EXECUTE sofia/external/7777777777@:5080 log(INFO Forwarding calls 9999999999@ )
Forwarding calls 9999999999@
…
Set Local audio crypto Key [1 AEAD_AES_256_GCM_8 inline:aHJ1yquBtm4Lzfi2oMpe6cV7IBEy3YgKxrJ3qjvLuRXSuZfHcV4VtVNwHDw]
Set Local video crypto Key [1 AEAD_AES_256_GCM_8 inline:qeJbqlSbnKBNew575hSZ3LX78o6GBsjgOrSMxzGH/zb1E7mkls1Mda93U9w]
Set Local text crypto Key [1 AEAD_AES_256_GCM_8 inline:VghMVsjWQwnOAAjBJ1NTB3jZgfpNV/Yu4poxkAPMqkC7C+fhPKApCJrWg3U]
Set Local audio crypto Key [2 AEAD_AES_128_GCM_8 inline:7XNrjjwC/eOVnWlBSp74DfiIGAEYn/BN+latfA]
Set Local video crypto Key [2 AEAD_AES_128_GCM_8 inline:UQrFpy9Q7L5DI/ww4e5IAmwy7BxSw5yd/T0v0Q]
Set Local text crypto Key [2 AEAD_AES_128_GCM_8 inline:ZqkEPrUFHkaQ+7CROp52H/JO0MbrYWk/Eyl9lQ]
Set Local audio crypto Key [3 AES_CM_256_HMAC_SHA1_80 inline:PTGAm2KlbfuKtIUVGtXknKKzALAzfILZJuPOjfO9S07eWRE6FR0aMUvjuehJgw]
Set Local video crypto Key [3 AES_CM_256_HMAC_SHA1_80 inline:ahHIB0o/dp3SliYWK9BkxM7TfzILwG0bjDn7JuvYi+puRkTM4mYvvsSmywLaYA]
Set Local text crypto Key [3 AES_CM_256_HMAC_SHA1_80 inline:crAs8dPcWJkEEGj5nqTvFGl/TWpxxb86k+dX5gBXhh+q6DO2pEqWNkQmm55aLA]
Set Local audio crypto Key [4 AES_CM_192_HMAC_SHA1_80 inline:SLBJWjgMdfiYX7TUwWQ9CmqUsILLJrpBIVjbfuQmpBIFLvvA/XU]
Set Local video crypto Key [4 AES_CM_192_HMAC_SHA1_80 fNazWgWwNRPjUKNHVqkz44]
Set Local text crypto Key [4 AES_CM_192_HMAC_SHA1_80 inline:hbe9qqETBSK5hRQ8DI9mXL4QAjjGSR8tGDiTHCJF3yxCrRk1ajk]
Set Local audio crypto Key [5 AES_CM_128_HMAC_SHA1_80 inline:8q8mer9N2V4qVxnaazuJeT0KXgW2scONy36J3KaS]
Set Local video crypto Key [5 AES_CM_128_HMAC_SHA1_80 inline:TP5NQ1yB8ZSCCwZMgXur9VHZ5SlpNfnXePj7eZrk]
Set Local text crypto Key [5 AES_CM_128_HMAC_SHA1_80 inline:HT3F3iYG8H/majhBZbOs2Z8ye/WEVGT5Oytx2oQS]
Set Local audio crypto Key [6 AES_CM_256_HMAC_SHA1_32 inline:fEohh92lX2xLmeFYlt8YouM2jN4z5pU05d90BYfoAKU6m4CWv8g8AnifDUKk9A]
Set Local video crypto Key [6 AES_CM_256_HMAC_SHA1_32 inline:+uBNmLcvj41hXoMxNlMNBpq68gU4PmLwYcdopEB/X/jfPElkUgHfguPIgIFJUg]
Set Local text crypto Key [6 AES_CM_256_HMAC_SHA1_32 inline:cqk7D3+KMQ+31R4FFDRRzn/aluyIgjxBL59vfxcsdf5OW9izEJtU+06GewJyIA]
Set Local audio crypto Key [7 AES_CM_192_HMAC_SHA1_32 inline:Tv25TfP9fQZ+ljs/tFlHohkckiK4F6cemzEjHSvo2+q6No4ai+o]
Set Local video crypto Key [7 AES_CM_192_HMAC_SHA1_32 inline:CY/Dizd1QrlobZtgnigr0hWE+oDSx4S1F51Zpo4aZamN+8ZMdp8]
Set Local text crypto Key [7 AES_CM_192_HMAC_SHA1_32 inline:aEox/7IMps5c+uOWbosZ618+opkJV/GnrKc2EnAhVnDNeo91+No]
Set Local audio crypto Key [8 AES_CM_128_HMAC_SHA1_32 inline:0LwKGyljIed0zhukiMMyD5ive0ZsyybwBrnevcAv]
Set Local video crypto Key [8 AES_CM_128_HMAC_SHA1_32 inline:eZN8rAG8UPPntdYxsg1kkWL4qMsVgTiGGiS4UeUM]
Set Local text crypto Key [8 AES_CM_128_HMAC_SHA1_32 inline:bAYzbfr+El8usaTkPBR6iFuTda4uLNGjyx9lQWkX]
Set Local audio crypto Key [9 AES_CM_128_NULL_AUTH inline:5m3142gGG1HZ5VnoXsAOyopSwDCYbrIsGpdbEO3D]
Set Local video crypto Key [9 AES_CM_128_NULL_AUTH inline:zXk67wjwRhSilq0kiz5TWxXqrxuTaWTA3qqbVo/G]
Set Local text crypto Key [9 AES_CM_128_NULL_AUTH inline:FRP9CJbBO+PRj6I9RSBAiMxRZ/qFtyrEXPfxocG0]
sending invite version: 1.9.0 -742-8f1b7e0 64bit
Local SDP:
v=0
o=FreeSWITCH 1552960557 1552960558 IN IP4
s=FreeSWITCH
c=IN IP4
t=0 0
m=audio 18750 RTP/SAVP 102 9 0 8 103 101
a=rtpmap:102 opus/48000/2
a=fmtp:102 useinbandfec=1; maxaveragebitrate=30000; maxplaybackrate=48000; ptime=20; minptime=10; maxptime=40; stereo=1
a=rtpmap:9 G722/8000
a=rtpmap:0 PCMU/8000
a=rtpmap:8 PCMA/8000
a=rtpmap:103 telephone-event/48000
a=fmtp:103 0-16
a=rtpmap:101 telephone-event/8000
a=fmtp:101 0-16
a=crypto:1 AEAD_AES_256_GCM_8 inline:aHJ1yquBtm4Lzfi2oMpe6cV7IBEy3YgKxrJ3qjvLuRXSuZfHcV4VtVNwHDw
a=crypto:2 AEAD_AES_128_GCM_8 inline:7XNrjjwC/eOVnWlBSp74DfiIGAEYn/BN+latfA
a=crypto:3 AES_CM_256_HMAC_SHA1_80 inline:PTGAm2KlbfuKtIUVGtXknKKzALAzfILZJuPOjfO9S07eWRE6FR0aMUvjuehJgw
a=crypto:4 AES_CM_192_HMAC_SHA1_80 inline:SLBJWjgMdfiYX7TUwWQ9CmqUsILLJrpBIVjbfuQmpBIFLvvA/XU
a=crypto:5 AES_CM_128_HMAC_SHA1_80 inline:8q8mer9N2V4qVxnaazuJeT0KXgW2scONy36J3KaS
a=crypto:6 AES_CM_256_HMAC_SHA1_32 inline:fEohh92lX2xLmeFYlt8YouM2jN4z5pU05d90BYfoAKU6m4CWv8g8AnifDUKk9A
a=crypto:7 AES_CM_192_HMAC_SHA1_32 inline:Tv25TfP9fQZ+ljs/tFlHohkckiK4F6cemzEjHSvo2+q6No4ai+o
a=crypto:8 AES_CM_128_HMAC_SHA1_32 inline:0LwKGyljIed0zhukiMMyD5ive0ZsyybwBrnevcAv
a=crypto:9 AES_CM_128_NULL_AUTH inline:5m3142gGG1HZ5VnoXsAOyopSwDCYbrIsGpdbEO3D
a=ptime:20
a=sendrecv
Once the SDP is ready with crypto keys it is the forwarded to the next_up
can establish multiple security contexts or cryptographic sessions with a single message. Can be used in p2p or bradcast scenarios where one entity generates the key and needs to distribute the key to a number of participants.
SIPp is an opensource (GNU GPL license) performance testing tool for the SIP protocol and is widely used for Quality assurabce of callflows in voip applications for UAC / UASs cenarios.
It can emulate functioing of a sip phone such as REGISTER , establishes and releases multiple calls with the INVITE and BYE methods , send other SIP requests and wait for reponses based on dafult of custom xml scenario files.
Plus factor is the dynamic display of statistics about running tests (call rate, round trip delay, and message statistics), periodic CSV statistics dumps, TCP and UDP over multiple sockets or multiplexed with retransmission management, regular expressions and variables in scenario files, and dynamically adjustable call rates.
It is widley used as aperformnace and load testing tool since it can test SIP equipements like SIP proxies, B2BUAs, SIP media servers, SIP/x gateways, and SIP PBXes and can also emulate thousands of user agents calling your SIP system.
More on SIPp scripts and various exmaples can be read from
Pre-requisites to compile SIPp are: – C++ Compiler – curses or ncurses library – For TLS support: OpenSSL >= 0.9.8 – For pcap play support: libpcap and libnet – For SCTP support: lksctp-tools – For distributed pauses: Gnu Scientific Libraries
On the same host, run sipp with embedded client (uac) scenario:
sipp -sn uac 127.0.0.1 -trace_msg -trace_err
output for server
# sipp -sn uas
------------------------------ Scenario Screen -------- [1-9]: Change Screen --
Port Total-time Total-calls Transport
5060 32.95 s 61 UDP
0 new calls during 0.874 s period 1 ms scheduler resolution
19 calls Peak was 41 calls, after 28 s
0 Running, 63 Paused, 12 Woken up
0 dead call msg (discarded)
3 open sockets
———————————————– 2019-02-04 13:08:13.940159
UDP message sent (371 bytes):
ACK sip:service@127.0.0.1:5060 SIP/2.0
Via: SIP/2.0/UDP 192.x.x.x:5061;branch=z9hG4bK-52422-1-5
From: sipp ;tag=52422SIPpTag001
To: service ;tag=52416SIPpTag011
Call-ID: 1-52422@192.x.x.x
CSeq: 1 ACK
Contact: sip:sipp@192.x.x.x:5061
Max-Forwards: 70
Subject: Performance Test
Content-Length: 0
~ RTP
———————————————– 2019-02-04 13:08:13.941658
UDP message sent (371 bytes):
BYE sip:service@127.0.0.1:5060 SIP/2.0
Via: SIP/2.0/UDP 192.x.x.x:5061;branch=z9hG4bK-52422-1-7
From: sipp ;tag=52422SIPpTag001
To: service ;tag=52416SIPpTag011
Call-ID: 1-52422@192.x.x.x
CSeq: 2 BYE
Contact: sip:sipp@192.x.x.x:5061
Max-Forwards: 70
Subject: Performance Test
Content-Length: 0
———————————————– 2019-02-04 13:08:13.952888
UDP message received [313] bytes :
SIP/2.0 200 OK
Via: SIP/2.0/UDP 192.x.x.x:5061;branch=z9hG4bK-52422-1-7
From: sipp ;tag=52422SIPpTag001
To: service ;tag=52416SIPpTag011
Call-ID: 1-52422@192.x.x.x
CSeq: 2 BYE
Contact:
Content-Length: 0
Time
---------------------------- Repartition Screen ------- [1-9]: Change Screen --
Average Response Time Repartition 1
0 ms <= n < 10 ms : 293
10 ms <= n < 20 ms : 9
20 ms <= n < 30 ms : 0
30 ms <= n < 40 ms : 0
40 ms <= n < 50 ms : 0
50 ms <= n < 100 ms : 0
100 ms <= n < 150 ms : 0
150 ms <= n < 200 ms : 0
n >= 200 ms : 0
Average Call Length Repartition
0 ms <= n < 10 ms : 0
10 ms <= n < 50 ms : 0
50 ms <= n < 100 ms : 0
100 ms <= n < 500 ms : 0
500 ms <= n < 1000 ms : 0
1000 ms <= n < 5000 ms : 262
5000 ms <= n < 10000 ms : 0
n >= 10000 ms : 0
------------------------------ Sipp Server Mode -------------------------------
3 new calls during 0.286 s period 1 ms scheduler resolution
0 calls (limit 30) Peak was 25 calls, after 10 s
0 Running, 101 Paused, 7 Woken up
0 dead call msg (discarded) 0 out-of-call msg (discarded)
3 open sockets
----------------------------- Statistics Screen ------- [1-9]: Change Screen --
Start Time | 2019-02-04 13:08:03.908208 1549265883.908208
Last Reset Time | 2019-02-04 13:08:20.954289 1549265900.954289
Current Time | 2019-02-04 13:08:21.241152 1549265901.241152
-------------------------+---------------------------+--------------------------
Counter Name | Periodic value | Cumulative value
-------------------------+---------------------------+--------------------------
Elapsed Time | 00:00:00:286000 | 00:00:17:332000
Call Rate
Tracings
———————————————– 2019-02-04 13:08:13.934840
UDP message received [527] bytes :
INVITE sip:service@127.0.0.1:5060 SIP/2.0
Via: SIP/2.0/UDP 192.x.x.x:5061;branch=z9hG4bK-52422-1-0
From: sipp ;tag=52422SIPpTag001
To: service
Call-ID: 1-52422@192.x.x.x
CSeq: 1 INVITE
Contact: sip:sipp@192.x.x.x:5061
Max-Forwards: 70
Subject: Performance Test
Content-Type: application/sdp
Content-Length: 135
———————————————– 2019-02-04 13:08:13.948679
UDP message received [371] bytes :
ACK sip:service@127.0.0.1:5060 SIP/2.0
Via: SIP/2.0/UDP 192.x.x.x:5061;branch=z9hG4bK-52422-1-5
From: sipp ;tag=52422SIPpTag001
To: service ;tag=52416SIPpTag011
Call-ID: 1-52422@192.x.x.x
CSeq: 1 ACK
Contact: sip:sipp@192.x.x.x:5061
Max-Forwards: 70
Subject: Performance Test
Content-Length: 0
~ RTP
———————————————– 2019-02-04 13:08:13.949168
UDP message received [371] bytes :
BYE sip:service@127.0.0.1:5060 SIP/2.0
Via: SIP/2.0/UDP 192.x.x.x:5061;branch=z9hG4bK-52422-1-7
From: sipp ;tag=52422SIPpTag001
To: service ;tag=52416SIPpTag011
Call-ID: 1-52422@192.x.x.x
CSeq: 2 BYE
Contact: sip:sipp@192.x.x.x:5061
Max-Forwards: 70
Subject: Performance Test
Content-Length: 0
———————————————– 2019-02-04 13:08:13.949245
UDP message sent (313 bytes):
SIP/2.0 200 OK
Via: SIP/2.0/UDP 192.x.x.x:5061;branch=z9hG4bK-52422-1-7
From: sipp ;tag=52422SIPpTag001
To: service ;tag=52416SIPpTag011
Call-ID: 1-52422@192.x.x.x
CSeq: 2 BYE
Contact:
Content-Length: 0
time
---------------------------- Repartition Screen ------- [1-9]: Change Screen --
Average Response Time Repartition 1
0 ms <= n < 10 ms : 657
10 ms <= n < 20 ms : 20
20 ms <= n < 30 ms : 0
30 ms <= n < 40 ms : 0
40 ms <= n < 50 ms : 0
50 ms <= n < 100 ms : 0
100 ms <= n < 150 ms : 0
150 ms <= n < 200 ms : 0
n >= 200 ms : 0
Average Call Length Repartition
0 ms <= n < 10 ms : 649
10 ms <= n < 50 ms : 28
50 ms <= n < 100 ms : 0
100 ms <= n < 500 ms : 0
500 ms <= n < 1000 ms : 0
1000 ms <= n < 5000 ms : 0
5000 ms <= n < 10000 ms : 0
n >= 10000 ms : 0
------ [+|-|*|/]: Adjust rate ---- [q]: Soft exit ---- [p]: Pause traffic -----
Last Error: Overload warning: the major watchdog timer 3000ms has been t…
Run SIPp with embedded server (uas) scenario: ./sipp -sn uas On the same host, run SIPp with embedded client (uac) scenario: ./sipp -sn uac 127.0.0.1
Scenario file options:
-sd : Dumps a default scenario (embedded in the SIPp executable)
-sf : Loads an alternate XML scenario file. To learn more about XML scenario syntax, use the -sd option to dump embedded scenarios. They contain all the necessary help.
-oocsf : Load out-of-call scenario.
-oocsn : Load out-of-call scenario.
-sn : Use a default scenario (embedded in the SIPp executable). If this option is omitted, the Standard SipStone UAC scenario is loaded. Available values in this version:
‘uac’ : Standard SipStone UAC (default).
‘uas’ : Simple UAS responder.
‘regexp’ : Standard SipStone UAC – with regexp and variables.
‘branchc’ : Branching and conditional branching in scenarios – client.
‘branchs’ : Branching and conditional branching in scenarios – server.
Default 3pcc scenarios (see -3pcc option):
‘3pcc-C-A’ : Controller A side (must be started after all other 3pcc scenarios)
‘3pcc-C-B’ : Controller B side.
‘3pcc-A’ : A side.
‘3pcc-B’ : B side.
IP, port and protocol options
-t : Set the transport mode:
u1: UDP with one socket (default),
un: UDP with one socket per call,
ui: UDP with one socket per IP address. The IP addresses must be defined in the injection file.
t1: TCP with one socket,
tn: TCP with one socket per call,
l1: TLS with one socket,
ln: TLS with one socket per call,
c1: u1 + compression (only if compression plugin loaded),
cn: un + compression (only if compression plugin loaded). This plugin is not provided with SIPp.
-i : Set the local IP address for ‘Contact:’,’Via:’, and ‘From:’ headers. Default is primary host IP address.
-p : Set the local port number. Default is a random free port chosen by the system
-bind_local : Bind socket to local IP address, i.e. the local IP address is used as the source IP address. If SIPp runs in server mode it will only listen on the local IP address instead of all IP addresses.
-ci : Set the local control IP address
-cp : Set the local control port number. Default is 8888.
-max_socket : Set the max number of sockets to open simultaneously. This option is significant if you use one socket per call. Once this limit is reached, traffic is distributed over the sockets already opened. Default value is 50000
-max_reconnect : Set the the maximum number of reconnection.
-reconnect_close : Should calls be closed on reconnect?
-reconnect_sleep : How long (in milliseconds) to sleep between the close and reconnect?
-rsa : Set the remote sending address to host:port for sending the messages.
-tls_cert : Set the name for TLS Certificate file. Default is ‘cacert.pem
-tls_key : Set the name for TLS Private Key file. Default is ‘cakey.pem’
-tls_ca : Set the name for TLS CA file. If not specified, X509 verification is not activated.
-tls_crl : Set the name for Certificate Revocation List file. If not specified, X509 CRL is not activated.
-tls_version : Set the TLS protocol version to use (1.0, 1.1, 1.2) — default is autonegotiate
SIPp overall behavior options:
-v : Display version and copyright information.
-bg : Launch SIPp in background mode.
-nostdin : Disable stdin.
-plugin : Load a plugin.
-sleep : How long to sleep for at startup. Default unit is seconds.
-skip_rlimit : Do not perform rlimit tuning of file descriptor limits. Default: false.
-buff_size : Set the send and receive buffer size.
-sendbuffer_warn : Produce warnings instead of errors on SendBuffer failures.
-lost : Set the number of packets to lose by default (scenario specifications override this value).
-key : keyword value Set the generic parameter named “keyword” to “value”.
-set : variable value Set the global variable parameter named “variable” to “value”.
-tdmmap : Generate and handle a table of TDM circuits. A circuit must be available for the call to be placed. Format: -tdmmap {0-3}{99}{5-8}{1-31}
-dynamicStart : variable value Set the start offset of dynamic_id variable
-dynamicMax : variable value Set the maximum of dynamic_id variable
-dynamicStep : variable value Set the increment of dynamic_id variable
Call behavior options:
-aa : Enable automatic 200 OK answer for INFO, NOTIFY, OPTIONS and UPDATE.
-base_cseq : Start value of [cseq] for each call.
-cid_str : Call ID string (default %u-%p@%s). %u=call_number, %s=ip_address, %p=process_number, %%=% (in any order).
-d : Controls the length of calls. More precisely, this controls the duration of ‘pause’ instructions in the scenario, if they do not have a ‘milliseconds’ section. Default value is 0 and default unit is milliseconds.
-deadcall_wait : How long the Call-ID and final status of calls should be kept to improve message and error logs (default unit is ms).
-auth_uri : Force the value of the URI for authentication. By default, the URI is composed of remote_ip:remote_port.
-au : Set authorization username for authentication challenges. Default is taken from -s argument
-ap : Set the password for authentication challenges. Default is ‘password’
-s : Set the username part of the request URI. Default is ‘service’.
-default_behaviors: Set the default behaviors that SIPp will use. Possible values are:
all Use all default behaviors
none Use no default behaviors
bye Send byes for aborted calls
abortunexp Abort calls on unexpected messages
pingreply Reply to ping requests If a behavior is prefaced with a -, then it is turned off. Example: all,-bye
-nd : No Default. Disable all default behavior of SIPp which are the following:
On UDP retransmission timeout, abort the call by sending a BYE or a CANCEL
On receive timeout with no ontimeout attribute, abort the call by sending a BYE or a CANCEL
On unexpected BYE send a 200 OK and close the call
On unexpected CANCEL send a 200 OK and close the call
On unexpected PING send a 200 OK and continue the call
On any other unexpected message, abort the call by sending a BYE or a CANCEL
-pause_msg_ign : Ignore the messages received during a pause defined in the scenario
-callid_slash_ign: Don’t treat a triple-slash in Call-IDs as indicating an extra SIPp prefix.
Injection file options:
-inf : Inject values from an external CSV file during calls into the scenarios. First line of this file say whether the data is to be read in sequence (SEQUENTIAL), random (RANDOM), or user (USER) order. Each line corresponds to one call and has one or more ‘;’ delimited data fields. Those fields can be referred as [field0], [field1], … in the xml scenario file. Several CSV files can be used simultaneously (syntax: -inf f1.csv -inf f2.csv …)
-infindex : file field Create an index of file using field. For example -inf ../path/to/users.csv -infindex users.csv 0 creates an index on the first key.
-ip_field : Set which field from the injection file contains the IP address from which the client will send its messages. If this option is omitted and the ‘-t ui’ option is present, then field 0 is assumed. Use this option together with ‘-t ui’
RTP behaviour options:
-mi : Set the local media IP address (default: local primary host IP address)
-rtp_echo : Enable RTP echo. RTP/UDP packets received on port defined by -mp are echoed to their sender. RTP/UDP packets coming on this port + 2 are also echoed to their sender (used for sound and video echo).
-mb : Set the RTP echo buffer size (default: 2048).
-mp : Set the local RTP echo port number. Default is 6000.
-rtp_payload : RTP default payload type.
-rtp_threadtasks : RTP number of playback tasks per thread.
-rtp_buffsize : Set the rtp socket send/receive buffer size.
Call rate options:
-r : Set the call rate (in calls per seconds). This value can bechanged during test by pressing ‘+’, ‘_’, ‘*’ or ‘/’. Default is 10.
pressing ‘+’ key to increase call rate by 1 * rate_scale,
pressing ‘-‘ key to decrease call rate by 1 * rate_scale,
pressing ‘*’ key to increase call rate by 10 * rate_scale,
pressing ‘/’ key to decrease call rate by 10 * rate_scale.
-rp : Specify the rate period for the call rate. Default is 1 second and default unit is milliseconds. This allows you to have n calls every m milliseconds(by using -r n -rp m). Example: -r 7 -rp 2000 ==> 7 calls every 2 seconds. -r 10 -rp 5s => 10 calls every 5 seconds.
-rate_scale : Control the units for the ‘+’, ‘-‘, ‘*’, and ‘/’ keys.
-rate_increase : Specify the rate increase every -rate_interval units (default is seconds). This allows you to increase the load for each independent logging period. Example: -rate_increase 10 -rate_interval 10s ==> increase calls by 10 every 10 seconds.
-rate_max :
If -rate_increase is set, then quit after the rate reaches this value. Example: -rate_increase 10 -rate_max 100 ==> increase calls by 10 until 100 cps is hit.
-rate_interval : Set the interval by which the call rate is increased. Defaults to the value of -fd.
-no_rate_quit : If -rate_increase is set, do not quit after the rate reaches -rate_max.
-l : Set the maximum number of simultaneous calls. Once this limit is reached, traffic is decreased until the number of open calls goes down. Default: (3 * call_duration (s) * rate).
-m : Stop the test and exit when ‘calls’ calls are processed
-users : Instead of starting calls at a fixed rate, begin ‘users’ calls at startup, and keep the number of calls constant.
Retransmission and timeout options:
-recv_timeout : Global receive timeout. Default unit is milliseconds. If the expected message is not received, the call times out and is aborted.
-send_timeout : Global send timeout. Default unit is milliseconds. If a message is not sent (due to congestion), the call times out and is aborted.
-timeout : Global timeout. Default unit is seconds. If this option is set, SIPp quits after nb units (-timeout 20s quits after 20 seconds).
-timeout_error : SIPp fails if the global timeout is reached is set (-timeout option required).
-max_retrans : Maximum number of UDP retransmissions before call ends on timeout. Default is 5 for INVITE transactions and 7 for others.
-max_invite_retrans: Maximum number of UDP retransmissions for invite transactions before call ends on timeout.
-max_non_invite_retrans: Maximum number of UDP retransmissions for non-invite transactions before call ends on timeout.
-nr : Disable retransmission in UDP mode.
-rtcheck : Select the retransmission detection method: full (default) or loose.
-T2 : Global T2-timer in milli seconds
Third-party call control options:
-3pcc : Launch the tool in 3pcc mode (“Third Party call control”). The passed IP address depends on the 3PCC role.
When the first twin command is ‘sendCmd’ then this is the address of the remote twin socket. SIPp will try to connect to this address:port to send the twin command (This instance must be started after all other 3PCC scenarios). Example: 3PCC-C-A scenario.
When the first twin command is ‘recvCmd’ then this is the address of the local twin socket. SIPp will open this address:port to listen for twin command. Example: 3PCC-C-B scenario.
-master : 3pcc extended mode: indicates the master number
-slave : 3pcc extended mode: indicates the slave number
-slave_cfg : 3pcc extended mode: indicates the file where the master and slave addresses are stored
Performance and watchdog options:
-timer_resol Set the timer resolution. Default unit is milliseconds. This option has an impact on timers precision.Small values allow more precise scheduling but impacts CPU usage.If the compression is on, the value is set to 50ms. The default value is 10ms.
-max_recv_loops Set the maximum number of messages received read per cycle. Increase this value for high traffic level. The default value is 1000.
-max_sched_loops Set the maximum number of calls run per event loop. Increase this value for high traffic level. The default value is 1000.
-watchdog_interval : Set gap between watchdog timer firings. Default is 400.
-watchdog_reset : If the watchdog timer has not fired in more than this time period, then reset the max triggers counters. Default is 10 minutes.
-watchdog_minor_threshold: If it has been longer than this period between watchdog executions count a minor trip. Default is 500.
-watchdog_major_threshold: If it has been longer than this period between watchdog executions count a major trip. Default is 3000.
-watchdog_major_maxtriggers : How many times the major watchdog timer can be tripped before the test is terminated. Default is 10.
-watchdog_minor_maxtriggers: How many times the minor watchdog timer can be tripped before the test is terminated. Default is 120.
Tracing, logging and statistics options:
-f : Set the statistics report frequency on screen. Default is 1 and default unit is seconds.
-trace_stat : Dumps all statistics in <scenario_name>_.csv file. Use the ‘-h stat’ option for a detailed description of the statistics file content.
-stat_delimiter : Set the delimiter for the statistics file
-stf : Set the file name to use to dump statistics
-fd : Set the statistics dump log report frequency. Default is 60 and default unit is seconds.
-periodic_rtd : Reset response time partition counters each logging interval.
-trace_msg : Displays sent and received SIP messages in __messages.log
-message_file : Set the name of the message log file.
-message_overwrite: Overwrite the message log file (default true).
-trace_shortmsg : Displays sent and received SIP messages as CSV in <scenario file name>__shortmessages.log
-shortmessage_file: Set the name of the short message log file.
-shortmessage_overwrite: Overwrite the short message log file (default true).
-trace_counts : Dumps individual message counts in a CSV file.
-trace_err : Trace all unexpected messages in __errors.log.
-error_file : Set the name of the error log file.
-error_overwrite : Overwrite the error log file (default true).
-trace_error_codes: Dumps the SIP response codes of unexpected messages to <scenario file name>__error_codes.log.
-trace_calldebug : Dumps debugging information about aborted calls to <scenario_name>__calldebug.log file.
-calldebug_file : Set the name of the call debug file.
-calldebug_overwrite: Overwrite the call debug file (default true).
-trace_screen : Dump statistic screens in the <scenario_name>__screens.log file when quitting SIPp. Useful to get a final status report in background mode (-bg option).
-screen_file : Set the name of the screen file.
-screen_overwrite: Overwrite the screen file (default true).
-trace_rtt : Allow tracing of all response times in __rtt.csv.
-rtt_freq : freq is mandatory. Dump response times every freq calls in the log file defined by -trace_rtt. Default value is 200.
-trace_logs : Allow tracing of actions in __logs.log.
-log_file : Set the name of the log actions log file.
-log_overwrite : Overwrite the log actions log file (default true).
-ringbuffer_files: How many error, message, shortmessage and calldebug files should be kept after rotation?
-ringbuffer_size : How large should error, message, shortmessage and calldebug files be before they get rotated?
-max_log_size : What is the limit for error, message, shortmessage and calldebug file sizes.
Signal handling:
SIPp can be controlled using POSIX signals. The following signals are handled: USR1: Similar to pressing the ‘q’ key. It triggers a soft exit of SIPp. No more new calls are placed and all ongoing calls are finished before SIPp exits. Example: kill -SIGUSR1 732 USR2: Triggers a dump of all statistics screens in <scenario_name>__screens.log file. Especially useful in background mode to know what the current status is. Example: kill -SIGUSR2 732
Exit codes:
Upon exit (on fatal error or when the number of asked calls (-m option) is reached, SIPp exits with one of the following exit code: 0: All calls were successful 1: At least one call failed 97: Exit on internal command. Calls may have been processed 99: Normal exit without calls processed -1: Fatal error -2: Fatal error binding a socket
Debugging
Issue1 The commonName field needed to be supplied and was missing
Solution Given the common name while generating the certs
Issue2 If cmake error appears such as “command not found: cmake” then
GStreamer ( LGPL )ia a media handling library written in C for applicatioan such as streaming , recording, playback , mixing and editing attributes etc. Even enhnaced applicaiosn such as tsrancoding , media ormat conversion , streaming servers for embeeded devices ( read more about Gstreamer in RPi in my srticle here).
It encompases various codecs, filters and is modular with plugins developement to enhance its capabilities. Media Streaming application developers use it as part of their framework at either the broadcaster’s end or as media player.
Streaming / broadcasting Live Video call to non webrtc supported browsers and media players
attempts of streaming / broadcasting Live Video WebRTC call to non WebRTC supported browsers and media players such as VLC , ffplay , default video player in Linux etc .
continue : Streaming / broadcasting Live Video call to non webrtc supported browsers and media players
httontinuation to the attempts / outcomes and problems in building a WebRTC to RTP media framework that successfully stream / broadcast WebRTC content to non webrtc supported browsers ( safari / IE ) / media players ( VLC )
gstreamer-audio-1.0 GStreamer Audio library – Audio helper functions and base classes
gstreamer-plugins-bad-1.0 GStreamer Bad Plugin libraries – Streaming media framework, bad plugins libraries
gstreamer-rtsp-server-1.0 gst-rtsp-server – GStreamer based RTSP server
At the time of writing this article Gstreamer an much early version in 1.X , which was newer than its then stable version 0.x. Since then the library has updated many fold. summarising release highlights for major versions as the blog was updated over time .
Project : Making and IP survillance system using gstreamer and Janus
To build a turn-key easily deployable surveillance solution
Features :
Paring of Android Mobile with box
Live streaming from Box to Android
Video Recording inside the box
Auto parsing of recorded video around motion detection
Event listeners
2 way audio
Inbuild Media Control Unit
Efficient use of bandwidth
Secure session while live-streaming
Modules
Authentication ( OTP / username- password)
Livestreaming on Opus / vp8
Session Security and keepalives for live-streaming sessions
Sync local videos to cloud storage
Record and playback with timeline and events
Parsing and restructuring video ( transcoding may also be required )
Coturn server for NAT and ICE
Web platform on box ( user interface )+ NoSQL
Web platform on Cloud server ( Admin interface )+ NoSQL
REST APIs for third party add-ons ( Node based )
Android demo app for receiving the live stream and feeds
Varrying experiments and working gstreamer commands
Local Network Stream
To create /dev/video0
modprobe bcm2835-v4l2
To stream on rtspserver using rpicamsrc using h264 parse
Contains code for Android and ios Publishers , players on various platforms including HLS and Flash , streamings servers , Wowza playing mosules , webrtc broadcast
Gstreamer 1.8.0 – 24 March 2016
Features Hardware-accelerated zero-copy video decoding on Android
New video capture source for Android using the android.hardware.Camera API
Windows Media reverse playback support (ASF/WMV/WMA)
tracing system provides support for more sophisticated debugging tools
high-level GstPlayer playback convenience API
Initial support for the new Vulkan API
Improved Opus audio codec support: Support for more than two channels; MPEG-TS demuxer/muxer can handle Opus; sample-accurate encoding/decoding/transmuxing with Ogg, Matroska, ISOBMFF (Quicktime/MP4), and MPEG-TS as container; new codec utility functions for Opus header and caps handling in pbutils library. The Opus encoder/decoder elements were also moved to gst-plugins-base (from -bad), and the opus RTP depayloader/payloader to -good.
Asset proxy support in the GStreamer Editing Services
GStreamer 1.16.0 – 19 April 2019.
GStreamer WebRTC stack gained support for data channels for peer-to-peer communication based on SCTP, BUNDLE support, as well as support for multiple TURN servers.
AV1 video codec support for Matroska and QuickTime/MP4 containers and more configuration options and supported input formats for the AOMedia AV1 encoder
Closed Captions and other Ancillary Data in video
planar (non-interleaved) raw audio
GstVideoAggregator, compositor and OpenGL mixer elements are now in -base
New alternate fields interlace mode where each buffer carries a single field
WebM and Matroska ContentEncryption support in the Matroska demuxer
new WebKit WPE-based web browser source element
Video4Linux: HEVC encoding and decoding, JPEG encoding, and improved dmabuf import/export
Hardware-accelerated Nvidia video decoder gained support for VP8/VP9 decoding, whilst the encoder gained support for H.265/HEVC encoding.
Improvements to the Intel Media SDK based hardware-accelerated video decoder and encoder plugin (msdk): dmabuf import/export for zero-copy integration with other components; VP9 decoding; 10-bit HEVC encoding; video post-processing (vpp) support including deinterlacing; and the video decoder now handles dynamic resolution changes.
ASS/SSA subtitle overlay renderer can now handle multiple subtitles that overlap in time and will show them on screen simultaneously
Meson build feature-complete (with the exception of plugin docs) and it is now the recommended build system on all platforms. The Autotools build is scheduled to be removed in the next cycle.
GStreamer Rust bindings and Rust plugins module
GStreamer Editing Services allows directly playing back serialized edit list with playbin or (uri)decodebin
Market trends are not in favour of Telecom Service /providers with increasing use of OTT ( Over The Top ) applications like WhatsApp, Facebook messenger, Google hangouts, skype, Viber, etc. OTT applications are often blamed to take a stake in voice traffic revenue by using IP calls where the telco could’ve charged based on its rate plan of call seconds. This especially intensifies for long-distance or international calls where customers can use OTT providers instead of expensive telco rate plans.
What is an OTT ?
An Over The Top ( OTT ) application is one which provides communication services over Internet . Therefore these bypass the communication billing system setup by a Telecom Operator , resulting in no gain or loss of revenue to Telecom Operator who is providing the Internet service to user in first place .
Hence we see that OTT are major source of concern for Telecom Operators whose traditional and obviously expensive ( when compared to OTTs free service ) billing models are facing disruption .
Telecom Regulatory bodies around the world
The telecom regulatory authorities in some of the countries are for example listed as :
Canadian Radio-television and Telecommunications Commission (CRTC) – Canada
Ministry of Information Industry (MII) – China
Autorité de Régulation des Communications Électroniques et des Postes (ARCEP) – France
Bundesnetzagentur (BNA) – Germany
Telecom Regulatory Authority of India (TRAI) – India
Ministry for Communications and Informatization of the Russian Federation (Minsvyaz) – Russia
Infocomm Development Authority of Singapore (IDA) – Singapore
Independent Communications Authority of South Africa (ICASA) – south Africa
Federal Communications Commission (FCC) , National Association of Regulatory Utility Commissioners (regulators of individual states) (NARUC) , CTIA – The Wireless Association (CTIA) – USA
Such telecom regulatory bodies get to decide whether to enforce differential price to end consumers for using OTT so that telecom service providers can benefit or keep the Internet fair and open by passing Net Neutrality Laws and Bills and amendments .
What is Net Neaurality ?
The fundamental principle of Net Neurality is that Telecom Operators should not block , slow down or charge consumers extra for using other services as their means of communication. This states that it is wrong to charge users above the regular data rates for using VOIP apps and other internet based communication services.
The following counteries have adopted principles of Net Neutrality by passing bills or making law .
Chile – Chile’s General Law of Telecommunications, “No [ISP] can block, interfere with, discriminate, hinder, nor restrict the right of any Internet user of using, send, receive, or offer any content, application, or legitimate service through the Internet, as well as any activity or legitimate use conducted through the Internet.”
Brazil – ” Internet Bill of Rights ” makes equal access to internet mandatory in Brazil .
Netherlands – Even European Union has adopted Netherlands’ Net Neutrality amendment which reads “traffic should be treated equally, without discrimination, restriction or interference, independent of the sender, receiver, type, content, device, service or application.”
USA – Citizens make ‘We the People’ platform to ‘Restore Net Neutrality By Directing the Federal Communications Commission (FCC) to Classify Internet Providers as ‘Common Carriers‘. Therefore not allowing them to either throttle speed by paid prioritization , discriminate in pricing or block any broadband access to legal content . Above facts are from this tech.firstpost.com article.
Inspite of the fact that I Support Net Neutrality with all my heart , as a telecom engineer I understand the cost investment made by Telecom operators in providing am efficient communication network to its subscribers ( Access , Network and Application layers ). Therefor I do have my sympathies with the Telcos and to level out the wide ranging conflict between Telcos and ISP ( Internet Service Providers ) , I pen down the following points which reflect the Telecom Operators Problems and also highlight the solutions that can be adopted to counteract the OTT threat .
Depleting revenue for Telco
Messaging – OTT messaging cost operators $13.9 billion, or 9% of message revenue in 2013
Voice – Voice services under threat from VOIP services like Skype, Viber
OTT apps – Voice & Message apps have been the operator’s biggest headache. Its time Operator should launch its own OTT Services
Data Traffic – The utilization is yet to reach its peak. Will face challenges from WiFi access
Critical Pain areas – Erosion of Operator’s revenue from voice and (especially) messaging
Telco’s OTT Application
At this stage, a telecom Service provider / Operator must enter the apps market and bring forth a Messenger which is more powerful, interactive and awesome than an OTT application. Fortunately, the Operator can always couple this application with his background telecom infrastructure to provide the edge in performance and functionalities.
Road block while developing a OTT application for a Telecom Service Provider :
Investment in Data Network is not being utilized due to lack of service
Reuse of Existing business Logic and extending the service reach across devices and networks is tough
Operator already has full fledged network Infrastructure in Place
Desire for minimum CAPEX while investing in new technologies
compete with OTT players and open new revenue streams is a challenge
Next we find the way of solving the problems and integrating them together to form a Solution .
OTT Application for Telecom Service provider
Introduce new services to benefit from investment on Data Plans and Bandwidth
Expose REST API to enable 3trd party Integration with existing network Infrastructure
Partner with individual OTT players to make new services that do not compete on core competencies like billing etc
Use protocols like SIP that reduce CAPEX and have goto market more quickly
Go for enriched service that lead to better user experience
This write-up outlines the process of creating an OTT application for a Telecom Service Provider. Components for the application include cloud Address Book, Video Chatting, Location share, Contact synchronization ,REST-based thin client , OS and device agnostic etc shown in the figure below:
telco’s OTT app
The Application is designed to close knit with Operator’s own infrastructure hence the crucial entities like Network Address Book , Location Service are synced and fetched from Backend Network .
OTT application Feature Overview
Smart Address Book
Automatic: Get contacts from Gmail, Facebook
Fast search by first, last name, frequently
dialed number
Roadmap: View calendar events
Personal: Get image from Gmail and display in contacts list
Geo Location
Share own location during chatting
Get map for calculating the distance between two chat users
Roadmap : Trigger device (say Switch on/off AC before reaching home) from a threshold distance away from home location
Messaging
Ad-hoc Chat
Session Based Chat
Voice Input for texting
Presence information of contacts
RoadMap: Legacy message integration
Telephony
Voice call to mobile
Voice call to PSTN
Video call to other @imAll user
Share images during voice call to other
Device agnostic
Compatible with IOS, windows
Can run as native app on ipad
Can run as browser client on windows
RoadMap: native app for android, windows phone,blackberry10
Roadmap
features of Unified Communications ( UC)
To upgrade the application and provide enganced and enrich service support the I propose the following roadmap.
From plain vanilla voice and video calling ( supported by every other OTT application ) our application should progress towards legacy telecom support whihc included PSTN , GSM , ISDN etc . This requires backbone of telecom network and a good setup for media codec conversion to suit various legacy media codecs .
Road Map from Traditional to New age services
Voice and video calling
Legacy services support like MMS and SMS
Integration with 3rd party Vendors
Give new enriched services like Multilingual support , file transfer , screen-sharing etc
give facility to integrated web plugins for web calling
To keep the interest of customers it is essential that the application be supported on other popular OTT services like skype , Gtalk . for exmaple a caller should be able to make call from Skype / Gtalk to our application .Multilingual capabilities, support for larger protocol spectrum will just act like icing on the cake .
How does it benefit the Operator??
Saves on development cost and time
Device Agnostic OTT Applications
Simplified Service deployment
Saves licensing cost per client
Reuses existing Messaging and Address Book service logic.
Open New Revenue Streams for operator
No separate SIP stack required for the client
Faster Time to Market
Update : At the time of writing this post I did not anticipate the wave of change that bring focus on subjects like “net neutrality” , ” Save the internet” and “free internet”. However now most of the telcos providers have either joined the bandwagon by prividing SIP trunk endpoinst for cloud teelphony providers ( eg twilio, Google Calls) or have made their own IP call application for B2B customers.
A communication system can be made up of many components which are individually undergoing evolution such as access layer generations, and core layer upgrades. Harmonized and uniform open standard-based service delivery platforms over legacy Proprietary codebase is the preferred choice for most service providers to save the investment in their infrastructure and programming while keeping up with the shift in technology. I shall be editing this post to discuss more on the process of Service Harmonization.This saves the Telecom Service Provider the trouble of rewriting call logic with every telecom generation evolution ie IN to SIP to Web based WebRTC phones.
Landscape shift for Telecommunication Service providers includes Transmission layer which is ATM/Frame relays moving towards IP/MPLS. Access Layer hardware specific to POTS / PSTN / ISDN upgrading towards NGN and VOIP. Packet Switched Next gen Soft Switches based on SIP.
The Service Harmonization Layer does the job of holding all new and legacy services while providing uniform interface to interact with access network regardless of the back-end Call program logic. It involves consolidation for the service layers across IMS and legacy mobile network and Orchestration to extend the capability of underlying platform to support multiple IN variants. Diagrammatic depiction of scope of Service Harmonization.
Gateways based Harmonization
Service Broker based Harmonization
As CSPs evolve their networks for LTE, the resulting networks present tremendous challenges in voice services and application delivery. Realizing this opportunity, the telecom software industry has come forward with a purpose-built network element: the Service Broker, a solution specifically designed to overcome network architecture challenges and ensure voice service delivery from any network domain to any other network domain. Service Brokers are placed between the application layer and the control layer.
A service broker is a service abstraction layer between the network and application layer in a telecom environment. SB( Service Broker ) enables us to make use of existing applications and services from Intelligent Network’s SCP ( Service Control Point ), IMS’s Application Server as well as other sources in a harmonized manner
Legacy switches are circuit-switched, monolithic, propertiary and expensive while Softswitch is packet-switched and open interfaced. They are scalable and vendor-independent which enables easy convergence. Softswitches forms the basis for a service harmonization engine as they increase the granularity and power processing distribution of the Network
Legacy Service Layer has a function-centric architecture having multiple domain-specific session types such as Mobile calls, IPTV and broadband. Harmonized service delivery layer has Open APIs and is essentially Data-centric. This leads to fast and agile development and deployment of convergent services specifically IMS system providing the framework for underline network agnosticism across fixed and mobile.
Telecom R & D 