SIP Trunks


With the dawn of IP telephony service and cloud communication platforms in recent years, the SIP has caught the attention of many application developers. while SIP is essentially a session management multimedia signalling protocol its generic stack can be used for various use cases from IoT camera streaming sessions to call centres even auto calling for purpose of sharing OTP(one-time password) etc. In this I will highlight the usecase of large calltraffic and the use of SIP trunks.

SIP based trunking can provide significant cost savings and business process improvements by supporting the native SIP protocol that controls the VoIP systems used in call centres and business communication platforms.

  • (+) unified communication
  • (+) lower telco network
  • (+) streamline operations for multicountry/ geography

Traditional trunk call

In the past, telephone systems used trunk lines to connect different parts of the network. Trunk lines were long-distance communication lines that connected telephone exchanges in different locations. Trunk calls were calls made over these trunk lines. They were typically used for long-distance communication, as they allowed calls to be made between exchanges that were geographically far apart. Trunk calls were generally more expensive than local calls, as they involved the use of long-distance communication lines.

Traditional trunk calls operated like a circuit with local loops , trunk lines and switching offices. The telco acted as carriers that sell of lease communication lines to facilitate communication over long distances using local exchanges and interexchange carriers.

In the early days of telephone systems, trunk lines were typically made of copper wires or cables. Later, trunk lines were replaced with satellite links and fiber optic cables, which provided higher capacity and faster transmission speeds. Today, with the widespread adoption of VoIP (Voice over Internet Protocol) technology, many telephone systems no longer use trunk lines in the traditional sense. Instead, they use virtual connections, such as SIP trunks (Session Initiation Protocol trunks), which allow organizations to make and receive phone calls over the internet. SIP trunks are generally more flexible and cost-effective than traditional trunk lines, and do not require the installation of additional hardware.

Voice trunk Lines in SS7 based Next Generations IN networks used media gate ways and MGCP, H323 protocols

Image credits : Unknown

SIP trunk (older) systems

SIP is a protocol that is commonly used in VoIP (Voice over Internet Protocol) systems to set up, modify, and terminate sessions that involve the exchange of audio, video, and other media. SIP Trunks are virtual voice channels (or paths) which deliver media (voice, video, IM) over an IP network to a designated endpoint. SIP Trunks can be thought of as a virtual line or concurrent call path. SIP Trunks are delivered over an IP connection like Tier One Carrier or Voice Optimized Recommended or UDP. SIP Trunk may be over-subscribed ie can have more numbers than trunks for example G.711 – 17 calls over T1 or G.729a – 45 calls over T1. SIP Trunking can be provided as one-way or two-way lines. Direct Inward Dialing (DIDs) can be used for toll-free number service.

Centralized SIP Trunk Model

Centralized SIP Trunk Model is designed to aggregate all calls from all sites and funnels them into a single entry point. Each site has its own SIP trunk termination of the appropriate capacity for calls to and from that site.

Such SIP trunks models offer benefits in three significant areas:

  1. Cost savings, arising from many factors including reduced telecommunications network charges and streamlined operations.
  2. Unified communications, where voice, video, email, text and other messaging technologies are combined to provide greater flexibility for users by enabling new ways to transfer information and manage connectivity. Many SIP trunk providers offer advanced features such as call forwarding, call waiting, and voicemail, which can improve the overall communication experience for employees.
  3. Business Continuity and Disaster Recovery, where the right physical configuration in conjunction with intelligence in the network can be leveraged to provide uninterrupted communications and alternative means to stay connected for employees in the event of system bottlenecks or failures.

SIP trunking is an IP-based alternative to ISDN trunking services

SIP Trunking is a low-cost IP-based alternative to ISDN offering for medium to large businesses needing upwards of several tens of channels in a trunk, often across multiple sites, with IP VPN access. 

  • (+) Optimal utilization of bandwidth by delivering both data and voice in the same bandwidth

A telephony company such as a telecom service provider may expose SIP trunks as a means of connecting inbound or outbound calls through its telecom network. For the integrator ( or the service provider managing the other enedpoint of the call leg ) it can be no different that a traditional phone call.The SIP signalling however is useful for enabling better session understaning using standard SIP requests and responses as compared to SS7 or PRI lines.

Planning to set up SIP trunk

•Cost analysis
•Assess traffic volumes and patterns
•Assess network design implications
•Emergency call policy
•Define production user community phases
•Define user community to pilot
•Evaluate future new services
•Assess security precautions

The steps to set up a SIP trunk connection may vary depending on the specific provider and the equipment being used. However, here are some general steps that are often involved in the process:

  1. Choose a SIP trunk provider: Research and compare different SIP trunk providers to find one that meets your organization’s needs and budget.
  2. Sign up for a SIP trunk account: Follow the provider’s instructions to sign up for a SIP trunk account. This may involve completing an online form, providing contact information and payment details, and selecting the desired features and services.
  3. Configure your VoIP phone system: Consult your VoIP phone system’s documentation to learn how to configure it to work with a SIP trunk. This may involve specifying the SIP trunk’s IP address and port number, as well as any authentication credentials that are required.
  4. Test the connection: Once the SIP trunk is set up, it is a good idea to test the connection to ensure that it is working properly. Make a few test calls to verify that the connection is functioning as expected.
  5. Use the SIP trunk: Once the SIP trunk is set up and tested, it can be used to make and receive calls using your VoIP phone system.

SIP Trunking platform has to integrate with multiple networks seamlessly. Components for setting up a SIP trunking system requires atleast these

  • Compliance with standrad signalling protol, like SIP.
  • SBC( Session Border Controller ) facing the private PBX
  • Gateway for specific endpoints such as PSTN gateway , public internet gateway etc
  • L3/L4 Layer switches
  • Telco operator lines
  • Codec support

Kamailio is an open-source SIP (Session Initiation Protocol) server that can be used to create a SIP trunk. Kamailio can be PBX used to connect different locations within an organization, enabling employees to communicate with each other using their VoIP phones. Kamailio can also be used to set up a SIP trunk in a number of ways. For example, it can be used to connect an organization’s VoIP phone system to the public telephone network, allowing employees to make and receive calls from outside the organization.

https://telecom.altanai.com/2016/08/02/session-border-controller-for-webrtc/

Kamailio is a highly flexible and customizable SIP server that can be configured to meet the specific needs of an organization. It offers a range of features and functionality, including call routing, load balancing, and security. Kamailio is a popular choice for organizations that want to set up a SIP trunk because it is open-source and can be customized to meet their specific needs.

Features of SIP trunking

SIP trunk with VoIP phone systems are often preferred over traditional phone systems because they are generally more flexible and cost-effective. They allow employees to make and receive calls from any device with an internet connection, including desk phones, smartphones, and laptops. They can be easily scaled up or down to meet changing communication needs and do not require the installation of additional physical hardware. Some factors to consider when evaluating SIP trunks include:

  1. Cost: It is important to compare the costs of different SIP trunk providers and consider factors such as monthly fees, per-minute charges, and any additional fees for features or services.
  2. Coverage: Make sure that the SIP trunk provider has coverage in the areas where your organization needs to make and receive calls.
  3. Quality: The quality of a SIP trunk can vary greatly depending on the provider and the connection. Be sure to research the provider’s reputation for call quality and reliability.
  4. Features: Different SIP trunk providers may offer different features, such as call forwarding, call waiting, and voicemail. Consider which features are important to your organization and make sure that the SIP trunk provider offers them.
  5. Customer support: It is important to choose a SIP trunk provider that offers reliable customer support in case you experience any issues with your service.

Other features that are good to have is integration to existing backend for OSS/BSS stack. Some of the feature set for a carrier grade SIP trunking solution are listed here

  • Inbound and outbound trunks
  • Number Import/Export
  • Security
    • Dynamic registeration of users
    • Authentication and Authorization
    • Security (SRTP)
  • Cost Savings
    • Low cost for large traffic volumes instead of charges of call per second
    • CDR for tracing and monitoring call failures
  • Clear media stream ( no robotic or choopy audio). Good MOS score
  • realtime traffic monitoring to rule out bad players.
  • Inbound and Outbound call – Call Establishment, Rejection, Termination
  • DDI: Direct Dialling-In ranges can be provided on the SIP Trunk
  • CLIP(Calling Line Identification Presentation )/CLIR Calling Line Identification Presentation Restriction) for Inbound and Outbound
  • Call Management
    • AUTH Code Screening
    • Combined Screening
    • Data Call Screening
    • Local Screening
    • Anonymous Call Rejection: Anonymous Call Rejection
    • Incoming Call Barring: bar receiving of calls to certain extensions
    • Outgoing Call Barring: Restrict calls to certain numbers
    • Incoming Call Diversion – unconditional, busy, and unreachable
    • Call Admission Control: Call Admission Control (CAC) is a mechanism to restrict the number of simultaneous sessions (calls) 
    • Incoming Call Diversion (DestNo not reachable, CAC exceeded, unconditional)
  • Geographic and Non-Geographic Number Support
  • Multiple Codec Support
  • Emergency Calling: Emergency Calls are routed on a priority basis irrespective of the customer’s available channel

Trunking inbound services voice can be used to support contact centres, conferencing, number translation services etc. Regulatory requirements for the operation of the customer in the PSTN of respective countries must be met with Country Specific Emergency Calling support Enhanced feature set for SIP trunking should include the features of the SIP Trunking with Multicountry support

  • Enhanced CAC(Call Admission Control) – Directional & Network
  • Global Dial Plan Support
  • Proactive MCID (Malicious CallerId) Identification and tracing
  • Call Distribution(CD)
  • Intelligent Routing involving machine learning and constant feedback
    • Origin Based Routing
    • Menu Routing
    • Origin Dependent Routing (ODR)
    • PIN Routing
    • Dynamic Route Select
    • Time-Dependent Routing (TDR)
    • Uniform Load Distribution(ULD)
    • International Routing
    • Mobile Routing
    • Payphone Routing
  • Product Association

Ultimately, the most useful SIP trunk for your organization will depend on your specific needs and budget. It is a good idea to research and compare different SIP trunk providers to find the one that best meets your organization’s needs.

Future of SIP trunks

SIP trunking systems are likely to continue to be an important part of the telecommunications landscape in the future. As more and more organizations adopt WebRTC or SRT based VoIP (Voice over Internet Protocol) technology for their phone systems, the demand for SIP trunks is likely to continue to grow. One trend that is expected to shape the future of SIP trunking is the increasing adoption of cloud-based communication systems. As more organizations move their communication systems to the cloud, they are likely to turn to SIP trunks as a way to connect their phone systems to the public telephone network and enable remote communication. Another trend that is expected to impact the future of SIP trunking is the increasing adoption of 5G technology. 5G networks offer faster speeds and lower latency, which may make it possible to use SIP trunks for real-time communication applications such as interactive and/or immersive video conferencing.


Certificates, compliances and Security in VoIP

This article describes various Certificates and compliances, Bill and Acts on data privacy, Security and prevention of Robocalls as adopted by countries around the world pertaining to Interconnected VoIP providers, telecommunications services, wireless telephone companies etc

Compliance certificates by Industry types

HIPAA (Health Insurance Portability and Accountability Act)

Deals with privacy and security of personal medical records and electronic health care transaction

Applicability  : If voip company handles medical information

Includes : 

  • Not allowed Voice mail transcription
  • Should have End-to-End Encryption
  • Restrict  using unsecured WiFi networks to prevent Snooping
  • User security , strong password rules  and mandatory monthly change
  • Secure Firmware on VoIP phones
  • Maintaining Call and Access Logs

SOX( Sarbanes Oxley Act of 2002)

Also known as SOX, SarbOX or Public Company Accounting Reform and Investor Protection Act

Applicability : if managing the communications operations of a regulated, publicly traded company 

Includes : 

  • Retain records which include financial and other sensitive data
  • ways employees are provided or denied access to records or data based on their roles and responsibilities
  • do information audit by a trusted third party. 
  • Retention and deletion of files such as audio files like voicemails, text messages, video clips, declared paper records, storage, and logs of communications activities
  • Physical and digital security controls around cloud-based VoIP applications and the networks

Privacy Related Compliance certificates

COPPA (Children’s Online Privacy Protection Act ) of 1998 

prohibits deceptive marketing to children under the age of 13, or collecting personal information without disclosure to their parents. 

any information is to be passed on to a third party, must be easy for the child’s guardian to review and/or protect

2011 amendment  requires that the data collected was erased after a period of time,

2014 FTC issued guidelines that apps and app stores require “verifiable parental consent.”

CPNI (Customer Proprietary Network Information) 2007

CPNI (Customer Proprietary Network Information) in united states is the information that communication providers  acquire about their subscribers. This Individually identifiable information that is created by a customer’s relationship with a provider, such as data about the frequency, duration, and timing of calls, the information on a customer’s bill, and call identifying information. This processing information is governed strictly by FCC and certification should be renewed on an annual basis

Provider can pass along that information to marketers to sell other services, as long as the customer is notified

In 2007, the FCC explicitly extended the application of the Commission’s CPNI rules of the Telecommunications Act of 1996 to providers of interconnected VoIP service.

CALEA

Communications Assistance for Law Enforcement Act (CALEA) conduct electronic surveillance by imposing specific obligations on “telecommunications carriers” for assisting law enforcement, including delivering call interception and call identification functionality to the government with a minimum of interference to customer service and privacy.

Read more about CALEA and its roles in VoIP here Regulatory and Legal Considerations with WebRTC development

GDPR (General Data Protection Regulation)  in European Union 2018

Supersedes the 1995 Data Protection Directive

Establishes requirements of organizations that process data, defines the rights of individuals to manage their data, and outlines penalties for those who violate these rights.

No personal data may be processed unless this processing is done under one of six lawful bases specified by the regulation (consent, contract, public task, vital interest, legitimate interest or legal requirement). When the processing is based on consent the data subject has the right to revoke it at any time.

Controllers must notify Supervising Authorities (SA)s of a personal data breach within 72 hours of learning of the breach.

California Consumer Privacy Act (CCPA) 2019

consumer rights relating to the access to, deletion of, and sharing of personal information that is collected by businesses. 

Allows consumers to know whether their personal data is sold or disclosed , to whom .

Allows opt-out right for sales of personal information

Right to deletion – to request a business to delete any personal information about a consumer collected from that consumer

Personal Data Protection Bill (PDP) – India 2018

This bill introduces various private and sensitive protection frameworks  like restriction on retention of personal data, Right to correction and erasure (such as right to be forgotten) , Prohibition and transparency of processing of personal data. It also classifies data fiduciaries  including certain social media intermediaries. 

The Bill amends the Information Technology Act, 2000 to delete the provisions related to compensation payable by companies for failure to protect personal data.

Other data privacy acts similar to GDPR 

  • South Korea’s Personal Information Protection Act  2011
  • Brazil’s Lei Geral de Proteçao de Dados (LGPD)  2020
  • Privacy Amendment (Notifiable Data Breaches) to Australia’s Privacy Act 2018
  • Japan’s Act on Protection of Personal Information 2017
  • Thailand Personal Data Protection Act (PDPA) 2020

Features offered by VOIP companies for Data privacy 

  • Access Control & Logging
  • Auto Data Redaction / Account Deletion policy 
  • SIEM (Security information and event management) alerts 
  • Information security , Encrypted Storage For Recordings & Transcripts
  • Disclosing all third party services that are involved in data processing too
  • Role Based Access Control and 2 Factor Authentication
  • Data Security Audits and appointing  data protection officer to oversee GDPR compliance

Against Robocalls and SPIT ( SPAM over Internet Telephony)

 2009 Truth in Caller ID Act 

Telephone Consumer Protection Act of 1991

Implementation of Do not call registry against use of robocalls, automatic dialers, and other methods of communication

Do-Not-Call Implementation Act of 2003

if a business has an established relationship with a customer, it can continue to call them for up to 18 months. If a consumer calls the company, say, to ask for information about the product or service, the company has three months to get back to him.

if the customer asks to not receive calls, the company must stop calling, or be subject to fines.

Exemptions – Calls from a not-for-profit B organisation , informational messages as flight cancellations , Calls from sales and debt collectors etc

Personal Data Privacy and Security Act 2009

Implemented to curb  identity theft and computer hacking. Sensitive personal identifiable information includes : victim’s name, social security number, home address, fingerprint/biometrics data, date of birth, and bank account numbers.

Any company that is breached must notify the affected individuals by mail, telephone, or email, and the message must include information on the company and how to get in touch with credit reporting agencies

If the breach involves government or national security , company must also contact the Secret Service within fourteen days 

TRACED Act (Telephone Robocall Abuse Criminal Enforcement and Deterrence) 2019

Canadian Radio-television and Telecommunications Commission (CRTC) 2018 -32

A solution mechanism has already been standardised and active in adoption called STIR / SHAKEN ( Secure Telephony Identity Revisited / Signature-based Handling of Asserted information using toKENs) described in another article here.

Emergency services 

FCC E911 E911 / VoIP E911 rules

Unlike traditional telephone connections, which are tied to a physical location, VOIP’s packet switched technology allows a particular number to be anywhere making it more difficult for it to reach localised services like emergency numbers of Public Safety Answering Points (PSAPs) . Thus FCC regulations as well as the New and Emerging Technologies 911 Improvement Act of 2008 (NET 911 Act), interconnected VoIP providers are required to provide 911 and E911 service. 

Ref : 

Video Codecs – H264 , H265 , AV1

Article discusses the popularly adopted current standards for video codecs( compression / decompression) namely MPEG2, H264, H265 and AV1


Compression algorithms differ from media containers since they involves compressing the information in raw stream to reduce the size for streaming applications while media files are containers which are just used for playback from a set location.

Examples of Codecs: H.261, H.263, VC-1, MPEG-1, MPEG-2, MPEG-4, AVS1, AVS2, AVS3, VP8, VP9, AV1, AVC/H.264, HEVC/H.265, VVC/H.266, EVC, LCEVC

Examples of containers inlude :MPEG-1 System Stream, MPEG-2 Program Stream, MPEG-2 Transport Stream, MP4, MOV, MKV, WebM, AVI, FLV, IVF, MXF, HEIC so on.

MPEG 2

MPEG-2 (a.k.a. H.222/H.262 as defined by the ITU)
Generic coding of moving pictures and associated audio information
Combination of lossy video compression and lossy audio data compression methods, which permit storage and transmission of movies using currently available storage media and transmission bandwidth.

MPEG2 is better than MPEG 1

Evolved out of the shortcomings of MPEG-1 such as audio compression system limited to two channels (stereo), No standardized support for interlaced video with poor compression , Only one standardized “profile” (Constrained Parameters Bitstream), which was unsuited for higher resolution video.

Application

  • over-the-air digital television broadcasting and in the DVD-Video standard.
  • TV stations, TV receivers, DVD players, and other equipment
  • MOD and TOD – recording formats for use in consumer digital file-based camcorders.
  • XDCAM – professional file-based video recording format.
  • DVB – Application-specific restrictions on MPEG-2 video in the DVB standard

MPEG -4

Video coding standards :-
MPEG-4 Part 2 Visual (ISO/IEC 14496-2) released in 1999 as MPEG-4 video codec
MPEG-4 Part 10 Advanced Video Coding (ISO/IEC 14496-10) released in 2003 as AVC/H.264 video codec;
MPEG-4 Part 14 (ISO/IEC 14496-14) MP4 file format is a media container. rather than a Codec ( compression algorithm).

H264

Introduced in 2004 as Advanced Video Coding (AVC)/H.264 or MPEG-4 AVC or ITU-T H.264/MPEG-4 Part 10 ‘Advanced Video Coding’ (AVC). It is a widely supported vendor agnostic solution.

MPEG-4 Part 10 AVC/H.264 is better than MPEG2

  • 40-50% bit rate reduction compared to MPEG-2
  • Resolution support 4K (4,096×2,304) and 59.94 fps
  • 21 profiles ; 17 levels

Compression Model

Video compression relies on predicting motion between frames. It works by comparing different parts of a video frame to find the ones that are redundant within the subsequent frames ie not changed such as background sections in video. These areas are replaced with a short information, referencing the original pixels(intraframe motion prediction) using mathematical function and direction of motion

Hybrid spatial-temporal prediction model
Flexible partition of Macro Block(MB), sub MB for motion estimation
Intra Prediction (extrapolate already decoded neighbouring pixels for prediction)
Introduced multi-view extension
9 directional modes for intra prediction
Macro Blocks structure with maximum size of 16×16
Entropy coding is CABAC(Context-adaptive binary arithmetic coding) and CAVLC(Context-adaptive variable-length coding )

Applications of H264

  • most deployed video compression standard
  • Delivers high definition video images over direct-broadcast satellite-based television services,
  • Digital storage media and Blu-Ray disc formats,
  • Terrestrial, Cable, Satellite and Internet Protocol television (IPTV)
  • Security and surveillance systems and DVB
  • Mobile video, media players, video chat

H265

High Efficiency Video Coding (HEVC), or H.265 or MPEG-H HEVC. Streams high-quality videos in congested network environments or bandwidth constrained mobile networks.

  • (+) 2 times the video compression with the same video quality as H264.
  • (-) higher processing power required

Introduced in Jan 2013 as product of collaboration between the ITU Video Coding Experts Group (VCEG) and the ISO/IEC Moving Picture Experts Group (MPEG).

H265 is better than H264

Overcome shortage of bandwidth, spectrum, storage and performs bandwidth savings of approx. 45% over H.264 encoded content

  • Resolutions up to 8192×4320, including 8K UHD
  • Supports up to 300 fps
  • 3 approved profiles, draft for additional 5 ; 13 levels

Whereas macroblocks can span 4×4 to 16×16 block sizes, CTUs can process as many as 64×64 blocks, giving it the ability to compress information more efficiently.

Multiview encoding – stereoscopic video coding standard for video compression that allows for the efficient encoding of video sequences captured simultaneously from multiple camera angles in a single video stream. It also packs a large amount of inter-view statistical dependencies.

Compression Model

  1. Enhanced Hybrid spatial-temporal prediction model
  2. CTU ( coding tree units) supporting larger block structure (64×64) with more variable sub partition structures
  3. Motion Estimation – Intra prediction with more nodes, asymmetric partitions in Inter Prediction). Individual rectangular regions that divide the image are independent
  4. Paralleling processing computing – decoding process can be split up across multiple parallel process threads, taking advantage multi-core processors.
  5. Wavefront Parallel Processing (WPP)- sort of decision tree that grants a more productive and effectual compression.
  6. 33 directional nodes – DC intra prediction , planar prediction. , Adaptive Motion Vector Prediction
  7. Entropy coding is only CABAC

Applications of H265

  • cater to growing HD content for multi platform delivery
  • differentiated and premium 4K content

Reduced bitrate enables broadcasters and OTT vendors to bundle more channels / content on existing delivery mediums
also provide greater video quality experience at same bitrate

Using ffmpeg for H265 encoding

I took a h264 file (640×480) , duration 30 seconds of size 39,08,744 bytes (3.9 MB on disk) and converted using ffnpeg

After conversion it was a HEVC (Parameter Sets in Bitstream) , MPEG-4 movie – 621 KB only !!! without any loss of clarity.

> ffmpeg -i pivideo3.mp4 -c:v libx265 -crf 28 -c:a aac -b:a 128k output.mp4  
                                            
ffmpeg version 4.1.4 Copyright (c) 2000-2019 the FFmpeg developers   
built with Apple LLVM version 10.0.1 (clang-1001.0.46.4)   
configuration: --prefix=/usr/local/Cellar/ffmpeg/4.1.4_2 --enable-shared --enable-pthreads --enable-version3 --enable-avresample --cc=clang --host-cflags='-I/Library/Java/JavaVirtualMachines/adoptopenjdk-12.0.1.jdk/Contents/Home/include -I/Library/Java/JavaVirtualMachines/adoptopenjdk-12.0.1.jdk/Contents/Home/include/darwin' --host-ldflags= --enable-ffplay --enable-gnutls --enable-gpl --enable-libaom --enable-libbluray --enable-libmp3lame --enable-libopus --enable-librubberband --enable-libsnappy --enable-libtesseract --enable-libtheora --enable-libvorbis --enable-libvpx --enable-libx264 --enable-libx265 --enable-libxvid --enable-lzma --enable-libfontconfig --enable-libfreetype --enable-frei0r --enable-libass --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-librtmp --enable-libspeex --enable-videotoolbox --disable-libjack --disable-indev=jack --enable-libaom --enable-libsoxr   
libavutil      56. 22.100 / 56. 22.100   
libavcodec     58. 35.100 / 58. 35.100   
libavformat    58. 20.100 / 58. 20.100   
libavdevice    58.  5.100 / 58.  5.100   
libavfilter     7. 40.101 /  7. 40.101   
libavresample   4.  0.  0 /  4.  0.  0   
libswscale      5.  3.100 /  5.  3.100   
libswresample   3.  3.100 /  3.  3.100   
libpostproc    55.  3.100 / 55.  3.100 
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'pivideo3.mp4':   
...

If you get error like

Unknown encoder 'libx265'

then reinstall ffmpeg with h265 support

HEVC bitstream is an ordered sequence of the syntax elements. Each syntax element is placed into a logical packet called a NAL (network abstraction layer) Unit. There are 64 different NAL Unit types. They can be grouped into 10 classes:

  1. VPS – Video parameter set
  2. SPS – Sequence parameter set
  3. PPS – Picture parameter set
  4. Slice (different types)
  5. AUD – Access unit delimiter signals the start of video frame
  6. EOS – End of sequence
  7. EOB – End of bitstream
  8. FD – Filler data for bitrate smoothening
  9. SEI – Supplemental enhancement information such as picture timing, color space information, etc.
  10. Reserved and unspecified

AV1

Realtime High quality video encoder , product of the Alliance for Open Media (AOM). Contained by Matroska , WebM , ISOBMFF , RTP (WebRTC).

Av1 is better than H265

  • (+) AV1 is royalty free and overcomes the patent complexities around H265/HVEC

Applications

  • Video transmission over internet , voip , multi conference
  • Virtual / Augmented reality
  • self driving cars streaming
  • intended for use in HTML5 web video and WebRTC together with the Opus audio format

SVC ( scalable video encoding)

SVC standardises the encoding of a high-quality video bitstream that also contains one or more subset bitstreams. Asubset bitsteam can represent a lower spatial resolution (smaller screen), lower temporal resolution (lower frame rate), or lower quality video signal ( dropped packet) compared to the base stream it is derieved from.

  • Temporal (frame rate) scalability is enabled through structuring motion compensation dependencie so that complete pictures (i.e. their associated packets) can be dropped from the bitstream.
  • Spatial (picture size) scalability is enabled with video coded at multiple spatial resolutions
  • SNR/Quality/Fidelity scalability: video is coded at a single spatial resolution but at different qualities. The data and decoded samples of lower qualities can be used to predict data or samples of higher qualities in order to reduce the bit rate to code the higher qualities.
  • Combined scalability: a combination of the 3 scalability modalities described above.

Not all codecs sypport all modes. While the Av1 and VP9 support majority of modes defined in the table, VP8 only supports temporal scalability (e.g. “L1T2”, “L1T3”);H.264/SVC supports both temporal and spatial scalability but only permits transport of simulcast on distinct SSRCs.

Vp8

{
      "clockRate": 90000,
      "mimeType": "video/VP8",
      "scalabilityModes": [
        "L1T1",
        "L1T2",
        "L1T3"
      ]
    },
 const stream = await navigator.mediaDevices.getUserMedia(constraints);
    selfView.srcObject = stream;
    pc.addTransceiver(stream.getAudioTracks()[0], {direction: 'sendonly'});
    pc.addTransceiver(stream.getVideoTracks()[0], {
      direction: 'sendonly',
      sendEncodings: [
        {rid: 'q', scaleResolutionDownBy: 4.0, scalabilityMode: 'L1T3'}
        {rid: 'h', scaleResolutionDownBy: 2.0, scalabilityMode: 'L1T3'},
        {rid: 'f', scalabilityMode: 'L1T3'},
      ]    
    });


References :

NLP ( Natural Language Processing ) in VoIP

NLP ( Natural Language Processing ) can be defined as the automatic manipulation of natural languages ( text or audio) using computer algorithms and softwares. As such NLP has great potential in cognitive and artificial intelligence , but also with increasing human to machine interaction and enhancement in Machine learning ,NLP is set to revolutionize the Voice over IP space.

Note : although not obvious but some people confuse Natural language procession with Neurolinguistic pressing which is a science in Psychology.

NLP evolves from linguistics which itself is a study of language along with its semantics , phonetics and gramer. Every language has rules and NLP uses mathematical formulation to understand it. Discrete mathematical formalisms will be discussed later in this article.

Inputs for NLP is usually though conversation, speech, correspondence, reading, print, written composition, dictation, publishing, translation, lip reading, signing etc .

Rule based vs Statistical NLP – In contrast to rule based engines which work on hard preset values using maybe a decision tree , statistical models work in a more probabilistic fashion which produces more reliable results even in unfamiliar scenarios.

Linear classifier vs Convolutional Neural Nets– CNNs are powerful supervised deep learning technique. As opposed to a linear classifier whose decision boundary on feature space is linear function , CNN increases model complexity by adding more layers . tbd-

NLP tasks

Syntax

Grammer induction , lemmitization , morphological segmentation , part of speech tagging , parsing , sentence breaking , stemming , word segmentation , terminology extraction

Semantics

lexical , distributional , machine translation , Named entity recognition ( NER) , natural language understanding and generation, relationship establishment , sentimental analysis , work sense disambiguation , OCR( optical Character recognition) , recognizing textual entailment

Speech

speech recognition , specch segmentation , text to speech , dialogues

Discourse

automatic summarizations , conference resolution , discourse analysis

Key techniques

Out of above its worthy to point out few key techniques

Parts of speech (POS )

A primary tasks in NLP is to extract tokens and sentences, identify parts of speech ( like nouns , verbs , adjectives ) and create parse trees.

POS tagging is the process of marking up a word in a corpus to a corresponding part of a speech tag . By tagging, algorithm builds lemmatizers which are used to reduce a word to its root form.

POS methods significantly differs from Bag-of-words(BOW) methods which disregards semantic relation relationship and only takes into account words and their frequencies. Whereas POS takes context and definition into consideration.

POS tagging techniques include lexical , rule based , probablistic and deep learning methods.

Named entity recognition (NER)

Given a stream of text, determine which items in the text map to proper names, such as people or places, and their types such as person, location, Organization. Example for raw test as below using Spacy.io

“Hello ! My name is Atanai and I work on Solution design and architecture, developed many custom WebRTC and SIP based solutions such as telecom applications, media stream inetgration into IOT,Unified communication-collaboration ,signalling gateways ,SBC etc. I passed out from Anna university with Betch degree in 2011 and currenlty stay in Bangalore India.”

Analysis of NER is

Noun phrases: ['My name', 'Atanai', 'I', 'Solution design', 'architecture', 'many custom', 'WebRTC and SIP based solutions', 'telecom applications', 'media stream integration', 'IOT', 'Unified communication-collaboration', 'signalling gateways', 'I', 'Anna university', 'Betch degree', 'currently stay', 'Bangalore India'] 
Verbs: ['be', 'work', 'develop', 'base', 'signal', 'pass'] 
Atanai PERSON 
WebRTC PRODUCT 
SIP ORG 
IOT ORG 
Betch NORP 
2011 DATE 
Bangalore India LOC

Sentiment Analysis

Understand the overall opinion, feeling, or attitude expressed in given media ( speech , text or video) .

NLP in action

NLP application layout

Steps to obtain insights and relevant information from an unclassified document , raw tex file or speech to text content such as recording from VOIP meeting

step 1 : upload a document which could be an invoice , order , feedback , complaint or any other unstructured raw text

Step 2 : Collect the data from the document

  • use OCR (optical character recognition) for hand written or signed components
  • perform search , index , duplication detection etc
  • can use MNIST database as
  • phrase matching and vocabulary
  • Can use translation APIs to trans late from other languages

Step 3 : Collect meaning-full data

  • perform Part of Speech (POS) tagging and chunking process
  • topic discovery and modelling
  • tokenizations and text classification , obtain domain specific entities from the document
  • can use standard model language to collect relevant frequently used words
  • NER ( Named Entity recognition ) to validate names , places and locations
  • can extract out time and date from mentioned entities
  • build relationship graphs

step 4 : extract sentiments using a trained model

  • utilize Regular Expressions for pattern searching
  • sentiment analysis

General Applications:

Application of NLP find its way into many domains

1.VOIP platforms ,media servers and automatic summarization of conference / meetings like “Minutes of Meetings” to highlight the key takeaways from a VOIP session

2. Automatic essay assessment and scripting in education setting alike.

3. Image annotation using metadata describing digital images for categorizations and easy retrieval based on keywords.

4. Spam filtering

5. Building automatic assistants and chatbots with Speech Recognition and using auto suggest with sentence completion ( Siri , Alexa , google voice etc )

6. Social Media Analytics , to track sentiments about topic , figure out influencers such as for movie or restaurant reviews .

NLP in VOIP system

To know more about sound waves go here which describes fundamental characteristics of analog waves . To know more about analog wave modulation go here , this describes how waves are modulated such as frequency , phase , amplitude etc to hold information for propagation . click here to know more about digital wave modulation such as amplitude , frequency , phase shift keying etc . This section build on top of audio streams captured or live .

Based on NLP and trained models on extracted features ,an unknown audio wave can be classified and possibly identified.

Replacing auto attendants with IVR

tbd–

Ref :

Tools ref:

Kamailio DNS and NAT

  • DNS sub-system in Kamailio
    • DNS failover
    • DNS load balancing
    • NAT ( Network Address Translation)
  • NAT ( Network Address Translation)
  • Why is NAT is important in SIP?
  • Types of NAT solutions
  • NAT behaviours
  • RTP NAT
  • Fixing NAT
  • NAT Traversal Module
    • Why use keepalive when Registrations are already there for NATing ?
    • How keepalives work for NATing ?
    • function nat_keepalive()
    • Params
    • Functions
    • client_nat_test()
      • fix_contact()
      • nat_keepalive()
    • Pseudo Variables
    • Statistics
  • NATHelper Module
    • NAT pinging types
      • UDP packet
      • SIP request
    • params
    • functions
    • Pseudo Variables
    • RPC Commands

In this article, we discuss Nating in a SIP Server like Kamailio. Types of NAT pings, their behaviour and types. Also some implementation of some of the Kamailios modules like

  • NAT Traversal module
  • NAT helper module
  • STUN module

A repository for extensive application of kamailio modules for various usecases is https://github.com/altanai/kamailioexamples

DNS sub-system in Kamailio

To resolve hostname into IPs Kamailio can do either of below

  • use libresolv and a combination of the locally configured DNS server /etc/hosts and the local Network Information Service (NIS/YP a.s.o) or
  • cache the query results and first look into internal cache

DNS failover – if destination resolves to multiple addresses tm can try all of them until it finds one to which it can successfully send the packet or it exhausts all of them, with internal DNS cache. Also used when the destination host doesn’t send any reply to a forwarded invite within the SIP timeout interval (tm fr_timer parameter).

DNS load balancing – SRV based load balancing with weight value in the DNS SRV record.

  • (-) Only the locally configured DNS server (usually in /etc/resolv.conf) is
    used for the requests (/etc/hosts and the local Network Information Service are ignored).
    • optional: disable the DNS cache (use_dns_cache=off or compile without -DUSE_DNS_CACHE).
  • (-) DNS cache uses extra memory
    • optional: disable the DNS cache.
  • (-) DNS failover introduces a very small performance penalty
    • optional: disable the DNS failover (use_dns_failover=off).
  • (-) DNS failover increases the memory usage (the internal structures used to represent the transaction are bigger when the DNS failover support is compiled).
    • optional: compile without DNS failover support (DUSE_DNS_FAILOVER).Turning it off from the config file is not enough in this case (the extra memory will still be used).

NAT ( Network Address Translation)

Network address translation replaces the IP address within packets with a different IP address which internet endpoints can relate with. This enables multiple hosts in a private subnet with their pwn private address ( 10.x.x.x or 192.x.x.x etc ) to share single public IP address interface, to access the Internet.

NAT is bidirectional- If the private ip:port got translated to public ip:port on the inside interface while entering outside internet, on arriving from outside interface it will get translated from public ip:port to private ip:port.

For a SBC ( Session border controller ) or where the kamailio server is directly customer facing, where you dont have a private line or VPN to clients, then it is often encountered with NATed endpoints. Read more about NAT traversal using STUN and TURN here

Why is NAT is important in SIP?

These characteristics of SIP design and operation flows demonstrate why NAT solutions are so important

  • RFC 3261 for SIP presumed end-to-end reachability and does not specify much around ANT issues .
  • No NLRI (Network Layer Reachability Information) translation layer exists, such as DNS or ARP
  • SIP is designed to used RTP which uses dynamically allocated ports to stream media.
    It is comparable to FTP which creates ephemeral connections on unpredictable dynamic ports to send multiplexed data and “metadata”, instead of protocol like HTTP where all data is sent on same connection.
  • UDP (default transport for SIP) is connection less and session tracking requires these be mapped onto a statelful flow, rigorous keepalives and other such techniques like using TCP instead have their own tradeoffs
  • since sip packets put network and transport information right on sip header they are limited by the rateability and awareness of their network interface thereby prevent other endpoint from reaching its ip or port

Types of NAT solutions

  • Client-side NAT traversal – clients are responsible for identifying their WAN NLRI and adding ip and port to navigate them in outside world
  • Server-side NAT traversal – SIP server should discover the client’s WAN addressing while clients continue to work transparently behind NAT. Requires that DIP server look at the source and destination ip and port of actual packets instead of relying on the encapsulated sip headers and SDP body.
  • ALG (Application Layer Gateways) – mostly applied at router itself. wodk by susbtitung public IP/port information inplace of provate and vice versa for return packets. Limitataions – they dont provide a fullproof fix example they may fix Via but not the Contact address or SDP body or RTP ports
WebRTC signalling and media flow on Open public network
Open network leading to smooth p2p media stream

Far-end NAT traversal solution ( TURN server)

WebRTC media flow when peers are behind NAT . Uses TURN relay mechanism
public private ip mapping , firewalls and private network obstruct p2p media flow. TURN is useful to relay media pckets

NAT behaviours

Cone NATSymmetric NAT
Local client performs an outbound connection to a remote UA and a dynamic rule is created for the destination IP tuple, allowing the remote machine to connect back.Local client allows inbound connections from a specific source IP address and port, also NAT assigns a new random source port for each destination IP tuple
Further subdivied into:
1. Full Cone NAT
2. Restricted Cone NAT – all requests from the same internal IP address and port are mapped to the same external IP address and port.
-3. Port-Restricted Cone NAT

RTP NAT

NAT not only applies to sip signalling packets but also to RTP. Even RTP packets are not able to transverse accross private -public network interfaces to the right place across a NAT’d connection.

To solve two-way media RTP performs RTP latching, where client listens for at least one RTP frame arriving at the destination port it advertised, and harvests the source IP and port from that packet and uses that for the return RTP path. RTP latching works out of the box for public RTP endpoints but not for ones behind NAT.

It is thus recommended to use an intermediate RTP relay such as RTPengine on kamailio. It is controlled via a UDP control socket by kamailio as an external process. More on installation and descrition of RTP engine on kamailio is covered here. When RTPengine control module receives RTP offer /answer from Kamailio, it opens a pair of RTP/RTCP ports to receive traffic and substitues in SDP. Doing so for both ends makes RTP engine come in the path of media stream packets for both directions.

A first INVITE has no no corresponding on NAT ( as no port has been allocated yet ) so the c= contact and m = media lines would look like

c=IN IP4 192.0.2.13.
m=audio 23767 RTP/AVP 0 101.

We can force RTP to go through a proxy by changing c line and m line to

c=IN IP4 address-of-proxy  
m=audio port-on-proxy RTP/AVP 0 101

A separate daemon called an RTP proxy (with a public IP address) that both user agents can send their audio to, can be setup by calling force_rtp_proxy().

Fixing NAT

When the client is behind NAT, following needs to be taken careof to provide smooth operation

  1. Ensuring Tranactional replies are sent to correct source address ( maybe using ;rport param and forcerport() method ) instead of just relying on via header transport protocol and port.
    example:
if (client_nat_test("3")){
    //CALL RE-INVITE/UPDATE Nat DETECTED $ci\n");
    force_rport();
    fix_contact();
    ...
}

also Change Media ip address to public IP

if(nat_uac_test("8") && search("Content-type: application/sdp")) {
        // RE-INVITE/UPDATE CALL fix SDP- NAT
        fix_nated_sdp("2");
}

Any far-end NAT traversal solution ( TURN server) if employed should stay in path of entire Dialog not just for initial INVITE transaction which many times results in ACK being dropped. This can be achived by adding Record-Route header of rr module to the initial INVITE request itself

Set the advertised address of the public-facing inetrface to the Public NAT IP using “listen” parameter

Ensure contact URI is NAT processed by using NATHelper modules which rewrites the domain portion of the Contact URI to contain the source IP and port of the request or reply. add_contact_alias([ip_addr, port, proto]) in NAThelper module which adds “;alias=ip~port~transport” parameter to the contact URI containing either received ip, port, and transport protocol or those given as parameters

Implement RTP proxy which performs NAT for streams such as rtpengine module

NAT Traversal Module

Provides far-end NAT traversal to kamailio’s SIP signalling. Its role is

  • detect user agents behind NAT
  • manipulate SIP headers so that user agents can continue working behind NAT transparently
  • keepalives to UA behind NAT to preserve their visibility in network

Some pros and cons of NATTravsersal module

  • (+) detect even UAs behind multiple cascaded NAT boxes, complex distributed env with multiple proxies
  • (+) handle env where incoming and outgoing paths are diff for SIP messages
  • (+) handle cases when routing path may even change between consecutive dialogs
  • (+) can work for other than registered UA’s also
  • (-) built for IPv4 NAT handling not adapted to support IPv6 session keepalives.

Why use keepalive when Registrations are already there for NATing ?

  1. NAT binding works for registered users who want incoming calls. However for cases like outgoing calls or for presence subscription notifications, failings registration implies inability to receive further in-dialog messages after the NAT binding expires. This artificial binding for registrations makes system unreliable and volatile as it doesnot guarantee the delivery of in-dialog messages for outgoing calls without registration renewal. Therefore keepalive are adopted which also works for unregistered users.
  2. Minimizes the traffic as only border proxies send keepalives which send keepalives statelessly, instead of having to relay messages generated by the registrars.
  3. Also for situations when DNS resolves diff proxies for outgoing or incoming path traditional register based keepalives fail to associate or dissociate correct routes.

How keepalives work for NATing ?

This mechanism works by sending a SIP request to a user agent behind NAT to make that user agent send back a reply. The purpose is to have packets sent from inside the NAT to the proxy often enough to prevent the NAT box from timing out the connection.

Module sends Keeplaives to preserve their visibility only in :

  • Registration – for user agent that have registered to for incoming calls, triggering keepalive for a REGISTER request.
  • Subscription – for presence agents that have subscribed to some events for receiving back notifications with SUBSCRIBE request.
  • Dialogs – for user agents that have initiated an outgoing call for receiving further in-dialog messages.
    When all the conditions to keepalive a NAT endpoint will disappear, that endpoint will be removed from the list with the NAT endpoints that need to be kept alive.

function nat_keepalive()

  • the function needs to be called on proxy directly interacting with UA behind NAT.
  • call only once for the requests (REGISTER, SUBSCRIBE or outgoing INVITEs) that triggers the need for network visibility.
  • call before the request gets either a stateless reply or it is relayed with t_relay()
  • for outgoing INVITE , it triggers dialog tracing for that dialog and will use the dialog callbacks to detect changes in the dialog state.

Dependencies – sl , tm and dialog module

Params

  • keepalive_interval – time interval between sending a keepalive message to all the endpoints that need being kept alive. A negative value or zero will disable the keepalive functionality.
modparam("nat_traversal", "keepalive_interval", 30) // 30 seconds keeplaive inetrval
  • keepalive_method – SIP method to use to send keepalive messages.usual ones are NOTIFY and OPTIONS. Default value is “NOTIFY”.
modparam("nat_traversal", "keepalive_method", "OPTIONS")
  • keepalive_from – SIP URI to use in the From header of the keepalive requests. default sip:keepalive@proxy_ip,with IP address of the outgoing interface
modparam("nat_traversal", "keepalive_from", "sip:keepalive@altanai.com")
  • keepalive_extra_headers – extra headers that should be added to the keepalive messages. Header must also include the CRLF (\r\n) line separator. Multiple headers can be specified by concatenating with \r\n separator.
modparam("nat_traversal", "keepalive_extra_headers", "User-Agent: Kamailio\r\nX-MyHeader: some_value\r\n")
  • keepalive_state_file – filename where information about the NAT endpoints and the conditions for which they are being kept alive is saved . It is used when Kamailio starts to restore its internal state and continue to send keepalive messages to the NAT endpoints that have not expired in the meantime. Also used at kamailio restart as it avoids losing keepalive state information about the NAT endpoints.
modparam("nat_traversal", "keepalive_state_file", "/var/run/kamailio/keepalive_state")

Functions

  • client_nat_test ()– Check if the client is behind NAT. Tests to be performed gievn by int can be :
    1 – tests if client has a private IP address or one from shared address space in the Contact field of the SIP message.
    2 – tests if client has contacted Kamailio from an address that is different from the one in the Via field.
    4 – tests if client has a private IP address or one from shared address space in the top Via field of the SIP message.

For example calling client_nat_test(“3”) will perform test 1 and test 2 and return true if at least one succeeds, otherwise false.

  • fix_contact() – replace the IP and port in the Contact header with the IP and port the SIP message was received from. Usually called after a succesfull call to client_nat_test(type)
if (client_nat_test("3")) {
    fix_contact();
}
  • nat_keepalive() – Triggers keepalive functionality for the source address of the request. When called it only sets some internal flags, which will trigger later the addition of the endpoint to the keepalive list if a positive reply is generated/received (for REGISTER and SUBSCRIBE) or when the dialog is started/replied (for INVITEs). For this reason, it can be called early or late in the script. The only condition is to call it before replying to the request or before sending it to another proxy. If the request needs to be sent to another proxy, t_relay() must be used to be able to intercept replies via TM or dialog callbacks.

If stateless forwarding is used, the keepalive functionality will not work. Also for outgoing INVITEs, record_route() should also be used to make sure the proxy that keeps the caller endpoint alive stays in the path.

if ((method=="REGISTER" || method=="SUBSCRIBE" ||
    (method=="INVITE" && !has_totag())) && client_nat_test("3"))
{
    nat_keepalive();
}

Pseudo Variables

  • $keepalive.socket(nat_endpoint)
  • $source_uri

Statistics

  • keepalive_endpoints – total number of NAT endpoints that are being kept alive.
  • registered_endpoints – NAT endpoints kept alive for registrations
  • subscribed_endpoints – NAT endpoints kept alive for subscriptions.
  • dialog_endpoints – Indicates how many of the NAT endpoints are kept alive for taking part in an INVITE dialog.

NATHelper Module

NAT traversal and reuse of TCP connections.
Helps symmetric UAs who are not able to determine their public address.

NAT pinging types

UDP packet
4 bytes (zero filled) UDP packets are sent to the contact address.
SIP request
a stateless SIP request is sent to the UDP contact address.
(+) low bandwitdh traffic,
(+) easy to generate by Kamailio;
(+) bidirectional traffic through NAT
(+) since each PING request from Kamailio (inbound traffic) will force the SIP client to generate a SIP reply (outbound traffic) – the NAT bind will be surely kept open.
(-) unidirectional traffic through NAT (inbound – from outside to inside);
if many NATs do update the bind timeout only on outbound traffic, the bind may expire and closed.
(-) higher bandwitdh traffic
(-) more expensive (as time) to generate by Kamailio;

Dependencies – usrloc

Params

  • force_socket – Socket to be used when sending NAT pings for UDP communication.
modparam("nathelper", "force_socket", "127.0.0.1:5060")
  • natping_interval
  • ping_nated_only
  • natping_processes – How many timer processes should be created by the module for the exclusive task of sending the NAT pings.
  • natping_socket
  • received_avp – AVP used to store the URI containing the received IP, port, and protocol by fix_nated_register
  • sipping_bflag
  • sipping_from
  • sipping_method
  • natping_disable_bflag
  • nortpproxy_str
  • keepalive_timeout
  • udpping_from_path
  • append_sdp_oldmediaip
  • filter_server_id

Functions

  • fix_nated_contact() – rewrites the “Contact” header field with request’s source address:port pair
  • fix_nated_sdp() – adds the active direction indication to SDP and updates ource ip address information too
  • add_rcv_param() – add a received parameter to the “Contact” header fields or the Contact URI.
  • fix_nated_register() exports the request’s source address:port into an AVP to be used during save()
  • nat_uac_test()- check if client’s request originated behind a nat
  • is_rfc1918()
  • add_contact_alias() – Adds an “;alias=ip~port~transport” parameter to the contact URI
  • handle_ruri_alias() – Checks if the Request URI has an “alias” parameter and if so, removes it and sets the “$du” based on its value.
  • set_contact_alias()

Pseudo Variables

  • $rr_count – Number of Record Routes in received SIP request or reply.
  • $rr_top_count – If topmost Record Route in received SIP request or reply is a double Record Route, value of $rr_top_count is 2.

RPC Commands

nathelper.enable_ping 

References :

VoIP system DevOps, operations and Infrastructure management, Automation


Overview of VoIP platform DevOPS tools

This article is focussed around various tools required to operate and maintain a growing large scale VoIP Platform, which are mostly classified under following roles:

  • PCAP Collections
  • CICD on Jenkins pipeline
  • Configuration management using chef cookbooks
  • virtualization and containerization using Docker
  • Infrastructure management using terraform / Kubernetes
  • Logs Analysis and Alarming

PCAP Collection

Packet Capture (PCAP) is an API that captures live network packets. Besides tracking, audit and RTC visualizers, PCAP is widely used for debugging faults such as during production alarm on high failure occurrences.

Example usecase: Production alert on 503 SIP response or log entry from a gateway is not as helpful as PCAP tracking of the session ID of call across various endpoints in and out of the network to determine the point of failure.Debugging involves :

  1. Pre-specified SIP / RTP and related protocols capture 

Capture pcaps examples

tcpdump -i any -w alltraffic.pcap
rtpbreak -P2 -t100 -T100 -d logz -r alltraffic.pcap

2. Call SessionId to uniquely identify failed calls among tens of thousands of the packet 

3. Analyzer such as wireshark or tshark to track the packet

TShark inspection examples

brew cask install wireshark
tshark -r alltraffic.pcap -R "sip.CSeq.method eq INVITE"

Some of the useful call specs captured from PCAP

  • DTMF – Both in-band and out of band DTMF for every call, along with the time stamp.
  • Codec negotiations –  Extracting codecs from PCAP lets us 
    1. Validate later whether there were codec changes without prior SIP message,
    2. If the call has been hung up with 488 error code then it was due to which  codec 
  • SIP errors – track deviations from standard SIP messaging. 
    1. Identify known erroneous SIP messaging scenarios such as for MITM or replay attacks
  • RTCP Media stats – extract Jitter, Loss, RTT with RTCP reports for both the incoming and outgoing stream.
  • Identify Media or ACK Timeouts 
    1. Check whether a party has not sent any media packet for > 60 s (media time out threshold duration)
    2. When a call is hung up due to ACK time out.
  • Audio stream – After GDPR, take explicit permission from users before storing audio streams.
PCAP file analyzed in Wireshark ( PCAP source : https://wiki.wireshark.org/SampleCaptures#Sample_Captures)

Continuous Integration and Delivery Automation using Jenkins

CICD provides continous delivery hub , distribute work across multiple machines, helping drive builds, tests and deployments across multiple platforms .

Jenkins jobs is a self-contained Java-based program extensible using plugins.

Jenkins pieline– orchestrates and automates building project in Jenkins

Configuration management using chef cookbooks

Alternatives like puppet and Ansible, which are also a cross-platform configuration management platform

Compute virtualization and containerization using Docker

Docker containers can be used instead of virtual machines such as VirtualBox , to isolates applications and be OS and platform independent
Makes distributed development possible and automates the deployment possible

  • stop Stop one or more running containers
  • top Display the running processes of a container
> docker top 4417600169e8
UID PID PPID C STIME TTY TIME CMD
root 9913 9888 0 08:50 ? 00:00:00 bash /point.sh
root 10083 9913 0 08:50 ? 00:00:01 /usr/sbin/worker
root 10092 10083 0 08:50 ? 00:00:02 /micro-service
  • unpause Unpause all processes within one or more containers
  • update Update configuration of one or more containers
  • wait Block until one or more containers stop, then print their exit codes

see all iamges

> docker images
REPOSITORY                  TAG                 IMAGE ID            CREATED             SIZE
sipcapture/homer-cron       latest              fb2243f90cde        3 hours ago         476MB
sipcapture/homer-kamailio   latest              f159d46a22f3        3 hours ago         338MB
sipcapture/heplify          latest              9f5280306809        21 hours ago        9.61MB
<none>                      <none>              edaa5c708b3a        

See all stats

>  docker stats
CONTAINER ID        NAME                CPU %               MEM USAGE / LIMIT     MEM %               NET I/O             BLOCK I/O           PIDS
f42c71741107        homer-cron          0.00%               52KiB / 994.6MiB      0.01%               2.3kB / 0B          602MB / 0B          0
0111765091ae        mysql               0.04%               452.2MiB / 994.6MiB   45.46%              1.35kB / 0B         2.06GB / 49.2kB     22

Run command from within a docker instnace

docker exec -it  bash

First see all processes

docker ps

select a process and enter its bash

docker exec -it 0472a5127fff bash

to edit or update a file inside docker either install vim everytime u login in resh docker conainer like

apt-get update
apt-get install vim

or add this to dockerfile

RUN [“apt-get”, “update”]
RUN [“apt-get”, “install”, “-y”, “vim”]

see if ngrep is install , if not then install and run ngrep to get sip logs isnode that docker container

apt update
apt install ngrep
ngrep -p "14795778704" -W byline -d any port 5060

docker volume – Volumes are used for persisting data generated by and used by Docker containers.
docker volumes have advantages over blind mounts such as easier to backup or migrate , managed by docker APIs, can be safely shared among multiple containers etc

docker stack – Lets to manager a cluster of docker containers thorugh docker swarm can be defined via docker-compose.yml file

docker service

  • create Create a new service
  • inspect Display detailed information on one or more services
  • logs Fetch the logs of a service or task
  • ls List services
  • ps List the tasks of one or more services
  • rm Remove one or more services
  • rollback Revert changes to a service’s configuration
  • scale Scale one or multiple replicated services
  • update Update a service

Run docker containers

sample run command

docker run -it -d --name opensips -e ENV=dev imagename:2.2

-it flags attaches to an interactive tty in the container.
-e gives envrionment variables
-d runs it in background and prints container id

Remove docker entities

To remove all stopped containers, all dangling images, and all unused networks:

docker system prune -a

To remove all unused volumes

docker system prune --volumes

To remove all stopped containers

docker container prune
sometimes docker images keep piling with stopped congainer such as 

REPOSITORY                                                             TAG                 IMAGE ID            CREATED             SIZE                                                                              d1dcfe2438ae        15 minutes ago      753MB                                                                           2d353828889b        16 hours ago        910MB                                                          ...
CONTAINER ID        IMAGE               COMMAND                  CREATED             STATUS                        PORTS               NAMES

0dd6698a7517        2d353828889b        "/entrypoint.sh"         13 minutes ago      Exited (137) 13 minutes ago                       hardcore_wozniak

to remove such images and their conainer , first stop and remove confainers

docker stop $(docker ps -a -q)
docker rm $(docker ps -a -q)

then remove all dangling images

docker rmi  $(docker images -aq --filter dangling=true)

Infrastructure management using terraform

Terraform is used for building, changing and versioning infrastructure.
Infra as Code – can run single application to datacentres via configuration files which create execution plan.
It can manage low-level components such as compute instances, storage, and networking, as well as high-level components such as DNS entries, SaaS features, etc.
Resource Graph – builds a graph of all your resources

tfenv can be used to manage terraform versions

> brew unlink terraform
tfenv install 0.11.14
tfenv list 

Terraform configuration language

This is used for declaring resources and descriptions of infrastructure and associated files have a .tf or .tf.json file extension
Group of resources can be gathered into a module. Terraform configuration consists of a root module, where evaluation begins, along with a tree of child modules created when one module calls another.

Example : launch a single AWS EC2 instance , fle server1.tf

provider "aws" {
  profile    = "default"
  region     = "us-east-1"
}

resource "aws_instance" "server1" {
  ami           = "ami-2757fxxx"
  instance_type = "t2.micro"
}

note : AMI IDs are region specific.
profile attribute here refers to the AWS Config File in ~/.aws/credentials

Terraform command line interface (CLI)

engine for evaluating and applying Terraform configurations.
uses plugins called providers that each define and manage a set of resource types

Command Usage: terraform [-version] [-help] [args]

  • apply Builds or changes infrastructure
  • console Interactive console for Terraform interpolations
  • destroy Destroy Terraform-managed infrastructure
  • env Workspace management
  • fmt Rewrites config files to canonical format
  • get Download and install modules for the configuration
  • graph Create a visual graph of Terraform resources
  • import Import existing infrastructure into Terraform
  • init Initialize a Terraform working directory
  • output Read an output from a state file
  • plan Generate and show an execution plan
  • providers Prints a tree of the providers used in the configuration
  • refresh Update local state file against real resources
  • show Inspect Terraform state or plan
  • taint Manually mark a resource for recreation
  • untaint Manually unmark a resource as tainted
  • validate Validates the Terraform files
  • version Prints the Terraform version
  • workspace Workspace management
  • 0.12upgrade Rewrites pre-0.12 module source code for v0.12
  • debug Debug output management (experimental)
  • force-unlock Manually unlock the terraform state
  • push Obsolete command for Terraform Enterprise legacy (v1)
  • state Advanced state management

terraform init
Initialize a working directory containing Terraform configuration files.

terraform validate
checks that verify whether a configuration is internally-consistent, regardless of any provided variables or existing state.

Kubernetes

container orchestration platform , automating deployment, scaling, and management of containerized applications. Can deploy to cluster of computers, automating the distribution and scheduling as well

Service discovery and load balancing – gives Pods their own IP addresses and a single DNS name for a set of Pods, and can load-balance across them.

Automatic bin packing – Automatically places containers based on their resource requirements and other constraints, while not sacrificing availability. Mix critical and best-effort workloads in order to drive up utilization and save even more resources.

Storage orchestration – Automatically mount the storage system of your choice, whether from local storage, a public cloud provider such as GCP or AWS, or a network storage system such as NFS, iSCSI, Gluster, Ceph, Cinder, or Flocker.

Self-healing – Restarts containers that fail, replaces and reschedules containers when nodes die, kills containers that don’t respond to your user-defined health check, and doesn’t advertise them to clients until they are ready to serve.

Automated rollouts and rollbacks – progressively rolls out changes to your application or its configuration, while monitoring application health to ensure it doesn’t kill all your instances at the same time.

Secret and configuration management – Deploy and update secrets and application configuration without rebuilding your image and without exposing secrets in your stack configuration.

Batch execution– manage batch and CI workloads, replacing containers that fail, if desired.

Horizontal scaling – Scale application up and down with a simple command, with a UI, or automatically based on CPU usage.

create minikube cluster and deploy pods

prerequisities : docker , curl , redis , others

install minikube

curl -Lo minikube https://storage.googleapis.com/minikube/releases/latest/minikube-linux-amd64
chmod +x minikube
install minikube /usr/local/bin

Install kubectl

snap install kubectl --classic
ln -s /snap/bin/kubectl /usr/local/bin

Setup Minikube

minikube start --vm-driver=none
minikube addons enable registry-creds
kubectl -n kube-system create secret generic registry-creds-ecr
kubectl -n kube-system create secret generic registry-creds-gcr
kubectl -n kube-system create secret generic registry-creds-dpr
minikube addons configure registry-creds
Starting Kubernetes…minikube version: v1.3.0
 commit: 43969594266d77b555a207b0f3e9b3fa1dc92b1f
 minikube v1.3.0 on Ubuntu 18.04
 Running on localhost (CPUs=2, Memory=2461MB, Disk=47990MB) …
 OS release is Ubuntu 18.04.2 LTS
 Preparing Kubernetes v1.15.0 on Docker 18.09.5 …
 kubelet.resolv-conf=/run/systemd/resolve/resolv.conf
 Pulling images …
 Launching Kubernetes …
 Done! kubectl is now configured to use "minikube"
 dashboard was successfully enabled
 Kubernetes Started 

Basic Commands

  • start Starts a local kubernetes cluster
  • status Gets the status of a local kubernetes cluster
  • stop Stops a running local kubernetes cluster
  • delete Deletes a local kubernetes cluster
  • dashboard Access the kubernetes dashboard running within the minikube cluster

Images Commands:

  • docker-env Sets up docker env variables; similar to ‘$(docker-machine env)’
  • cache Add or delete an image from the local cache.

Configuration and Management Commands:

  • addons Modify minikube’s kubernetes addons
  • config Modify minikube config
  • profile Profile gets or sets the current minikube profile
  • update-context Verify the IP address of the running cluster in kubeconfig.

Networking and Connectivity Commands:

  • service Gets the kubernetes URL(s) for the specified service in your local cluster
  • tunnel tunnel makes services of type LoadBalancer accessible on localhost

Advanced Commands:

  • mount Mounts the specified directory into minikube
  • ssh Log into or run a command on a machine with SSH; similar to ‘docker-machine ssh’
  • kubectl Run kubectl

Troubleshooting Commands:

  • ssh-key Retrieve the ssh identity key path of the specified cluster
  • ip Retrieves the IP address of the running cluster
  • logs Gets the logs of the running instance, used for debugging minikube, not user code.
  • update-check Print current and latest version number

kubectl

controls the Kubernetes cluster manager.

Basic Commands (Beginner):

  • create Create a resource from a file or from stdin.
  • expose Take a replication controller, service, deployment or pod and expose it as a new Kubernetes Service
  • run Run a particular image on the cluster
  • set Set specific features on objects
  • explain Documentation of resources
  • get Display one or many resources
  • edit Edit a resource on the server
  • delete Delete resources by filenames, stdin, resources and names, or by resources and label selector

Deploy Commands:

  • rollout Manage the rollout of a resource
  • scale Set a new size for a Deployment, ReplicaSet, Replication Controller, or Job
  • autoscale Auto-scale a Deployment, ReplicaSet, or ReplicationController

Cluster Management Commands:

  • certificate Modify certificate resources.
  • cluster-info Display cluster info
  • top Display Resource (CPU/Memory/Storage) usage.
  • cordon Mark node as unschedulable
  • uncordon Mark node as schedulable
  • drain Drain node in preparation for maintenance
  • taint Update the taints on one or more nodes

Troubleshooting and Debugging Commands:

  • describe Show details of a specific resource or group of resources
  • logs Print the logs for a container in a pod
  • attach Attach to a running container
  • exec Execute a command in a container
  • port-forward Forward one or more local ports to a pod
  • proxy Run a proxy to the Kubernetes API server
  • cp Copy files and directories to and from containers.
  • auth Inspect authorization

Advanced Commands:

  • diff Diff live version against would-be applied version
  • apply Apply a configuration to a resource by filename or stdin
  • patch Update field(s) of a resource using strategic merge patch
  • replace Replace a resource by filename or stdin
  • wait Experimental: Wait for a specific condition on one or many resources.
  • convert Convert config files between different API versions
  • kustomize Build a kustomization target from a directory or a remote url.

Settings Commands:

  • label Update the labels on a resource
  • annotate Update the annotations on a resource
  • completion Output shell completion code for the specified shell (bash or zsh)

Other Commands:

  • api-resources Print the supported API resources on the server
  • api-versions Print the supported API versions on the server, in the form of “group/version”
  • config Modify kubeconfig files
  • plugin Provides utilities for interacting with plugins.
  • version Print the client and server version information

DevOps monitoring tools nagios

Manage Docker configs

  • create Create a config from a file or STDIN
  • inspect Display detailed information on one or more configs
  • ls List configs
  • rm Remove one or more configs

Manage containers

  • attach Attach local standard input, output, and error streams to a running container
  • commit Create a new image from a container’s changes
  • cp Copy files/folders between a container and the local filesystem
  • create Create a new container
  • diff Inspect changes to files or directories on a container’s filesystem
  • exec Run a command in a running container
  • export Export a container’s filesystem as a tar archive
  • inspect Display detailed information on one or more containers
  • kill Kill one or more running containers
  • logs Fetch the logs of a container
  • ls List containers
  • pause Pause all processes within one or more containers
  • port List port mappings or a specific mapping for the container
  • prune Remove all stopped containers
  • rename Rename a container
  • restart Restart one or more containers
  • rm Remove one or more containers
  • run Run a command in a new container
  • start Start one or more stopped containers
  • stats Display a live stream of container(s) resource usage statistics
  • stop Stop one or more running containers
  • top Display the running processes of a container
  • unpause Unpause all processes within one or more containers
  • update Update configuration of one or more containers
  • wait Block until one or more containers stop, then print their exit codes

Alternatives, Senu multi-cloud monitoring or Raygun

Monitoring, debugging, logs analysis and alarms

Aggregate logs into logstash and provide search and filtering via Elastic Search and Kibana. Can also trigger alerts or notifications on specific keyword searches in logs such as WARNING or ERRRO or call_failed. Some common alert scenarios include :

SBC and proxy gateways failures – check states of VM instance

DNS caching alerts – Domain Name System (DNS) caching, a Dynamic Host Configuration Protocol (DHCP) server, router advertisement and network boot alerts from service such as dnsmasq

Disk usage alert – setup alerts for 80% usage and trigger an alarm to either manually prune or create automatic timely archive backups.
check the percentage of DISK USAGE

df -h

Mostly it is either the logs file or pcap recorder which need to be archieved in external storage.

Use logrotate – it can rotates, compresses, and mails system logs

config file for logrorate – logrotate -vf /etc/logrotate.conf

/var/log/messages {
    rotate 5
    weekly
    postrotate
        /usr/bin/killall -HUP syslogd
    endscript
}

Elevated Call failure SIP 503 or Call timeout SIP 408 – high frequency of failed calls indicate an internal issue and must be followed up by smoke testing the entire system to identify any probable issue such as undetected frequent crashes of any individual component or any blacklisting by a destination endpoint etc

sudo tail -f sip.log | grep 503

or

sudo tail -f sip.log | grep WARNING

cron service or processed alerts

 ps axf
  PID TTY      STAT   TIME COMMAND
    2 ?        S      0:00 [kthreadd]
    3 ?        I<     0:00  \_ [rcu_gp]
    4 ?        I<     0:00  \_ [rcu_par_gp]
    5 ?        I      0:00  \_ [kworker/0:0-eve]
    6 ?        I<     0:00  \_ [kworker/0:0H-kb]
    7 ?        I      0:00  \_ [kworker/0:1-eve]
    8 ?        I      0:00  \_ [kworker/u4:0-nv]
    9 ?        I<     0:00  \_ [mm_percpu_wq]
   10 ?        S      0:00  \_ [ksoftirqd/0]
   11 ?        I      0:00  \_ [rcu_sched]
   12 ?        S      0:00  \_ [migration/0]
   13 ?        S      0:00  \_ [cpuhp/0]
   14 ?        S      0:00  \_ [cpuhp/1]
   15 ?        S      0:00  \_ [migration/1]
   16 ?        S      0:00  \_ [ksoftirqd/1]
   17 ?        I      0:00  \_ [kworker/1:0-eve]
   18 ?        I<     0:00  \_ [kworker/1:0H-kb]

or checks cron status

service cron status
● cron.service - Regular background program processing daemon
   Loaded: loaded (/lib/systemd/system/cron.service; enabled; vendor preset: enabled)
   Active: active (running) since Fri 2016-06-26 03:00:37 UTC; 1min 17s ago
     Docs: man:cron(8)
 Main PID: 845 (cron)
    Tasks: 1 (limit: 4383)
   CGroup: /system.slice/cron.service
           └─845 /usr/sbin/cron -f

Jun 26 03:00:37 ip-172-31-45-21 systemd[1]: Started Regular background program processing daemon.
Jun 26 03:00:37 ip-172-31-45-21 cron[845]: (CRON) INFO (pidfile fd = 3)
Jun 26 03:00:37 ip-172-31-45-21 cron[845]: (CRON) INFO (Running @reboot jobs)

restart or start cron service if required

DB connections / connection pool process – keep listening for any alerts on DB connections failure or even warnings as this can be due to too many read operations such as in DDOS and can escalate very quickly

netstat -nltp  | grep db 
tcp        0      0 0.0.0.0:5433            0.0.0.0:*               LISTEN      5792/db-server * 

Routine deepstatus checks is a good practice too. Raise alert if any check doesnt result as expected.

Port check, unexpected result alert– Regular checks if servers are lsietning on ports such as 5060 for SIP

netstat -nltp | grep 5060
tcp        0      0 x.x.x.x:5060       0.0.0.0:*               LISTEN      8970/kamailio  

cron zombie process checks – zombie process or defunct process is a process that has completed execution (via the exit system call) but still has an entry in the process table: it is a process in the “Terminated state”. List xombie process and kill them with pid to free up .

kill -9 <PID1>

Bulk calls checks – consult ongoing call cmd commands for application server such as
For Freeswitch use

fs_ctl> show channels 

For kamailio use kamcmd

kamcmd dlg.list

For asterisk watch or show cmmand

watch -n 1 "sudo asterisk -vvvvvrx 'core show channels' | grep call"

Incase of DDOS or other macious attacker IP identification block the IP

iptables -I INPUT -s y.y.y.y -j DROP   

Can also use fail2ban

>apt-get update && apt-get install fail2ban

Additionally check how many dispatchers are responding on outbound gateway

opensipsctl dispatcher dump

Process control supervisor or pm2 checks – supervisor is a Linux Process Control System that allows its users to monitor and control a number of processes

ps axf | grep supervisor

for pm2

> pm2 status
[PM2] Spawning PM2 daemon with pm2_home=/Users/altanai/.pm2
[PM2] PM2 Successfully daemonized
┌─────┬───────────┬─────────────┬─────────┬─────────┬──────────┬────────┬──────┬───────────┬──────────┬──────────┬──────────┬──────────┐
│ id │ name │ namespace │ version │ mode │ pid │ uptime │ ↺ │ status │ cpu │ mem │ user │ watching │

htop to check memeory and CPU

Health and load on the reverse proxy, load balancer as Nginx – perform a direct curl request to host to check if Nginx responds with a non 4xx / 5xx response or not

curl -v <public-fqdn-of-server> 

Incase of error response , restart

/etc/init.d/nginx start

Incase of updates restart ngnix config

nginx -s reload

For HTTP/SSL proxy daemon such as tiny proxy which are used for fast resposne , set the MinSpareServers, MaxSpareServers , MaxClients , MaxRequestsPerChild etc appropriately

VPN checks – restart fireealls or IPsec incase of ssues

/etc/init.d/ipsec restart

Additionally also check ssh service

ps axf | grep sshd

restart sshd if required

SSL cert expiry checks – to keep the operations running securely and prevent and abrupt termination it is a good practise to run regular certificate expiry checks for SSL certs especially on secure HTTP endpoint like APIs , web server and also on SIP applications servers for TLS. If any expiry is due in < 10 days to trigger an alert to renew the certs

Health of Task scheduling services such as RabbitMQ, Celery Distributed Task Queue – remote debugging of these can be set up via pdb which supports setting (conditional) breakpoints and single stepping at the source line level, inspection of stack frames, source code listing, and evaluation of arbitrary Python code in the context of any stack frame.

import pdb; pdb.set_trace()
python3 -m pdb myscript.py

It can also be set up via using the client libraries provided by these Queue services themselves

Cluster status – setup an efficient health check service which monitors the cluster status for High Availability. JSON object depicting the status of cluster shards

{
  "cluster_name" : "ABC-cluster",
  "status" : "green",
  "timed_out" : false,
  "number_of_nodes" : 14,
  "number_of_data_nodes" : 6,
  "active_primary_shards" : 200,
  "active_shards" : 300,
  "relocating_shards" : 0,
  "initializing_shards" : 0,
  "unassigned_shards" : 0,
  "delayed_unassigned_shards" : 0,
  "number_of_pending_tasks" : 0,
  "number_of_in_flight_fetch" : 0,
  "task_max_waiting_in_queue_millis" : 0
}

Status of Crticial Application Server

fscli > show status
UP 0 years, 0 days, 0 hours, 58 minutes, 33 seconds, 15 milliseconds, 58 microseconds
FreeSWITCH (Version 1.6.20 git 987c9b9 2018-01-23 21:49:09Z 64bit) is ready
3 session(s) since startup
0 session(s) - peak 1, last 5min 1
0 session(s) per Sec out of max 30, peak 1, last 5min 1
1000 session(s) max
min idle cpu 0.00/80.83
Current Stack Size/Max 240K/8192K

Programming or Syntax error in the production environment – mostly arising due to incomplete QA/testing before pushing new changes to production. Should trigger alerts for dev teams and meet with hot patches.

Many programing application development frameworks have inbuild libs for debugging , exceotion handling and reporting such as

  • backend service in Django
  • API service in Go

Distributed memory caching – redis , memcahe : Redis info shows the master -salve configuration for all the instances as well as their memeory and cpu status.

>redis-cli info
# Server
redis_version:6.0.4
redis_git_dirty:0
redis_mode:standalone
os:Darwin 18.7.0 x86_64
arch_bits:64
multiplexing_api:kqueue
atomicvar_api:atomic-builtin
gcc_version:4.2.1
tcp_port:6379

# Clients
connected_clients:1
client_recent_max_input_buffer:0
client_recent_max_output_buffer:0
blocked_clients:0
tracking_clients:0
clients_in_timeout_table:0

# Memory
used_memory:1065648
used_memory_human:1.02M
number_of_cached_scripts:0
maxmemory:0
allocator_frag_bytes:1123680
allocator_rss_ratio:1.00
rss_overhead_bytes:37888
mem_fragmentation_ratio:2.16
active_defrag_running:0
lazyfree_pending_objects:0

# Persistence
loading:0
rdb_changes_since_last_save:0
module_fork_last_cow_size:0

# Stats
total_connections_received:1
total_commands_processed:0
..

# Replication
role:master
connected_slaves:0
..

# CPU
used_cpu_sys:0.011198
used_cpu_sys_children:0.000000

# Modules

# Cluster
cluster_enabled:0

SMS service using smsc on Kannel : From the kannel servers, you should see the PANIC error (most of the time Assertion error crashing kannel):

grep PANIC /var/log/kannel/bearerbox.log

IF you are going to restart , Flush redis cache

sudo redis-cli FLUSHALL
sudo redis-cli SAVE

restart kannel

sudo /etc/init.d/kannel restart

If the carriers are throttling the SMS request , verify “ERROR” responses using

sudo grep -i "throttling" bearerbox.log

Alternatives include AWS logs services :

  • Scalyr logging
  • Sensu monitoring for multi-cloud monitoring using event pipeline

Read about VoIP/ OTT / Telecom Solution startup’s strategy for Building a scalable flexible SIP platform that includes :

  • Scalable and Flexible SIP platform building
  • Cluster SIP telephony Server for High Availability
  • Failure Recovery
  • Multi-tier cluster architecture
  • Role Abstraction / Micro-Service based architecture
  • Distributed Event management and Event-Driven architecture
  • Containerization
  • Autoscaling Cloud Servers
  • Open standards and Data Privacy
  • Flexibility for inter-working – NextGen911 , IMS , PSTN
  • security and Operational Efficiencies

Read more about SIP VoIP system Architecture which includes

  • Infrastructure Requirements
  • Integral Components of a VOIP SIP-based architecture
  • RTP ( Real-Time Transport Protocol ) / RtCP
  • SIP gateways, registrar, proxy, redirect, application
  • Developing SIP-based applications – basic call routing, media management
  • SIP platform Development – NAt and DNS , Cross-platform and integration to External Telecommunication provider landscape , Databases

References :


Opensips

It is an multi-functional, multi-purpose SIP server especially used in VoIP landscape as standalone SIP server or SBC ( Session Border Controller ) for inbound and outbound traffic by carriers, telecoms backend layers or ITSPs for call routing and trunking solutions. It can be deployed with Class4/5 Platforms, SIP Trunking , hosted or IP PBX setup , existing gateways/ Session Border Controllers, Application Servers, proxy server, Front-End Load Balancers, IMS Platforms, Call Center etc.

Combination of opensips working scenarios scripts with code is at https://github.com/altanai/opensipsexamples.

Features

Due to its very flexible and customisable routing engine it can be used in number of scenarios such as an SIP proxy or a  router and due to its high throughput it is widely recommended as an enterprise grade inbound/outbound proxy server.

  • Registrar
  • Router / proxy (lcr, dynamic routing, dialplan features)
  • Redirect server
  • Presence agent
  • Back-to-back User Agent
  • IM server
  • SIP to SMS gateway (bidirectional)
  • SIP to XMPP gateway for presence and IM (bidirectional)
  • Load-balancer or dispatcher
  • Inbound/front end for gateways/asterisk
  • SIP NAT traversal unit
  • Application server with custom logic

Since Opensips has emerged as a resilent SIP server , it is also used in specific usecases such as
– DID ( Direct Inward dialling ) for SIP trunking solutions ,
– Local Number Portability (LNP) providers,
– Canonical Name (CNAME) providers etc

It can act as Registrar with NAT traversal abilities (STUN, TURN, SIP pinging)

UDP , TCP , TLS , SCTP and WS transports over both IPv4 and IPv6
Additionally it can be connected to multiple networks ( Multihomed ) and do IP authentication or IP blacklisting

Class 4 routing capabilities in opensips include SIP aliases, Direct Inward Dialing , Speed dial, CPL,vDialplan , dispatcher with various algorithms, prefix-based routing to multiple carriers , failover support,
Load balancing , ENUM-based or Geolocation-based routing etc.
Some of the Class 5 capabilities in opensips are B2B , call queuing
UAC registration , authentication, mangling, White/Black list

SIP SIMPLE features as messaging , Presence , Busy Lamp Field (BLF), Shared Call/Line Appearance (SCA) , Bridged Line Appearance (BLA) , Message Waiting notifications (MWI), XCAP , Resource List Server ( RLS ) , XMPP , SMS gateway (AT and SMPP)

Although opensips has no built-in media capabilities, but it modules for external media engines for Media relaying (RTPProxy, MediaProxy, RTPEngine), Media transcoding (Sangoma D1 cards) , Codec manipulation.

I have explained the usage of these server components in my previous article on  SIP entities and Server here https://telecom.altanai.com/2013/07/13/sip-entities/

Modular Arhitecture

Opensips has majorly 2 parts core and addon-modules.

Opensips Core part is only a proxy stateless SIP server . It contains

  • SIP transport layer which supports UDP, TCP, TLS and WS for SIP. As per the listener in routing script transport protocols is selected .
  • SIP factory — the message parser and builder which can be used to add new headers or remove existing ones.
  • Routing script parser and interpreter for the routing script which loads it to the memory at the startup time. To load a new script server restart is required.
  • Memory and locking manager for the memory allocation and locking to prevent deadlocks and starvation. Although these arn’t accesible by route scripting, it can be configured at compile time.
  • Core script functions and variables which can be used in routing scripts in addition to the functions exported by add-on modules.

Interfaces

Events Interface

Used to notify external applications about events triggered internal to OpenSIPS such as
core events – E_CORE_THRESHOLD ,E_CORE_PKG_THRESHOLD , E_CORE_SHM_THRESHOLD , modules events , or even a custom event using raise_event() command

Statistics Interface

Provide insights to statistics of opensips in numerical results which could be used for services like  monitoring, load evaluation, realtime integration etc. The statictsics can be of two kinds :
1. counter like – variables that keep counting things that happened in OpenSIPS, like received requests, processed dialogs, failed DB queries, etc
2. computed values – variables that are calculated in realtime, like how much memory is used, the current load, active dialogs, active transactions, etc

These variable would reset form 0 at start sometimes even during runtime.

Binary Internal Interface

Provider communication between individual OpenSIPS instances. Used in cases such as failovers where dialogs needs to persist for service continuity. Hence with this interface one can replicate all the events related to the runtime data (creation / updating / deletion) to a backup OpenSIPS instance.

SQL interface and NoSQL interface

SQL interfaces provides interaction with Sql DB drivers and services such as MySQL, Postgres, Oracle, Berkeley, unixODBC etc , while NoSQL interface provides access to Redis, CouchBase, Cassandra, MongoDB, Memcached, and other databases which are more frequently implemented as external caches.

AAA interface definition

Currently, OpenSIPS supports the RADIUS driver for the AAA interface with upcoming support for DIAMETER.

Management interface

Allows the external applications to trigger predefined commands
Push data like setting a debug level, registering a contact etc
Fetch data like registered users, ongoing calls, get statistics etc
Trigger an internal action as reloading the data, sending a message so on

1. Functional SIP modules

SIP signalling modules such as B2B_ENTITIES , B2B_LOGIC , CALL CENTER ( for Inbound call center system ) , DIALOG , NAT_TRAVERSAL , NATHELPER
OPTIONS , REGISTRAR ,SIGNALING , UAC_REGISTRANT
TM (Transaction/stateful module) , SL (Stateless replier ) , SMS (SIP-to-SMS IM gateway)

SIP Routing modules such as CARRIERROUTE ( routing extension suitable for carriers) , CPL_C ( Call Processing Langugage interpreter ) ,
DISPATCHER , DROUTING ( Dynamic Routing / LCR ) , EMERGENCY ,ENUM ,
JABBER (JABBER IM and PRESENCE interconnection ) , IMC ( Instant Messaging Conferencing ),
LOAD_BALANCER , MSILO (SIP message silo) , RR ( Record-Route) , SCRIPT_HELPER ( Embedded SIP routing logic and dialog management) , OSP ( Open Settlement Protocol )

SIP messaging related , COMPRESSION , DIVERSION , IDENTITY ,MAXFWD , MANGLER
PATH , SIPMSGOPS ( SIP operations ) , TOPOLOGY_HIDING ,
UAC , UAC_AUTH , UAC_REDIRECT
URI , SST ( SIP Session Timer support )

Presence Modules like PRESENCE , PRESENCE_CALLINFO , PRESENCE_DIALOGINFO, PRESENCE_MWI (for Message Waiting Indication ) , PRESENCE_XCAPDIFF (for XCAP-DIFF event) , PRESENCE_XML
PUA , PUA_BLA , PUA_DIALOGINFO , PUA_MI , PUA_USRLOC , PUA_XMPP
B2B_SCA ( Back-to-Back Shared Call ), RLS ( Resource List Server )
XCAP , XCAP_CLIENT

2. Scripting modules

Script helper modules such as JSON , CFGUTILS , EXEC , TEXTOPS , AVPOPS , REGEX, MATHOPS , BENCHMARK ,
CARRIERROUTE , GFLAGS (Global shared flags )
PYTHON , LUA ,PERL , MMGEOIP ( MaxMind GeoIP )

Auth modules such as AUTH , AUTH_AAA ,AUTH_DB , PERMISSIONS

Accounting & Billing modules aas ACC ,CALL CONTROL

Dialplan Modules like ALIAS_DB , DIALPLAN , DOMAIN ( Multi-domain support ) , DOMAINPOLICY ,
GROUP , USERBLACKLIST , SPEEDDIAL ,PEERING ( Radius peering )

Data caching as DNS_CACHE , USRLOC ,SQL_CACHER

Traffic shaping module as PIKE ( Flood detector module ), QOS ,RATELIMIT ,FRAUD_DETECTION

3. Database modules

For SQL – DB_BERKELEY , DB_CACHEDB , DB_FLATSTORE , DB_HTTP , DB_MYSQL , DB_ORACLE ,DB_PERLVDB , DB_POSTGRES , DB_SQLITE
DB_TEXT , DB_UNIXODBC , DB_VIRTUAL

For noSQL – CACHEDB_CASSANDRA ,CACHEDB_COUCHBASE ,CACHEDB_LOCAL ,CACHEDB_MEMCACHED , CACHEDB_MONGODB , CACHEDB_REDIS , CACHEDB_SQL

4. External Integration modules

OpenSIPS API as EVENT_DATAGRAM , EVENT_FLATSTORE ( Text/File backend for events ), EVENT_ROUTE ,EVENT_RABBITMQ
EVENT_VIRTUAL ( Aggregator of event backends failover & balancing), EVENT_XMLRPC
MI_DATAGRAM ( DATAGRAM unix and network support for Management Interface )
MI_FIFO , MI_HTTP , MI_JSON , MI_XMLRPC_NG
HTTPD , PI_HTTP ( Provisioning Interface ) , STATISTICS

Media Relays
MEDIAPROXY – NAT traversal module
RTPENGINE – Connector to RTPengine external RTP relay
RTPPROXY – Connector to RTPproxy external RTP relay

non-SIP protocols modules such as AAA_RADIUS , H350 , LDAP – LDAP connector , stable
REST_CLIENT , SEAS ( Sip Express Application Server interface module), SIPCAPTURE , SIPTRACE ,
SNGTC ( Voice Transcoding in OpenSIPS using Sangoma hardware ),
SNMPStats , STUN , XMPP ( SIP-to-XMPP Gateway )

5. OpenSIPS protocols and infrastructure

CLUSTERER , TLS_MGM , PROTO_BIN ( Binary INterface protocol module to implements inter-OPENSIPS communication )
PROTO_HEP , PROTO_SCTP , PROTO_TCP, PROTO_TLS , PROTO_UDP , PROTO_WS , PROTO_WSS

How to Install and use Opensips on your VoIP platform

Install from git repo

git clone git@github.com:OpenSIPS/opensips.git opensips_head
install gcc
make all

Install from apt

apt-key adv --keyserver keyserver.ubuntu.com --recv-keys 049AD65B
echo "deb http://apt.opensips.org trusty 2.4-releases" >/etc/apt/sources.list.d/opensips.list

check if opensips is running

ps -uax|grep opensips

Configuration ( opensips.cfg )

configuring compilation flags for various compile time options use menuconfig.

For this first install the ncurses development library

apt-get install libncurses5-dev
make menuconfig

Running menuconfig post installation from path use osipsconfig

compiling modules

prerequisites

apt-get install build-essential openssl bison flex
make && make install

After succesfull installation check version

opensips -V
version: opensips 2.2.7 (x86_64/linux)
flags: STATS: On, DISABLE_NAGLE, USE_MCAST, SHM_MMAP, PKG_MALLOC, F_MALLOC, FAST_LOCK-ADAPTIVE_WAIT
ADAPTIVE_WAIT_LOOPS=1024, MAX_RECV_BUFFER_SIZE 262144, MAX_LISTEN 16, MAX_URI_SIZE 1024, BUF_SIZE 65535
poll method support: poll, epoll_lt, epoll_et, sigio_rt, select.
main.c compiled on  with gcc 4.8

A config file opensips.config has 3 main logical parts :

1.global parameters – network listeners, available transport protocols, forking (and number of processes), the logging

2.modules section – the modules that are to be loaded  with path to their .so file

3.routing logic – logic for routing sip traffic

Routes

OpenSIPS routing logic uses several types of routes. Each type of route is triggered by a certain event and allows you to process a certain type of message (request or reply).

route
SIP requests routing. The main ‘route’ block identified by ‘route{…}’ or ‘route[0]{…}’ is executed for each SIP request.
To send a reply or forward the request, explicit actions must be called inside the route block. in example below which sends 200 ok reply for each options request.

route {
if(is_method("OPTIONS")) {
sl_send_reply("200", "ok");
exit();
}
route(1);
}
route[1] {
forward();
}

branch_route
Handles different branches of a SIP request. if the branch is not dropped the branch will be automatically sent out. It is executed only by TM module after it was armed via t_on_branch(“branch_route_index”).

if (is_method("INVITE|BYE|SUBSCRIBE|UPDATE")) {
if(!t_is_set("branch_route")) t_on_branch("MANAGE_BRANCH");
}
branch_route[MANAGE_BRANCH] {
xdbg("new branch [$T_branch_idx] to $ru\n");
route(NATMANAGE);
}

or lookup location and discard branches where uri matches ip 1.2.3.4 by using drop()

route {
lookup("location");
t_on_branch("op3");
if(!t_relay()) {
sl_send_reply("500", "relaying failed");
}
}
branch_route[op3] {
if(uri=~"1\.2\.3\.4") {
drop();
}
}

failure_route
Failed transaction routing block. It contains a set of actions to be taken each transaction that received only negative replies (>=300) for all branches which completes the transaction. The ‘failure_route’ is executed only by TM module after it was armed via t_on_failure(“failure_route_index”).

if (is_method("INVITE")) {
if(!t_is_set("failure_route")) t_on_failure("MANAGE_FAILURE");
}

failure_route[MANAGE_FAILURE] {
route(NATMANAGE);
if (t_is_canceled()) {
exit;
}
}

or on failure relay to voice mail

route {
lookup("location");
t_on_failure("op1");
if(!t_relay()) {
sl_send_reply("500", "relaying failed");
}
}
failure_route[op1] {
if(is_method("INVITE")) {
t_relay("udp:voicemail.server.com:5060");
}
}

onreply_route
Reply routing block. It can be stateful (if bound to a transaction) or stateless (if global reply route).
If the reply is not dropped (only provisional replies can be), it will be injected and processed by the transaction engine. There are three types of onreply routes:

global – catches all replies and uses simple definition ‘onreply_route {…}’ or ‘onreply_route[0] {…}’.
Exmaple for “global” reply route set the whole transaction

route {
seturi("sip:bob@opensips.org");  first branch
append_branch("sip:alice@opensips.org");  second branch
t_on_reply("global");
t_on_branch("1");
t_relay();
}

onreply_route {
xlog("OpenSIPS received a reply from $si\n");
}

onreply_route[global] {
if (t_check_status("1[0-9][0-9]")) {
setflag(1);
log("provisional reply received\n");
if (t_check_status("183"))
drop;
}
}

per request/transaction – it catches all received replies belonging to a certain transaction and uses “t_on_reply()” at request time, in REQUEST ROUTE – named ‘onreply_route[N] {…}’.

per branch – it catches only the replies that belong to a certain branch from a transaction via “t_on_reply()” ) at request time, but in BRANCH ROUTE, when a certain outgoing branch is processed – named ‘onreply_route[N] {…}’.
Certain ‘onreply_route’ blocks can be executed by TM module for special replies. For this, the ‘onreply_route’ must be armed for the SIP requests whose replies should be processed within it, via t_on_reply(“onreply_route_index”).

Exmaple of reply route set for this branch only


branch_route[1] {
if ($rU=="alice")
t_on_reply("alice");
}

onreply_route[alice] {
xlog("received reply on the branch from alice\n");
}

error_route

executed automatically on meeting and error such as parsing error in SIP request processing, script assert failure. Performs error handling . The Default action is to discard request. In error_route, the following pseudo-variables are available to get access to error details:

$(err.class) - the class of error (now is '1' for parsing errors)
$(err.level) - severity level for the error
$(err.info) - text describing the error
$(err.rcode) - recommended reply code
$(err.rreason) - recommended reply reason phrase
error_route {
xlog("--- error route class=$(err.class) level=$(err.level)
info=$(err.info) rcode=$(err.rcode) rreason=$(err.rreason) ---\n");
xlog("--- error from [$si:$sp]\n+++++\n$mb\n++++\n");
sl_send_reply("$err.rcode", "$err.rreason");
exit;
}

local_route

executed automatically as TM created a new request, internally (no UAC side). This is a route intended to be used for message inspection, accounting and for applying last changes on the message headers. Routing and signaling functions are not allowed.

local_route {
if (is_method("INVITE") && $ru=~"@foreign.com") {
append_hf("P-hint: foreign request\r\n");
exit;
}
if (is_method("BYE") ) {
acc_log_request("internally generated BYE");
}
}

startup_route
Executed only once when OpenSIPS is started and before the processing of SIP messages begins. Used in initilization cases cases such as loading some data in the cache.

startup_route {
avp_db_query("select gwlist where ruleid==1",$avp(i:100));
cache_store("local", "rule1", "$avp(i:100)");
}

timer_route
Route executed periodically at a configured interval of time specified next to the name(in seconds).

timer_route[gw_update, 300] {
avp_db_query("select gwlist where ruleid==1",$avp(i:100));
$shv(i:100) =$avp(i:100);
}

event_route
execute script code when an event is triggered. If no way to handle the event specified, default will be synchronously.
Triggered by the event_route module when an event is raised by the OpenSIPS Event Interface such as event raised by the pike module when it decides an ip should be blocked called E_PIKE_BLOCKED or E_SCRIPT_EVENT etc ( checke events interface for more events)

event_route[E_PIKE_BLOCKED] {
xlog("The E_PIKE_BLOCKED event was raised\n");
}
event_route[E_PIKE_BLOCKED, async] {
xlog("The E_PIKE_BLOCKED event was raised\n");
}

Scripting Language

Opensips scripting provided more advanced controls

1. Core Keywords

Keywords specific to SIP messages which can be used mainly in ‘if’ expressions.
af – address family of the received SIP message. It is INET if the message was received over IPv4 or INET6 if the message was received over IPv6.

if(af==INET6) {
log("Message received over IPv6 link\n");
};

dst_ip – IP of the local interface where the SIP message was received.

if(dst_ip==127.0.0.1) {
log("message received on loopback interface\n");
};

dst_port – local port where the SIP packet was received

if(dst_port==5061)
{
log("message was received on port 5061\n");
};

from_uri – reference to the URI of ‘From’ header.

if(is_method("INVITE") &amp;amp;amp;amp;amp;amp;&amp;amp;amp;amp;amp;amp; from_uri=~".*@opensips.org")
{
log("the caller is from opensips.org\n");
};

method – SIP method of the message.

if(method=="REGISTER")
{
log("this SIP request is a REGISTER message\n");
};

msg:len – the size of the message

if(msg:len&amp;amp;amp;amp;amp;gt;2048)
{
sl_send_reply("413", "message too large");
exit;
};

$retcode – value returned by last function executed like $?. If tested after a call of a route, it is the value retuned by that route.

route {
route(1);
if($retcode==1)
{
log("The request is an INVITE\n");
};
}

route[1] {
if(is_method("INVITE"))
return(1);
return(2);
}

proto – transport protocol of the SIP message.

if(proto==UDP)
{
log("SIP message received over UDP\n");
};

status – status code of the reply.

if(status=="200")
{
log("this is a 200 OK reply\n");
};

1.10 src_ip – source IP address

if(src_ip==127.0.0.1)
{
log("the message was sent from localhost!\n");
};

1.11 src_port – source port of the SIP message (from which port the message was sent by previous hop).

if(src_port==5061)
{
log("message sent from port 5061\n");
}

1.12 to_uri – URI from To header.

if(to_uri=~"sip:.+@opensips.org")
{
log("this is a request for opensips.org users\n");
};

1.13 uri – request URI.

if(uri=~"sip:.+@opensips.org")
{
log("this is a request for opensips.org users\n");
};

2. Core Values

Values that can be used in ‘if’ expressions to check against Core Keywords

INET – IPv4 connection. , INET6 – IPv6 connection , TCP , UDP

max_len – test message’s size.

if(msg:len&amp;amp;amp;amp;amp;gt;max_len)
{
sl_send_reply("413", "message too large to be forwarded over UDP without fragmentation");
exit;
}

myself – reference to the list of local IP addresses, hostnames and aliases that has been set in OpenSIPS configuration file. This lists contain the domains served by OpenSIPS.

if(uri==myself) {
log("the request is for local processing\n");
};

null – reset the value of a per-script variable or to delete an avp.

$avp(i:12) = null;
$var(x) = null;

3. Core parameters

abort_on_assert – Set to true in order to make OpenSIPS shut down immediately in case a script assert fails.

abort_on_assert = true // default is false

advertised_address – address advertised in Via header and other destination lumps (e.g RR header).

advertised_port – port advertised in Via header and other destination lumps (e.g. RR).

alias – set alias hostnames for the server

alias=udp:company.tel.com:5060
alias=tcp:company.tel.com:5060

auto_aliases – to control if aliases should be automatically discovered from DNS lookup and added during fixing listening sockets.

auto_aliases=no // default value is no
auto_aliases=0

cfg_file – Returns the name of the corresponding OpenSIPS config file

cfg_line – corresponding line inside the OpenSIPS config file.
check_via – Check if the address in top most via of replies is local.

check_via=1 // Default value is 0 (check disabled)

children – Number of worker processes (children) to be created for each UDP or SCTP interface you have defined. children=16 // Default value is 8

chroot – If set, OpenSIPS will chroot (change root directory) to this valid path in the system value.

debug_mode – This option will automatically force:

  • staying in foreground
  • set logging level to 4 (debug)
  • set logging to standard error
  • enable core dumping
  • set UDP worker processes to 2
  • set TCP worker processes to 2
  • Default value is false/0 (disabled).

db_version_table – name of the table version to be used by the DB API to check the version of the used tables.

db_default_url – default DB URL to be used by modules if no per-module URL is given.

db_default_url=”mysql://opensips:opensipsrw@localhost/opensips”

db_max_async_connections – Maximum number of TCP connections opened from a single OpenSIPS worker to each individual SQL backend. Default value is 10.
Individual backends are determined from DB URLs as follows: [ scheme, user, pass, host, port, database ]

disable_503_translation – If ‘yes’, OpenSIPS will not translate the received 503 replies into 500 replies .
disable_core_dump – By default core dump limits are set to unlimited or a high enough value. Set this config variable to ‘yes’ to disable core dump-ing (will set core limits to 0).

disable_core_dump=yes //Default value is 'no'.

disable_dns_blacklist– DNS resolver, when configured with failover, can automatically store in a temporary blacklist the failed destinations. This will prevent (for a limited period of time) OpenSIPS to send requests to destination known as failed. So, the blacklist can be used as a memory for the DNS resolver.

The temporary blacklist created by DNS resolver is named “dns” and it is by default selected for usage (no need use the use_blacklist()) function. The rules from this list have a life time of 4 minutes – you can change it at compile time, from resolve.c . Can be ‘yes’ or ‘no’. By default the blacklist is disabled (Default value is ‘yes’).

disable_dns_failover – By default DNS-based failover is enabled. Set this config variable to ‘yes’ to disable the DNS-based failover. This is a global option, affecting the core and the modules also.

disable_stateless_fwd – controls the handling of stateless replies:

  • yes – drop stateless replies if stateless fwd functions (like forward) are not used in script
  • no – forward stateless replies

dns – controls if the SIP server should attempt to lookup its own domain name in DNS. Default is no.

dns_retr_time – Time in seconds before retrying a dns request. Default value is system specific, depends also on the ‘/etc/resolv.conf’ content (usually 5s).

dns_retr_no – Number of dns retransmissions before giving up.

dns_servers_no – How many dns servers from the ones defined in ‘/etc/resolv.conf’ will be used. Default value is to use all of them.

dns_try_ipv6 – If it is set to ‘yes’ and a DNS lookup fails, it will retry it for ipv6 (AAAA record). Default value is ‘no’.

dns_try_naptr – Disables the NAPTR lookups when doing DNS based routing for SIP requests – if disabled, the DNS lookup will start with SRV lookups. By default it is enabled, value ‘yes’.

dns_use_search_list

dst_blacklist – static (read-only) IP/destination blacklist. These lists can be selected from script (at runtime) to filter the outgoing requests, based on IP, protocol, port, etc.

filter out requests going to ips of my gws

dst_blacklist = gw:{( tcp , 192.168.2.200 , 5060 , "" ),( any , 192.168.2.201 , 0 , "" )}

block requests going to prohibited networks

dst_blacklist = net_filter:{ ( any , 192.168.1.120/255.255.255.0 , 0 , "" )}

block message requests with nasty words

dst_blacklist = msg_filter:{ ( any , 192.168.20.0/255.255.255.0 , 0 , "MESSAGE*ugly_word" )}

block requests not going to a specific subnet

dst_blacklist = net_filter2:{ !( any , 192.268.30.0/255.255.255.0 , 0 , "" )}

Each rule is defined by:

  • protocol : TCP, UDP, TLS or “any” for anything
  • port : number or 0 for any
  • ip/mask
  • test patter – is a filename like matching (see “man 3 fnmatch”) applied on the outgoing request buffer (first_line+hdrs+body)

enable_asserts – Set to true in order to enable the assert script statement.

event_pkg_threshold – A number representing the percentage threshold above which the E_CORE_PKG_THRESHOLD event is raised, warning about low amount of free private memory. It accepts integer values between 0 and 100. Default value is 0 ( event disabled ).

event_pkg_threshold = 90

event_shm_threshold
A number representing the percentage threshold above which the E_CORE_SHM_THRESHOLD event is raised, warning about low amount of free shared memory. It accepts integer values between 0 and 100.
Default value is 0 ( event disabled ).

event_shm_threshold = 90

exec_dns_threshold – A number representing the maximum number of microseconds a DNS query is expected to last. Anything above the set number will trigger a warning message to the logging facility. Default value is 0 ( logging disabled ).

exec_dns_threshold = 60000

exec_msg_threshold – A number representing the maximum number of microseconds the processing of a SIP msg is expected to last. Anything above the set number will trigger a warning message to the logging facility. Aside from the message and the processing time, the most time consuming function calls from the script will also be logged. Default value is 0 ( logging disabled ).

exec_msg_threshold = 60000

include_file – load additional routes/blocks with file path

include_file "proxy_regs.cfg"

import_file – Same as include_file but will not throw an error if file is not found.

import_file "proxy_regs.cfg"

listen – Set the network addresses the SIP server should listen to. syntax is protocol:address[:port]

The listen definition may accept several optional parameters for:

configuring an advertised IP and port only for an interface. Syntax “AS 11.22.33.44:5060”
setting a different number of children for this interface only (for UDP, SCTP and HEP_UDP interfaces only). This will override the global “children” parameter. Syntax “use_children 5”
Remember that the above parameters only affect the interface they are configured for; if they are not defined for a given interface, the global values will be used instead.

listen = udp:*
listen = udp:eth1
listen = tcp:eth1:5062
listen = tls:localhost:5061
listen = hep_udp:10.10.10.10:5064
listen = ws:127.0.0.1:5060 use_children 5
listen = sctp:127.0.0.1:5060 as 99.88.44.33:5060 use_children 3
On startup, OpenSIPS reports all the interfaces that it is listening on. The TCP engine processes will be created regardless if you specify only UDP interfaces here.
3.41 log_facility – control the facility for logging in syslog. Default value is LOG_DAEMON.

log_facility=LOG_LOCAL0

3.42 log_level – logging level (how verbose OpenSIPS should be). Higher values make OpenSIPS to print more messages.

log_level=1 — print only important messages (like errors or more critical situations) recommended for running proxy as daemon
log_level=4 — print a lot of debug messages use it only when doing debugging sessions

Actual values are:

-3 – Alert level
-2 – Critical level
-1 – Error level
1 – Warning level
2 – Notice level
3 – Info level
4 – Debug level
The ‘log_level’ parameter is usually used in concordance with ‘log_stderror’ parameter.

Value of ‘log_level’ parameter can also be get and set dynamically using log_level Core MI function or $log_level script variable.
3.43 log_name – Set the id to be printed in syslog. The value must be a string and has effect only when OpenSIPS runs in daemon mode (fork=yes), after daemonize. Default value is argv[0].

log_name=”osips-5070″

3.44 log_stderror – write log messages to standard error. Possible values are:
– “yes” – write the messages to standard error
– “no” – write the messages to syslog , also the default

max_while_loops – maximum loops that can be done within a “while”. Comes as a protection to avoid infinite loops in config file execution. Default is 100.

max_while_loops=200

maxbuffer – size in bytes not to be exceeded during the auto-probing procedure of discovering the maximum buffer size for receiving UDP messages. Default value is 262144.

maxbuffer=65536

mem-group – Defines a group of modules (by name) to get separate memory statistics.In order for the feature to work you have to run “make generate-mem-stats” and complile with the variable SHM_EXTRA_STATS defined and complile with the variable SHM_SHOW_DEFAULT_GROUP definedwill generate the statistics for the default group

mem-group = “interest”: “core” “tm”
mem-group = “runtime”: “dialog” “usrloc” “tm”

mem_warming – Only relevant when the HP_MALLOC compile flag is enabled. If set to “on”, on each startup, OpenSIPS will attempt to restore the memory fragmentation pattern it had before the stop/restart.
Memory warming is useful when dealing with high volumes of traffic (thousands of cps on multi-core machines – the more cores, the more useful), because processes must mutually exclude themselves when chopping up the initial big memory chunk. By performing fragmentation on startup, OpenSIPS will also behave optimally in the first minute(s) after a restart. Fragmentation usually lasts a few seconds (e.g. ~5 seconds on an 8GB shm pool and 2.4Ghz CPU) – traffic will not be processed at all during this period.

mem_warming = on

mem_warming_percentage – How much of OpenSIPS’s memory should be fragmented with the pattern of the previous run, upon a restart.

mem_warming_percentage = 50 //Default value: 75

mem_warming_pattern_file – Default value: “CFG_DIR/mem_warming_pattern”.The memory fragmentation pattern of a previous OpenSIPS run. Used at startup, if mem_warming is enabled.

mem_warming_pattern_file = “/var/tmp/my_memory_pattern”

memdump | mem_dump – Log level to print memory status information (runtime and shutdown). Default: memdump=L_DBG (4)

memlog | mem_lo – Log level to print memory debug info. It has to be less than the value of ‘log_level’ parameter if you want memory info to be logged. Default: memlog=L_DBG (4)

mcast_loopback – If set to ‘yes’, multicast datagram are sent over loopback. Default value is ‘no’.

mcast_loopback=yes

mcast_ttl – Set the value for multicast ttl. Default value is OS specific (usually 1).

mhomed – Set the server to try to locate outbound interface on multihomed host. By default is not (0) – it is rather time consuming.

mhomed=1

mpath – Set the module search path. This can be used to simplify the loadmodule parameter

mpath="/usr/local/lib/opensips/modules"<br> loadmodule "mysql.so"<br> loadmodule "uri.so"<br> loadmodule "uri_db.so"<br> loadmodule "sl.so"<br> loadmodule "tm.so"<br> ...

open_files_limit – If set and bigger than the current open file limit, OpenSIPS will try to increase its open file limit to this number. Note: OpenSIPS must be started as root to be able to increase a limit past the hard limit (which, for open files, is 1024 on most systems).

open_files_limit=2048

poll_method – (deprecated post 2.2) poll method to be used by the I/O internal reactor – by default the best one for the current OS is selected. The available types are: poll, epoll_lt, sigio_rt, select, kqueue, /dev/poll.

Starting with version 2.2, epoll_et is deprecated and if it is used in the script, it will be automatically replaced by epoll_lt.

poll_method=select

port – port the SIP server listens to. The default value for it is 5060.

reply_to_via – If it is set to 1, any local reply is sent to the address advertised in top most Via of the request. Default value is 0 (off).

reply_to_via=0

query_buffer_size -If set to a value greater than 1, inserts to DB will not be flushed one by one. Rows to be inserted will be kept in memory until until they gather up to query_buffer_size rows, and only then they will be flushed to the database.

query_buffer_size=5

query_flush_time – If query_buffer_size is set to a value greater than 1, a timer will trigger once every query_flush_time seconds, ensuring that no row will be kept for too long in memory.

query_flush_time=10

rev_dns – should the SIP server attempt to lookup its own IP address in DNS. If this parameter is set to yes and the IP address is not in DNS a warning is printed on syslog and a “received=” field is added to the via header. Default is no.

server_header – The body of Server header field generated by OpenSIPS when it sends a request as UAS. It defaults to “OpenSIPS (<version> (<arch>/<os>))”.

server_header="Server: My Company SIP Proxy"

server_signature – control “Server” header in any locally generated message. If it is enabled (default=yes) a header is generated as Server: OpenSIPS (0.9.5 (i386/linux))

shm_hash_split_percentage – Only relevant when the HP_MALLOC compile flag is enabled. It controls how many memory buckets will be optimized. (e.g. setting it to 2% will optimize the first 81 most used buckets as frequency). The default value is 1.

shm_secondary_hash_size – Only relevant when the HP_MALLOC compile flag is enabled. It represents the optimization factor of a single bucket (e.g. setting it to 4 will cause the optimized buckets to be further split into 4). The default value is 8.

sip_warning – Can be 0 or 1. If set to 1 (default value is 0) a ‘Warning’ header is added to each reply generated by OpenSIPS. The header contains several details that help troubleshooting using the network traffic dumps.

sip_warning=0

tcp_children – Number of children processes to be created for reading from TCP connections. If no value is explicitly set, the same number of TCP children as UDP children (see “children” parameter) will be used.

tcp_accept_aliases – If enabled, OpenSIPS will enforce RFC 5923 behaviour when detecting an “;alias” Via header field parameter and will reuse any TCP (or TLS, WS, WSS) connection opened for such SIP requests (source IP + Via port + proto) when sending other SIP requests backwards, towards the same (source IP + Via port + proto) pair. Default value 0 (disabled).

tcp_listen_backlog – maximum length for the queue of pending connections for the TCP listeners. Default configured value is 10.

tcp_connect_timeout – Time in milliseconds before an ongoing blocking attempt to connect will be aborted. Default value is 100ms.

tcp_connection_lifetime – Lifetime in seconds for TCP sessions. Default value is defined in tcp_conn.h: define DEFAULT_TCP_CONNECTION_LIFETIME 120.

tcp_connection_lifetime = 3600

tcp_max_connections – maximum number of tcp connections. Default is defined in tcp_conn.h: define DEFAULT_TCP_MAX_CONNECTIONS 2048

tcp_max_connections = 4096

tcp_max_msg_time – maximum number of seconds that a SIP message is expected to arrive via TCP. Default value is 4

tcp_max_msg_time = 8

tcp_no_new_conn_bflag -A branch flag to be used as marker to instruct OpenSIPS not to attempt to open a new TCP connection when delivering a request, but only to reuse an existing one (if available). If no existing conn, a generic send error will be returned.

This is intended to be used in NAT scenarios, where makes no sense to open a TCP connection towards a destination behind a NAT (like TCP connection created during registration was lost, so there is no way to contact the device until it re-REGISTER). Also this can be used to detect when a NATed registered user lost his TCP connection, so that opensips can disable his registration as useless.

 tcp_no_new_conn_bflag = TCP_NO_CONNECT<br> ...<br> route {<br> ...<br> if (isflagset(DST_NATED) &amp;&amp; $proto == "TCP")<br>     setbflag(TCP_NO_CONNECT);<br>     ...<br>     t_relay("0x02"); // no auto error reply<br>     $var(retcode) = $rc;<br>     if ($var(retcode) == -6) {<br>         xlog("unable to send request to destination");<br>         send_reply("404", "Not Found");<br>         exit;<br>     } else if ($var(retcode) &lt; 0) {<br>         sl_reply_error();<br>         exit;<br>     }<br> } 

3.77 tcp_threshold – A number representing the maximum number of microseconds sending of a TCP request is expected to last. Anything above the set number will trigger a warning message to the logging facility. Default value is 0 ( logging disabled ).

tcp_threshold = 60000

tcp_keepalive – Enable or disable TCP keepalive (OS level). Enabled by default.

tcp_keepcount -Number of keepalives to send before closing the connection (Linux only). Default value: 0 (not set). Setting tcp_keepcount to any value will enable tcp_keepalive.

tcp_keepidle – Amount of time before OpenSIPS will start to send keepalives if the connection is idle (Linux only).
Default value: 0 (not set)

tcp_keepidle = 30

tcp_keepinterval – Interval between keepalive probes, if the previous one failed (Linux only).Default value: 0 (not set). Setting tcp_keepinterval to any value will enable tcp_keepalive.

tcp_keepinterval = 10

tls_ca_list

tls_certificate

tls_ciphers_list

tls_domain

tls_handshake_timeout

tls_log

tls_method

tls_port_no

tls_private_key

tls_require_certificate

tls_send_timeout

tls_verify

tos – TOS (Type Of Service) to be used for the sent IP packages (both TCP and UDP).

 tos=IPTOS_LOWDELAY<br> tos=0x10<br> tos=IPTOS_RELIABILITY

user_agent_header – The body of User-Agent header field generated by OpenSIPS when it sends a request as UAC. It defaults to “OpenSIPS (<version> (<arch>/<os>))”.

user_agent_header=”User-Agent: My Company SIP Proxy”

wdir – working directory used by OpenSIPS at runtime.

 wdir="/usr/local/opensips" 

xlog_buf_size – Default value: 4096

Size of the buffer used to print a single line on the chosen logging facility of OpenSIPS. If the buffer is too small, an overflow error will be printed, and the concerned line will be skipped.

xlog_buf_size = 8388608 #given in bytes

xlog_force_color

xlog_default_level -Default value for the logging level of the xlog core function, when the log_level parameter is omitted.

xlog_default_level = 2 #L_NOTICE // Default value: -1

Routes

OpenSIPS routing logic uses several types of routes. Each type of route is triggered by a certain event and allows you to process a certain type of message (request or reply).

  • route
    SIP requests routing. The main ‘route’ block identified by ‘route{…}’ or ‘route[0]{…}’ is executed for each SIP request.
    To send a reply or forward the request, explicit actions must be called inside the route block. in example below which sends 200 ok reply for each options request.
route {
if(is_method("OPTIONS")) {
sl_send_reply("200", "ok");
exit();
}
route(1);
}
route[1] {
forward();
}
  • branch_route
    Handles different branches of a SIP request. if the branch is not dropped the branch will be automatically sent out. It is executed only by TM module after it was armed via t_on_branch(“branch_route_index”).
if (is_method("INVITE|BYE|SUBSCRIBE|UPDATE")) {
if(!t_is_set("branch_route")) t_on_branch("MANAGE_BRANCH");
}
branch_route[MANAGE_BRANCH] {
xdbg("new branch [$T_branch_idx] to $ru\n");
route(NATMANAGE);
}

or lookup location and discard branches where uri matches ip 1.2.3.4 by using drop()

route {
lookup("location");
t_on_branch("op3");
if(!t_relay()) {
sl_send_reply("500", "relaying failed");
}
}
branch_route[op3] {
if(uri=~"1\.2\.3\.4") {
drop();
}
}
  • failure_route
    Failed transaction routing block. It contains a set of actions to be taken each transaction that received only negative replies (>=300) for all branches which completes the transaction. The ‘failure_route’ is executed only by TM module after it was armed via t_on_failure(“failure_route_index”).
if (is_method("INVITE")) {
if(!t_is_set("failure_route")) t_on_failure("MANAGE_FAILURE");
}

failure_route[MANAGE_FAILURE] {
route(NATMANAGE);
if (t_is_canceled()) {
exit;
}
}

or on failure relay to voice mail

route {
lookup("location");
t_on_failure("op1");
if(!t_relay()) {
sl_send_reply("500", "relaying failed");
}
}
failure_route[op1] {
if(is_method("INVITE")) {
t_relay("udp:voicemail.server.com:5060");
}
}
  • onreply_route
    Reply routing block. It can be stateful (if bound to a transaction) or stateless (if global reply route).
    If the reply is not dropped (only provisional replies can be), it will be injected and processed by the transaction engine. There are three types of onreply routes:

global – catches all replies and uses simple definition ‘onreply_route {…}’ or ‘onreply_route[0] {…}’.
Exmaple for “global” reply route set the whole transaction

route {
seturi("sip:bob@opensips.org");  first branch
append_branch("sip:alice@opensips.org");  second branch
t_on_reply("global");
t_on_branch("1");
t_relay();
}

onreply_route {
xlog("OpenSIPS received a reply from $si\n");
}

onreply_route[global] {
if (t_check_status("1[0-9][0-9]")) {
setflag(1);
log("provisional reply received\n");
if (t_check_status("183"))
drop;
}
}

per request/transaction – it catches all received replies belonging to a certain transaction and uses “t_on_reply()” at request time, in REQUEST ROUTE – named ‘onreply_route[N] {…}’.

per branch – it catches only the replies that belong to a certain branch from a transaction via “t_on_reply()” ) at request time, but in BRANCH ROUTE, when a certain outgoing branch is processed – named ‘onreply_route[N] {…}’.
Certain ‘onreply_route’ blocks can be executed by TM module for special replies. For this, the ‘onreply_route’ must be armed for the SIP requests whose replies should be processed within it, via t_on_reply(“onreply_route_index”).

Exmaple of reply route set for this branch only


branch_route[1] {
if ($rU=="alice")
t_on_reply("alice");
}

onreply_route[alice] {
xlog("received reply on the branch from alice\n");
}
  • error_route

executed automatically on meeting and error such as parsing error in SIP request processing, script assert failure. Performs error handling . The Default action is to discard request. In error_route, the following pseudo-variables are available to get access to error details:

$(err.class) - the class of error (now is '1' for parsing errors)
$(err.level) - severity level for the error
$(err.info) - text describing the error
$(err.rcode) - recommended reply code
$(err.rreason) - recommended reply reason phrase
error_route {
xlog("--- error route class=$(err.class) level=$(err.level)
info=$(err.info) rcode=$(err.rcode) rreason=$(err.rreason) ---\n");
xlog("--- error from [$si:$sp]\n+++++\n$mb\n++++\n");
sl_send_reply("$err.rcode", "$err.rreason");
exit;
}
  • local_route

executed automatically as TM created a new request, internally (no UAC side). This is a route intended to be used for message inspection, accounting and for applying last changes on the message headers. Routing and signaling functions are not allowed.

local_route {
if (is_method("INVITE") && $ru=~"@foreign.com") {
append_hf("P-hint: foreign request\r\n");
exit;
}
if (is_method("BYE") ) {
acc_log_request("internally generated BYE");
}
}
  • startup_route
    Executed only once when OpenSIPS is started and before the processing of SIP messages begins. Used in initilization cases cases such as loading some data in the cache.
startup_route {
avp_db_query("select gwlist where ruleid==1",$avp(i:100));
cache_store("local", "rule1", "$avp(i:100)");
}
  • timer_route
    Route executed periodically at a configured interval of time specified next to the name(in seconds).
timer_route[gw_update, 300] {
avp_db_query("select gwlist where ruleid==1",$avp(i:100));
$shv(i:100) =$avp(i:100);
}
  • event_route
    execute script code when an event is triggered. If no way to handle the event specified, default will be synchronously.
    Triggered by the event_route module when an event is raised by the OpenSIPS Event Interface such as event raised by the pike module when it decides an ip should be blocked called E_PIKE_BLOCKED or E_SCRIPT_EVENT etc ( checke events interface for more events)
event_route[E_PIKE_BLOCKED] {
xlog("The E_PIKE_BLOCKED event was raised\n");
}
event_route[E_PIKE_BLOCKED, async] {
xlog("The E_PIKE_BLOCKED event was raised\n");
}

opensipsctlrc

opensipsctlrc file controls the opensipsctl and osipsconsole utilities. It is a shell script utility to manage opensips from command line

opensipsctl can manage

  • Start, stop, and restart
  • Show, grant, and revoke ACLs Add, remove, and list aliases
  • Add, remove, and configure an AVP
  • Low Cost Route (LCR)
  • Remote Party Identity (RPID)
  • Add, remove, and list subscribers
  • Add, remove, and show the usrloc table in-Random Access Memory (RAM)
  • Monitor

Ref :

VOIP Call Metric Monitoring and MOS ( Mean Opinion Score)


Metrics for monitoring a VOIP call can be obtained from any node in media path of the call flow . Essentially used for analysis via calculation and aggregation , and sometimes used for realtime performance tracking and rectification too.

VoIP call quality metrics

RTP provides real time media stream, payload type identification, packet sequencing and timestamping headers.

sequence num : tracks incremental succession of incoming packets by sendor and tracls out of order delivery.

timestamp : used by the receiver to play back the received samples at appropriate time and interval. 

source : wikipedia RTP

Note that all Synchronization source (SSRC) identifiers fields denote the synchronization source within the RTP session such as both legs of a call session

  • leg A between Caller and RTP proxy,
  • leg B between RTPproxy and Callee

RTCP provides detailed monitoring of stream to participants in an ongoing session with statistical data and enhanced metrices for QoS ( quality of service ) and synchronisation using it SR ( senders Report ) and RR ( Receivers report) segments .

  • Packet loss rate
  • Packet discard rate
  • round trip delay
  • R factor which is voice quality carried over RTP ssession
  • mos lq for listening quality and mos cq for conversation qualityy
  • jitter buffer current delay , maximum delay
RTCP – SR
RTCP – RR

Certain aspects of RTP media and its RTCP metrics were discussed before you can read more about RTCP and RTCP / AVPF here RealTime Transport protocol (RTP) and RTP control protocol (RTCP )

Other call realted factors which are not specifically part of RTCP but provide information about call quality are

  • signal level
  • noise level
  • gap density, gap threshold
  • Burst density
  • residual echo return loss

Delays like following also play a signaificant influence in VoIP Quality

  • end system delay
  • Paketzation Delay
  • Setup delay ( auth, TLS handshake, accessing mic/camera stream ..)
  • Queing Delay
  • Serialization dleay
  • Network latency
  • End device processing delay such as CPU of the end device

It should be noted that in addition to these values which can be caluvulated algorithimically and with high precsiion , there are more subjective quality parameters which can be only evaluted manually ( ie witha person listening on both ends ) such as

  • Robot voice
  • Perceptible sound but annoying speech quality

Rating Factor (R-Factor) and Mean Opinion Score (MOS)

Rating Factor (R-Factor) and Mean Opinion Score (MOS) are two commonly-used measurements of overall VoIP call quality.

What is R-Factor ?

This is a value derived from metrics such as latency, jitter, and packet loss per ITU‑T Recommendation G.107. It assess the quality-of-experience for VoIP calls on your network. Typical scores range from 50 (bad) to 90 (excellent).

  • R factor of 90 , Mos is 4.3 ( Excellent )
  • R factor 50 , Mos is 2.6 ( Bad)

What is MOS?

MOS is derived from the R-Factor per ITU‑T Recommendation G.10 which measures VoIP call quality. PacketShaper measures MOS using a scale of 10-50. To convert to a standard MOS score (which uses a scale of 1-5), divide the PacketShaper MOS value by 10.

MOS ( Mean Opinion Score )

MOS is terminology for audio, video and audiovisual quality expressions as per ITU-T P.800.1. It refers to listening, talking or conversational quality, whether they originate from subjective or objective models.

  • Very Good: 4.3-5.0
  • Bad: 3.1-3.6
  • Not Recommenced : 2.6-3.1
  • Very Bad: 1.0-2.6

It provides provisions for identifiers regarding the audio bandwidth, the type of interface (electrical or acoustical) and the video resolution too, such as

  • MOS-AVQE for audiovisual quality
  • MOS-CQE is for estimated conversational quality
  • MOS-LQE for listening quality
  • MOS-TQE is used for talking quality
  • MOS-VQE depicts video quality

For Audio Signal Speech Quality/ AV
– N denotes audio signals upto narrow-band (300-3400 Hz)
– W is for audio signals upto wideband (50-7000 Hz)
– S for upto super-wideband (20-14000 Hz)
– F is obtained for fullband (10-20000 Hz)

For Listening quality LQO

  • electrical measurement : done at electrical interfaces only. In order to predict the listening quality as perceived by the user, assumptions for the terminals are made in terms of intermediate reference system (IRS) or corrected IRS frequency response. A sealed condition between the handset receiver and the user’s ear is assumed.
  • acoustical measurement : done at acoustical interfaces. In order to predict the listening quality as perceived by the user, this measurement includes the actual telephone set products provided by the manufacturer or vendor. In combination with the choice of the acoustical receiver in the laboratory test , there will be a more or less leaky condition between the handset’s receiver and the artificial ear.

Conversational Quality / CQ

Arithmetic mean value of subjective judgments on a 5-point ACR quality scale, is calculated.

Talking Quality / TQ

This describes the quality of a telephone call as it is perceived by the talking party only. Factors affecting TQ include echo signal , background noise , double talk etc. It is calculated based on the arithmetic mean value of judgments on a 5-point ACR quality scale.

Video Quality / VQ

To account for differentiation in perceived quality for mobile and fixed devices and to allow for proper handling of different use-cases as
– M for mobile screen such as a smartphone or tablet (approximately 25 cm or less)
– T for PC/TV monitors
It is calculated based on the arithmetic mean value of subjective judgments, typically on a 5-point quality scale

Audio Visual Quality / AVQ

Refers to quality of audio visual stream under corresponding networking conditions. It is also calculated based on the arithmetic mean value of judgments on a 5-point ACR quality scale.

Other parameters also contributing to VoIP metric Analysis

Latency

Latency is primarily is the time required for packets to travel from one end to another, in milliseconds. For example, if the sum of measured latency is 800 ms and the number of latency samples is 20, then the average latency is 40 ms. The header of the RTP packets carry timestamps which later can also be used to calculate round-trip time.

Medium of propagation

  • The Terrestrial coaxial cable or radio-relay system over FDM and digital transmission submarine coaxial cables add up to 4- 6 microseconds of delay per km.
  • Similarly even the optical fibre cable using digital transmission added around 5 microseconds per km delay which also accounts for the delay in repeaters and regenerators
  • On the other hand satellite, communication system varies the delay based on altitude ( propagation delay through space and between earth stations)
    • 400 km above earth surface adds 12 ms delay,
    • 14000 km above earth adds 110 ms
    • much higher 36000 km of altitude adds 260 ms

Devices

  • FDM modem adds upto 0.75 ms delay
  • Transmultiplexer – 1.5 ms delay
  • Exchanges ( analog , digital , transit ..) add 0.45 – 0.825 ms delay
  • Echo cancellers 0.5 ms
  • DCME (Circuit manipulation, signal compression ) – 30 ms to 200 ms

RTT (Round Trip Time )

RTT is the time in milliseconds (ms) taken for data to travel to the target destination and back. In terms of SIP calls it is the time for a transaction to complete between caller/client and callee/server. It is calculated as when the packet was sent and when the acknowledgement for it was received.

High RTT : The media stream especially audio must not suffer a delay higher than 150 ms including all the processing delays at intermediate nodes and network latency. Any value above it is of poor quality. High RTT indicates a poor network quality and would result in the audio lag issue.

RTT vs Network ping calculation: RTT can represent full path network latency experienced by the packets and can do away with frequent ICMP ping/echo requests/probes to check network health. Although it should be noted that while pings happen in lower transport layers protocol, RTT happens at the high up application layer.

RTT is used to calculate RTO ( Request transmission timeouts )in TCP transmission ie how much time the sender should wait before retrying to send an unacknowledged packet.

Factors affecting RTT can include delays in propagation delay, processing delay, queuing delay and encoding delay. Porpogation delay can correlate to the

  • physical distance ( inter country/continents or intra ) ,
  • mediaum of tramsission ( copper cables , fiber , wireless)
  • bandwidth available

Simillarly propagation delay can occur due to large num of network hops like routers / servers . It should be noted that server respose time also plays a critical role in RTT as it depends on server’s processing caapcity and nature of request.

Star based network topology like MCU , SFU or TURN servers can introduce processing delays too for activities such as mixing, encdoing , NATing etc .

Network congestion can amplify the RTT the most.Traffic level must be monitored when RTT spikes such as during DDos attacks

Overcoming large RTT can be achieved by

  • identifying the choke points of network
  • ditributing the load evenly
  • ensuring scalaibility of the server side resources
  • ensuring points of presence(PoP) into geographic regions where caller/ callee is present and routing through it rather than unreliable open public network

Note : avg RTT of the session is misleading denotaion of latency as there maybe be assymetrically RTT between the two legs of the call

Calculation of RTT

EffectiveLatency = ( AverageLatency + Jitter * 2 + 10 )

In RTPengine
int eff_rtt = ssb->rtt / 1000 + ssb->jitter * 2 + 10;

Thus for RTT = 11338 and jitter =0
eff_RTT = 11338/1000 + 0*2 +10
= 11.651 + 10 = 21.651 , which is a good score as it is way below 150ms of latency

But for RTT = 129209 and jitter =7
eff_RTT = 129209/1000 + 7*2 +10
= 153.209 , which is a bad score > 150 ms

Packet Loss

When packet does not successfully make it to the destination , it is a lost packet.

It could happen due to multiple reasons such as

  • network bandwidth unavailable or network congestion
  • overloading of the buffer such that they do not have enough space to queue the packets or high priority preferences
  • intentionally configuring ACL or firewalls to drop the packets or discarding packets above rate limit by internet service provider
  • CPU unable to cope up with high security networks encryption and decryption speed requirements
  • Low battery on device may cause cause underworking of devices and hence lead to packet loss
  • limitation on physical device like softphone , hardphone or bluetooth headsets or if the hardware is broken at router , switch or cabling
  • for bluetooth headsets distance range could also be problem for weak signals and consequently packets drops
  • network errors as shown under Simple Network Management Protocol (SNMP) issues like FCS Errors, Alignment Errors, Frame Too Longs, MAC Receive Errors, Symbol Errors, Collisions, Carrier Sense Errors, Outbound Errors, Outbound Discards, Inbound Discards, Inbound Errors, and Unknown Protocol errors.
  • radio frequency interference from high voltage systems or microwaves can also cause packet drop in wireless networks

such that the packet can either not arrive or arrive late and be dropped out by the codec . To the listener it would appear like chopped voice or complete dropout for moments .

Obtaining packet loss details

  • Packet loss percentage is performed as per RFC 3550 using RTP header sequence numbers. If packets are missing sequence the media stream monitors flags that as lost packet.
  • It can also be concluded from the difference between total packets and received packets from CDR
  • RTP-XR (RFC-3611) records report real-time drops

Jitter

The variation in the delay of received packets in a flow, measured by comparing the interval when RTP packets were sent to the interval at which they were received.
For instance, if packet #1 and packet #2 leave 30 milliseconds apart and arrive 50 milliseconds apart, then the jitter is 20 milliseconds or if packets transmitted every 15ms and reach destination at every 15ms then there is no variability and the jitter is 0.

Causes of jitter

  • Frame bigger than jitter buffer size
  • algorithms to back-of collision by introducing delays in packet transmission in half duplex interfaces
  • even small jitter can get exponentially worse on slow or congestion links
  • jitter can be introduced due to bottlenecks near router buffer, rerouting / parallel routes to the same destination, load-sharing, or route tables changing the path

Handling jitter :

Jitter below 30ms is manageable with the help of jitter buffers in codecs however above that the codec starts to drop the late arrived packets and cannot reassemble / splice up the packets for a smooth media stream effectively, hence causing media quality issues like clipped audio

Detecting jitter:

  • looking at inter packet gap in the direction of RTP stream in wireshark
  • RTP-XR (RFC-3611 & RFC-7005) for real-time jitter buffer usage and drops.
  • software based detection : Network sniffers wireshark , path analyser, Application Performance Monitoring (APM) Tools , CDR analyser , Simple Network Management Protocol (SNMP) Collector
MetricGoodAverageBad
Jitter<= 10ms10ms – 30ms>=30ms
Packet Loss< 0.5%0.5% – 0.9%>= 0.9%
Audio Level>-40dB-80dB to -40dB< -80dB
RTT< 200ms200ms – 300ms> 300ms
Range for good bad attributes for calculating mos score

Ref : ITU P.800.1 : Mean opinion score (MOS) terminology 

Methods for objective and subjective assessment of speech and video quality.

Scheduling for low bandwidth networks

The ability of the end application or the RTP proxy to deal with packet loss or delays depends on its processing techniques , particularly with encoding and buffering techniquee to deal with high pac ket loss rate.

Mapping R-value to calculate MOS

To map MOS from R value using above defined metrics , a standard formula is used. First the latency and jitter are added and defined value for computation time is also added , resulting in effective latency

effectiveLatency = latency + jitter * latencyImpact + compTime

Subtracting effective latency from defined R

R = 93 – (effectiveLatency / factorLatencyBased)

Calculate percentage of packet loss

 R = R – (lostPackets * impact)
 MOS = ( (R - 60) * (100 – R) * 0.000007R) + 0.035R + 1)

Media Stats and MOS on RTP engine Kamailio

Minimum edge Values

mos_min_pv
minimum encountered MOS value for the call.
range – 1.0 to 5.0.

mos_min_at_pv
timestamp of when the minimum MOS value was encountered during the call

mos_min_packetloss_pv
amount of packetloss in percent at the time the minimum MOS value was encountered

mos_min_roundtrip_pv
packet round-trip time in milliseconds at the time the minimum MOS value was encountered

mos_min_jitter_pv
amount of jitter in milliseconds at the time the minimum MOS value was encountered

Maximum edge Values

mos_max_pv
maximum encountered MOS value for the call.

mos_max_at_pv
timestamp of when the maximum MOS value was encountered during the cal

mos_max_packetloss_pv
amount of packetloss in percent at maximum MOS moment

mos_max_roundtrip_pv
packet round-trip time in milliseconds at maximum MOS moment

mos_max_jitter_pv
amount of jitter in milliseconds at maximum moment

Average Values

mos_average_pv : average (median) MOS value for the call. Range – 1.0 through 5.0.

mos_average_packetloss_pv : average (median) amount of packetloss in percent present throughout the call.

mos_average_jitter_pv : average (median) amount of jitter in milliseconds present throughout the call.

mos_average_roundtrip_pv

mos_average_samples_pv : number of samples used to determine the other “average” MOS data points.

Labels

mos_A_label_pv : custom label used in rtpengine signalling.
If set, all the statistics pseudovariables with the A suffix will be filled in with statistics only from the call legs that match the label given in this variable.

A label’s min
mos_min_A_pv
mos_min_at_A_pv
mos_min_packetloss_A_pv
mos_min_jitter_A_pv
mos_min_roundtrip_A_pv

A label’s max
mos_max_A_pv
mos_max_at_A_pv
mos_max_packetloss_A_pv
mos_max_jitter_A_pv
mos_max_roundtrip_A_pv

A label’s average
mos_average_A_pv
mos_average_packetloss_A_pv
mos_average_jitter_A_pv
mos_average_roundtrip_A_pv
mos_average_samples_A_pv

B labels’s min
mos_B_label_pv
mos_min_B_pv
mos_min_at_B_pv
mos_min_packetloss_B_pv
mos_min_jitter_B_pv
mos_min_roundtrip_B_pv

B label’s max
mos_max_B_pv
mos_max_at_B_pv
mos_max_packetloss_B_pv
mos_max_jitter_B_pv
mos_max_roundtrip_B_pv

B label’s average
mos_average_B_pv
mos_average_packetloss_B_pv
mos_average_jitter_B_pv
mos_average_roundtrip_B_pv
mos_average_samples_B_pv

Setting MOS collection on kamailio

set the kamailio config rtpengine params for names the variable the hold specific mos values

modparam("rtpengine", "mos_max_pv", "$avp(mos_max)")
modparam("rtpengine", "mos_average_pv", "$avp(mos_average)")
modparam("rtpengine", "mos_min_pv", "$avp(mos_min)")

modparam("rtpengine", "mos_average_packetloss_pv", "$avp(mos_average_packetloss)")
modparam("rtpengine", "mos_average_jitter_pv", "$avp(mos_average_jitter)")
modparam("rtpengine", "mos_average_roundtrip_pv", "$avp(mos_average_roundtrip)")
modparam("rtpengine", "mos_average_samples_pv", "$avp(mos_average_samples)")

modparam("rtpengine", "mos_min_pv", "$avp(mos_min)")
modparam("rtpengine", "mos_min_at_pv", "$avp(mos_min_at)")
modparam("rtpengine", "mos_min_packetloss_pv", "$avp(mos_min_packetloss)")
modparam("rtpengine", "mos_min_jitter_pv", "$avp(mos_min_jitter)")
modparam("rtpengine", "mos_min_roundtrip_pv", "$avp(mos_min_roundtrip)")

modparam("rtpengine", "mos_max_pv", "$avp(mos_max)")
modparam("rtpengine", "mos_max_at_pv", "$avp(mos_max_at)")
modparam("rtpengine", "mos_max_packetloss_pv", "$avp(mos_max_packetloss)")
modparam("rtpengine", "mos_max_jitter_pv", "$avp(mos_max_jitter)")
modparam("rtpengine", "mos_max_roundtrip_pv", "$avp(mos_max_roundtrip)")

modparam("rtpengine", "mos_A_label_pv", "$avp(mos_A_label)")
modparam("rtpengine", "mos_average_packetloss_A_pv", "$avp(mos_average_packetloss_A)")
modparam("rtpengine", "mos_average_jitter_A_pv", "$avp(mos_average_jitter_A)")
modparam("rtpengine", "mos_average_roundtrip_A_pv", "$avp(mos_average_roundtrip_A)")
modparam("rtpengine", "mos_average_A_pv", "$avp(mos_average_A)")

modparam("rtpengine", "mos_B_label_pv", "$avp(mos_B_label)")
modparam("rtpengine", "mos_average_packetloss_B_pv", "$avp(mos_average_packetloss_B)")
modparam("rtpengine", "mos_average_jitter_B_pv", "$avp(mos_average_jitter_B)")
modparam("rtpengine", "mos_average_roundtrip_B_pv", "$avp(mos_average_roundtrip_B)")
modparam("rtpengine", "mos_average_B_pv", "$avp(mos_average_B)")

For individual leg labbeling fill up the lables

KSR.pv.sets("$avp(mos_A_label)","Aleg_label")
KSR.pv.sets("$avp(mos_B_label)","Bleg_label")

Gather the mos stats from the code . Given exmaple is in Lua.
The values are filled in after invoking“rtpengine_delete”, “rtpengine_query”, or “rtpengine_manage” if the command resulted in a deletion of the call (or call branch).

KSR.log("info", " mos avg " .. KSR.pv.get("$avp(mos_average)"))
KSR.log("info", " mos max " .. KSR.pv.get("$avp(mos_max)"))
KSR.log("info", " mos min " .. KSR.pv.get("$avp(mos_min)"))

KSR.log("info", "mos_average_packetloss_pv" .. KSR.pv.get("$avp(mos_average_packetloss)"))
KSR.log("info", "mos_average_jitter_pv" .. KSR.pv.get("$avp(mos_average_jitter)"))
KSR.log("info", "mos_average_roundtrip_pv" .. KSR.pv.get("$avp(mos_average_roundtrip)"))
KSR.log("info", "mos_average_samples_pv" .. KSR.pv.get("$avp(mos_average_samples)"))

KSR.log("info", "mos_min_pv" .. KSR.pv.get("$avp(mos_min)"))
KSR.log("info", "mos_min_at_pv" .. KSR.pv.get("$avp(mos_min_at)"))
KSR.log("info", "mos_min_packetloss_pv" .. KSR.pv.get("$avp(mos_min_packetloss)"))
KSR.log("info", "mos_min_jitter_pv" .. KSR.pv.get("$avp(mos_min_jitter)"))
KSR.log("info", "mos_min_roundtrip_pv" .. KSR.pv.get("$avp(mos_min_roundtrip)"))

KSR.log("info", "mos_max_pv" .. KSR.pv.get("$avp(mos_max)"))
KSR.log("info", "mos_max_at_pv" .. KSR.pv.get("$avp(mos_max_at)"))
KSR.log("info", "mos_max_packetloss_pv" .. KSR.pv.get("$avp(mos_max_packetloss)"))
KSR.log("info", "mos_max_jitter_pv" .. KSR.pv.get("$avp(mos_max_jitter)"))
KSR.log("info", "mos_max_roundtrip_pv" .. KSR.pv.get("$avp(mos_max_roundtrip)"))

local mos_A_label = KSR.pv.get("$avp(mos_A_label)")
if not (mos_A_label == nil) then
    KSR.log("info", "mos_average_packetloss_A_pv" .. KSR.pv.get("$avp(mos_average_packetloss_A)"))
    KSR.log("info", "mos_average_jitter_A_pv" .. KSR.pv.get("$avp(mos_average_jitter_A)"))
    KSR.log("info", "mos_average_roundtrip_A_pv" .. KSR.pv.get("$avp(mos_average_roundtrip_A)"))
    KSR.log("info", "mos_average_A_pv" .. KSR.pv.get("$avp(mos_average_A)"))
end

local mos_B_label = KSR.pv.get("$avp(mos_B_label)")
if not (mos_B_label == nil) then
    KSR.log("info", "mos_average_packetloss_B_pv" .. KSR.pv.get("$avp(mos_average_packetloss_B)"))
    KSR.log("info", "mos_average_jitter_B_pv" .. KSR.pv.get("$avp(mos_average_jitter_B)"))
    KSR.log("info", "mos_average_roundtrip_B_pv" .. KSR.pv.get("$avp(mos_average_roundtrip_B)"))
    KSR.log("info", "mos_average_B_pv" .. KSR.pv.get("$avp(mos_average_B)"))
end

Sample obtained result for one leg

      "average MOS": {
        "MOS": 43,
        "round-trip time": 13430,
        "jitter": 0,
        "packet loss": 0,
        "samples": 4
      },
      "lowest MOS": {
        "MOS": 43,
        "round-trip time": 24184,
        "jitter": 0,
        "packet loss": 0,
        "reported at": 1590498085
      },
      "highest MOS": {
        "MOS": 44,
        "round-trip time": 8218,
        "jitter": 0,
        "packet loss": 0,
        "reported at": 1590498089
      },

CDR with MOS on Freeswitch

<?xmlversion="1.0"?>
					
<cdr core-uuid="[UUID]" switchname="freeswitch">
<channel_data>
	<state>
	<direction>
	<state_number>
	<flags>	
	<caps>
</channel_data>
					
<call-stats>			
//Audio Compoennts 		
// Video Component
</call-stats>

// Variables 			

<app_log>			
	<application app_name="..."app_data="...">
	<application app_name="..."app_data="...">
</app_log>
				
// Callflow 
				
</cdr>		

Audio

<audio>	
	<inbound>
		<raw_bytes>	
		<media_bytes>
		<packet_count>
		<media_packet_count>		
		<skip_packet_count>
		<jitter_packet_count>
		<dtmf_packet_count>	
		<cng_packet_count>		
		<flush_packet_count>
		<largest_jb_size>
		<jitter_min_variance>
		<jitter_max_variance>
		<jitter_loss_rate>
		<jitter_burst_rate>
		<mean_interval>
		<flaw_total>
		<quality_percentage>
		<mos>
	</inbound>				
	<outbound>
		<raw_bytes>
		<media_bytes>
		<packet_count>
		<media_packet_count>
		<skip_packet_count>
		<dtmf_packet_count>
		<cng_packet_count>
		<rtcp_packet_count>
		<rtcp_octet_count>
	</outbound>	
</audio>

Video

<video>	
	<inbound>
		<raw_bytes>
		<media_bytes>
		<packet_count>
		<media_packet_count>
		<skip_packet_count>
		<jitter_packet_count>
		<dtmf_packet_count>
		<cng_packet_count>
		<flush_packet_count>
		<largest_jb_size>
		<jitter_min_variance>
		<jitter_max_variance>
		<jitter_loss_rate>
		<jitter_burst_rate>
		<mean_interval>
		<flaw_total>
		<quality_percentage>
		<mos>
	</inbound>	
	<outbound>
		<raw_bytes>
		<media_bytes>
		<packet_count>
		<media_packet_count>
		<skip_packet_count>
		<dtmf_packet_count>
		<cng_packet_count>
		<rtcp_packet_count>
		<rtcp_octet_count>	
	</outbound>
</video>

The variables

<variables>		
<is_outbound>
<uuid><session_id><text_media_flow>
<direction>
<ep_codec_string>
<channel_name>
<secondary_recovery_module>
<verto_dvar_email><verto_dvar_avatar><jsock_uuid_str>
<verto_user><presence_id>
<verto_client_address><chat_proto>
<verto_host><event_channel_cookie>
<verto_profile_name>
<record_stereo><default_areacode><transfer_fallback_extension>
<toll_allow><accountcode><user_context><effective_caller_id_name><effective_caller_id_number>
<outbound_caller_id_name><outbound_caller_id_number><callgroup><user_name><domain_name>
<Event-Name>
<Core-UUID>
<FreeSWITCH-Hostname><FreeSWITCH-Switchname><FreeSWITCH-IPv4><FreeSWITCH-IPv6><Event-Date-Local><Event-Date-GMT><Event-Date-Timestamp>
<Event-Calling-File>
<Event-Calling-Function>
<Event-Calling-Line-Number>
<Event-Sequence>
<verto_remote_caller_id_name><verto_remote_caller_id_number>
<switch_r_sdp>

<call_uuid><open>
<rtp_secure_media>
<export_vars><conference_enter_sound>
<conference_exit_sound><video_banner_text>
<rtp_use_codec_string><remote_audio_media_flow>
<audio_media_flow>
<rtp_audio_recv_pt>
<rtp_use_codec_name> 
<rtp_use_codec_fmtp>
<rtp_use_codec_rate>
<rtp_use_codec_ptime>
<rtp_use_codec_channels>
<rtp_last_audio_codec_string>
<original_read_codec>
<original_read_rate>
<write_codec><write_rate>
<remote_audio_ip>
<remote_audio_port>
<remote_audio_rtcp_ip>
<remote_audio_rtcp_port>
<dtmf_type>
<remote_video_media_flow>
<video_media_flow>
<video_possible>
<rtp_video_pt>
<rtp_video_recv_pt>
<video_read_codec>
<video_read_rate><video_write_codec><video_write_rate><rtp_last_video_codec_string>
<rtp_use_video_codec_name>
<rtp_use_video_codec_rate>
<rtp_use_video_codec_ptime>
<remote_video_ip><remote_video_port>
<remote_video_rtcp_ip><remote_video_rtcp_port>
<local_media_ip><local_media_port>
<advertised_media_ip>
<rtp_use_timer_name><rtp_use_pt>
<rtp_use_ssrc><rtp_2833_send_payload>
<rtp_2833_recv_payload><remote_media_ip>
<remote_media_port><local_video_ip>
<local_video_port><rtp_use_video_pt><rtp_use_video_ssrc><rtp_local_sdp_str><current_application_data><current_application><send_silence_when_idle><rtp_has_crypto><endpoint_disposition><conference_name><conference_member_id><conference_moderator><conference_ghost><conference_uuid><video_width><video_height><video_fps><verto_hangup_disposition><read_codec><read_rate><hangup_cause><hangup_cause_q850>
<digits_dialed>
<start_stamp><profile_start_stamp><answer_stamp><progress_media_stamp><end_stamp>
<start_epoch><start_uepoch>
<profile_start_epoch><profile_start_uepoch>
<answer_epoch><answer_uepoch>
<bridge_epoch><bridge_uepoch>
<last_hold_epoch><last_hold_uepoch>
<hold_accum_seconds><hold_accum_usec><hold_accum_ms><resurrect_epoch><resurrect_uepoch>
<progress_epoch><progress_uepoch><progress_media_epoch><progress_media_uepoch>
<end_epoch><end_uepoch>
<last_app><last_arg><caller_id><duration><billsec><progresssec><answersec><waitsec><progress_mediasec>

<flow_billsec>
   <mduration><billmsec><progressmsec><answermsec><waitmsec><progress_mediamsec><flow_billmsec><uduration>  <billusec><progressusec><answerusec><waitusec><progress_mediausec>
<flow_billusec>

<rtp_audio_in_raw_bytes>
<rtp_audio_in_media_bytes>
<rtp_audio_in_packet_count>
<rtp_audio_in_media_packet_count>
<rtp_audio_in_skip_packet_count><rtp_audio_in_jitter_packet_count><rtp_audio_in_dtmf_packet_count>
<rtp_audio_in_cng_packet_count>
<rtp_audio_in_flush_packet_count>
<rtp_audio_in_largest_jb_size>
<rtp_audio_in_jitter_min_variance><rtp_audio_in_jitter_max_variance>
<rtp_audio_in_jitter_loss_rate>
<rtp_audio_in_jitter_burst_rate>
<rtp_audio_in_mean_interval>
<rtp_audio_in_flaw_total>
<rtp_audio_in_quality_percentage>
<rtp_audio_in_mos>
<rtp_audio_out_raw_bytes>
<rtp_audio_out_media_bytes>
<rtp_audio_out_packet_count>
<rtp_audio_out_media_packet_count><rtp_audio_out_skip_packet_count><rtp_audio_out_dtmf_packet_count>
<rtp_audio_out_cng_packet_count>
<rtp_audio_rtcp_packet_count>
<rtp_audio_rtcp_octet_count>
<rtp_video_in_raw_bytes>
<rtp_video_in_media_bytes>
<rtp_video_in_packet_count>
<rtp_video_in_media_packet_count>
<rtp_video_in_skip_packet_count><rtp_video_in_jitter_packet_count><rtp_video_in_dtmf_packet_count>
<rtp_video_in_cng_packet_count>
<rtp_video_in_flush_packet_count>
<rtp_video_in_largest_jb_size>
<rtp_video_in_jitter_min_variance><rtp_video_in_jitter_max_variance>
<rtp_video_in_jitter_loss_rate>
<rtp_video_in_jitter_burst_rate>
<rtp_video_in_mean_interval
><rtp_video_in_flaw_total>
<rtp_video_in_quality_percentage>
<rtp_video_in_mos>
<rtp_video_out_raw_bytes>
<rtp_video_out_media_bytes>
<rtp_video_out_packet_count>
<rtp_video_out_media_packet_count><rtp_video_out_skip_packet_count><rtp_video_out_dtmf_packet_count>
<rtp_video_out_cng_packet_count>
<rtp_video_rtcp_packet_count>
<rtp_video_rtcp_octet_count>

</variables>

The Callflow components

<callflow dialplan="XML" unique-id="[UUID]" profile_index="1">
	
	<extension name="myconference" number="3500">		
		<application app_name="..." app_data="...">
	</extension>	
	<caller_profile>
		<username>
		<dialplan>
		<caller_id_name>
		<caller_id_number>
		<callee_id_name>
		<callee_id_number>
		<ani>
		<aniii>
		<network_addr>
		<rdnis>
		<destination_number>
		<uuid>
		<source>
		<context>
		<chan_name>
	</caller_profile>
				
			
	<times>
		<created_time>
		<profile_created_time>
		<progress_time>	
		<progress_media_time>
		<answered_time>
		<bridged_time>
		<last_hold_time>	
		<hold_accum_time>
		<hangup_time>
		<resurrect_time>	
		<transfer_time>	
	</times>
</callflow>

Standardising bodies :-

  • ITU (The International Telecommunication Union) is the United Nations specialised agency in the field of telecommunications, information and communication technologies (ICTs).
  • ITU-T ( ITU Telecommunication Standardisation Sector) is responsible for studying technical, operating and tariff questions and issuing Recommendations on them with a view to standardising tele-communications on a worldwide basis.

As the technology for packet switching matured, the voice quality between circuit-switched and packet-switched networks is mostly indistinguishable. However, the flaws in the VoIP communication system reappear under low network conditions and bad architecture design. Especially with applications that are greedy for network bandwidth such as large scale conferencing or HD streaming, the need for monitoring and quality control is very high, which can be only met by above described QoS parameters.

References

  • CDR on freeswitch
  • ITU-T G.114 TELECOMMUNICATION STANDARDIZATION SECTOR OF ITU (05/2003) SERIES G: TRANSMISSION SYSTEMS AND MEDIA, DIGITAL SYSTEMS AND NETWORKS , International telephone connections and circuits – General Recommendations on the transmission qua
  • Kamailio RTP engine https://www.kamailio.org/docs/modules/devel/modules/rtpengine.html

RTPengine on kamailio SIP server


RTPengine is a proxy for RTP traffic and other UDP based media traffic over either IPv4 or IPv6. It can even bridge between diff IP networks and interfaces. It can do TOS/QoS field setting. It is Multi-threaded, can advertise different addresses for operation behind NAT.

This article focuses on setting up sipwise rtpegine to proxy RTP traffic from the Kamailio app server. This is an updated version of the old article on RTPEngine, since then there have many many updates on the software. I also wrote an article covering all relevant and important Kamailio modules earlier including RTPProxy and RTP engine https://telecom.altanai.com/2014/11/18/kamailio-modules/

It bears in-kernel packet forwarding for low-latency and low-CPU performance. When used with the Kamailio, the RTP engine module adds more features to media stream routing and management, especially around RTP proxy and Mos scores.

Features

  • Full SDP parsing and rewriting
  • Supports non-standard RTCP ports (RFC 3605)
  • ICE (RFC 5245):
    • Bridging between ICE-enabled and ICE-unaware user agents
    • Optionally acting only as additional ICE relay/candidate
    • Optionally forcing relay of media streams by removing other ICE candidates
  • SRTP (RFC 3711):
    • Support for SDES (RFC 4568) and DTLS-SRTP (RFC 5764)
    • AES-CM and AES-F8 ciphers, both in userspace and in kernel
    • HMAC-SHA1 packet authentication
    • Bridging between RTP and SRTP user agents
  • RTCP profile with feedback extensions (RTP/AVPF, RFC 4585 and 5124)
  • Arbitrary bridging between any of the supported RTP profiles (RTP/AVP, RTP/AVPF, RTP/SAVP, RTP/SAVPF)
  • RTP/RTCP multiplexing (RFC 5761) and demultiplexing
  • Breaking of BUNDLE’d media streams (draft-ietf-mmusic-sdp-bundle-negotiation)
  • Recording of media streams, decrypted if possible
  • Transcoding and repacketization
  • Playback of pre-recorded streams/announcements

Sipwise NGCP RTP Engine Source Code

There are 3 parts of the source structure in sipwise NGCP ( Next Generation communication Platform) rtpengine :

1.daemon

The userspace daemon and workhorse, minimum requirement for anything to work. Running make will compile the binary, which will be called rtpengine.

Required packages including their development headers are required to compile the daemon:

  • pkg-config
  • GLib including GThread and GLib-JSON version 2.x
  • zlib
  • OpenSSL
  • PCRE library
  • XMLRPC-C version 1.16.08 or higher
  • hiredis library
  • gperf
  • libcurl version 3.x or 4.x
  • libevent version 2.x
  • libpcap
  • libsystemd
  • MySQL or MariaDB client library (optional for media playback and call recording daemon)
  • libiptc library for iptables management (optional)
  • ffmpeg codec libraries for transcoding (optional) such as libavcodec, libavfilter, libswresample
  • bcg729 for full G.729 transcoding support (optional)

options for make – with_iptables_option , with_transcoding

 with_transcoding=no make 

2.iptables-extension

Required for in-kernel packet forwarding. With the iptables development headers installed, issuing make will compile the plugin for iptables and ip6tables. The file will be called libxt_RTPENGINE.so and needs to be copied into the xtables module directory. The location of this directory can be determined through pkg-config xtables –variable=xtlibdir on newer systems, and/or is usually either /lib/xtables/ or /usr/lib/x86_64-linux-gnu/xtables/.

3.kernel-module

Required for in-kernel packet forwarding. Compilation of the kernel module requires the kernel development headers to be installed in/lib/modules/$VERSION/build/, where $VERSION is the output of the command uname -r.

Successful compilation of the module will produce the file xt_RTPENGINE.ko. The module can be inserted into the running kernel manually through insmod xt_RTPENGINE.ko

It is recommended to copy the module into /lib/modules/$VERSION/updates/, followed by running depmod -a.

After this, the module can be loaded by issuing modprobe xt_RTPENGINE.

Installation

Follow instructions on https://gist.github.com/altanai/0d8cadbe6876d545fd63d6b3e79dcf73

Requirements

sudo su apt-get install debhelper iptables-dev libcurl4-openssl-dev libglib2.0-dev  libjson-glib-dev libxmlrpc-core-c3-dev libhiredis-dev build-essential:native

For Pcap

apt install ibpcap-dev

Also instal ffmpeg pakages

apt install libavcodec-dev libavfilter-dev libavformat-dev libavresample-dev  libavutil-dev

Use dpkg

libcrypt-openssl-rsa-perl libdigest-crc-perl libio-multiplex-perl libnet-interface-perl libsystemd-dev markdown

For debhelper>10

vi /etc/apt/sources.list

add line

deb http://archive.ubuntu.com/ubuntu xenial-backports main restricted universe multiverse
sudo apt update

check version

apt-cache policy debhelper dh-autoreconf
debhelper:
Installed: 9.20160115ubuntu3
Candidate: 9.20160115ubuntu3
Version table:
10.2.2ubuntu1~ubuntu16.04.1 100
100 http://us-east-1.ec2.archive.ubuntu.com/ubuntu xenial-backports/main amd64 Packages
100 http://archive.ubuntu.com/ubuntu xenial-backports/main amd64 Packages
*** 9.20160115ubuntu3 500
500 http://us-east-1.ec2.archive.ubuntu.com/ubuntu xenial/main amd64 Packages
100 /var/lib/dpkg/status
dh-autoreconf:
Installed: (none)
Candidate: 11
Version table:
12~ubuntu16.04.1 100
100 http://us-east-1.ec2.archive.ubuntu.com/ubuntu xenial-backports/main amd64 Packages
100 http://archive.ubuntu.com/ubuntu xenial-backports/main amd64 Packages
11 500
500 http://us-east-1.ec2.archive.ubuntu.com/ubuntu xenial/main amd64 Packages

Force installing the version from backports repo as it have low priority.

sudo apt install dh-autoreconf=12~ubuntu16.04.1 debhelper=10.2.2ubuntu1~ubuntu16.04.1

so now new priority will be

debhelper:

  Installed: 10.2.2ubuntu1~ubuntu16.04.1
  Candidate: 10.2.2ubuntu1~ubuntu16.04.1
  Version table:
 *** 10.2.2ubuntu1~ubuntu16.04.1 100
        100 http://us-east-1.ec2.archive.ubuntu.com/ubuntu xenial-backports/main amd64 Packages
        100 http://archive.ubuntu.com/ubuntu xenial-backports/main amd64 Packages
        100 /var/lib/dpkg/status
     9.20160115ubuntu3 500
        500 http://us-east-1.ec2.archive.ubuntu.com/ubuntu xenial/main amd64 Packages
dh-autoreconf:
  Installed: 12~ubuntu16.04.1
  Candidate: 12~ubuntu16.04.1
  Version table:
 *** 12~ubuntu16.04.1 100
        100 http://us-east-1.ec2.archive.ubuntu.com/ubuntu xenial-backports/main amd64 Packages
        100 http://archive.ubuntu.com/ubuntu xenial-backports/main amd64 Packages
        100 /var/lib/dpkg/status
     11 500
        500 http://us-east-1.ec2.archive.ubuntu.com/ubuntu xenial/main amd64 Packages

ref :https://askubuntu.com/questions/863221/need-help-building-debhelper-10-2-2-bpo8-from-source

Get sourcecode

cd /usr/local/src
git clone https://github.com/sipwise/rtpengine.git
cd rtpengine
 ./debian/flavors/no_ngcp

use dpkg-checkbuilddeps to find any missing dependices

For missing dependencies

dpkg-checkbuilddeps: error: Unmet build dependencies: libbcg729-dev
remove the encoder for G.729 which is not supported by ffmoeg by exporting varible

export DEB_BUILD_PROFILES="pkg.ngcp-rtpengine.nobcg729"

Ref : https://github.com/sipwise/rtpengine#g729-support

for defaultlibmysqlclient-dev and libiptc-dev

vi debian/control
change from default-libmysqlclient-dev to libmysqlclient-dev, change from libiptcdata-dev to libiptc-dev and install the alternatives such as

apt install libmysqlclient-dev libiptcdata-dev 

Generated deb files should be outside the rtpegine home folder

generated ngcp-rtpegine deb files
cd ..
dpkg -i ngcp-rtpengine-daemon_7.3.0.0+0~mr7.3.0.0_amd64.deb
dpkg -i ngcp-rtpengine-iptables_7.3.0.0+0~mr7.3.0.0_amd64.deb
dpkg -i ngcp-rtpengine-kernel-dkms_7.3.0.0+0~mr7.3.0.0_all.deb
dpkg -i ngcp-rtpengine-kernel-source_7.3.0.0+0~mr7.3.0.0_all.deb
dpkg -i ngcp-rtpengine-recording-daemon_7.3.0.0+0~mr7.3.0.0_amd64.deb
dpkg -i ngcp-rtpengine-utils_7.3.0.0+0~mr7.3.0.0_all.deb
dpkg -i ngcp-rtpengine_7.3.0.0+0~mr7.3.0.0_all.deb
After depackaging

Manual installation and running all test cases

cd rtpengine
make check

If you dont find a package you are looking for , some alternatives are to do apt-cache search like

apt-cache search libavfilter
libavfilter-dev - FFmpeg library containing media filters - development files
libavfilter-ffmpeg5 - FFmpeg library containing media filters - runtime files

or to search in ubuntu packages web https://packages.ubuntu.com/

Running RTPEngine

rtpegine application options

  • -v, –version Print build time and exit
  • –config-file=FILE Load config from this file
  • –config-section=STRING Config file section to use
  • –log-facility=daemon|local0|…|local7 Syslog facility to use for logging
  • -L, –log-level=INT Mask log priorities above this level
  • -E, –log-stderr Log on stderr instead of syslog
  • –no-log-timestamps Drop timestamps from log lines to stderr
  • –log-mark-prefix Prefix for sensitive log info
  • –log-mark-suffix Suffix for sensitive log info
  • -p, –pidfile=FILE Write PID to file
  • -f, –foreground Don’t fork to background
  • -t, –table=INT Kernel table to use
  • -F, –no-fallback Only start when kernel module is available
  • -i, –interface=[NAME/]IP[!IP] Local interface for RTP
  • -k, –subscribe-keyspace=INT INT … Subscription keyspace list
  • -l, –listen-tcp=[IP:]PORT TCP port to listen on
  • -u, –listen-udp=[IP46|HOSTNAME:]PORT UDP port to listen on
  • -n, –listen-ng=[IP46|HOSTNAME:]PORT UDP port to listen on, NG protocol
  • -c, –listen-cli=[IP46|HOSTNAME:]PORT UDP port to listen on, CLI
  • -g, –graphite=IP46|HOSTNAME:PORT Address of the graphite server
  • -G, –graphite-interval=INT Graphite send interval in seconds
  • –graphite-prefix=STRING Prefix for graphite line
  • -T, –tos=INT Default TOS value to set on streams
  • –control-tos=INT Default TOS value to set on control-ng
  • -o, –timeout=SECS RTP timeout
  • -s, –silent-timeout=SECS RTP timeout for muted
  • -a, –final-timeout=SECS Call timeout
  • –offer-timeout=SECS Timeout for incomplete one-sided calls
  • -m, –port-min=INT Lowest port to use for RTP
  • -M, –port-max=INT Highest port to use for RTP
  • -r, –redis=[PW@]IP:PORT/INT Connect to Redis database
  • -w, –redis-write=[PW@]IP:PORT/INT Connect to Redis write database
  • –redis-num-threads=INT Number of Redis restore threads
  • –redis-expires=INT Expire time in seconds for redis keys
  • -q, –no-redis-required Start no matter of redis connection state
  • –redis-allowed-errors=INT Number of allowed errors before redis is temporarily disabled
  • –redis-disable-time=INT Number of seconds redis communication is disabled because of errors
  • –redis-cmd-timeout=INT Sets a timeout in milliseconds for redis commands
  • –redis-connect-timeout=INT Sets a timeout in milliseconds for redis connections
  • -b, –b2b-url=STRING XMLRPC URL of B2B UA
  • –log-facility-cdr=daemon|local0|…|local7 Syslog facility to use for logging CDRs
  • –log-facility-rtcp=daemon|local0|…|local7 Syslog facility to use for logging RTCP
  • –log-facility-dtmf=daemon|local0|…|local7 Syslog facility to use for logging DTMF
  • –log-format=default|parsable Log prefix format
  • -x, –xmlrpc-format=INT XMLRPC timeout request format to use. 0: SEMS DI, 1: call-id only, 2: Kamailio
  • –num-threads=INT Number of worker threads to create
  • –media-num-threads=INT Number of worker threads for media playback
  • -d, –delete-delay=INT Delay for deleting a session from memory.
  • –sip-source Use SIP source address by default
  • –dtls-passive Always prefer DTLS passive role
  • –max-sessions=INT Limit of maximum number of sessions
  • –max-load=FLOAT Reject new sessions if load averages exceeds this value
  • –max-cpu=FLOAT Reject new sessions if CPU usage (in percent) exceeds this value
  • –max-bandwidth=INT Reject new sessions if bandwidth usage (in bytes per second) exceeds this value
  • –homer=IP46|HOSTNAME:PORT Address of Homer server for RTCP stats
  • –homer-protocol=udp|tcp Transport protocol for Homer (default udp)
  • –homer-id=INT ‘Capture ID’ to use within the HEP protocol
  • –recording-dir=FILE Directory for storing pcap and metadata files
  • –recording-method=pcap|proc Strategy for call recording
  • –recording-format=raw|eth File format for stored pcap files
  • –iptables-chain=STRING Add explicit firewall rules to this iptables chain
  • –codecs Print a list of supported codecs and exit
  • –scheduling=default|none|fifo|rr|other|batch|idle Thread scheduling policy
  • –priority=INT Thread scheduling priority
  • –idle-scheduling=default|none|fifo|rr|other|batch|idle Idle thread scheduling policy
  • –idle-priority=INT Idle thread scheduling priority
  • –log-srtp-keys Log SRTP keys to error log
  • –mysql-host=HOST|IP MySQL host for stored media files
  • –mysql-port=INT MySQL port
  • –mysql-user=USERNAME MySQL connection credentials
  • –mysql-pass=PASSWORD MySQL connection credentials
  • –mysql-query=STRING MySQL select query

Run command

rtpengine --interface="10.10.10.10" --listen-ng=25061 --listen-cli=25062 --foreground --log-stderr --listen-udp=25060 --listen-tcp=25060

In-Kernal Packet Forwarding

To avoid the overhead involved in processing each individual RTP packet in userspace-only operation, especially as RTP traffic consists of many small packets at high rates, rtpengine provides a kernel module to offload the bulk of the packet forwarding duties from user space to kernel space. This also results in increasing the number of concurrent calls as CPU usage decreases.In-kernel packet forwarding is implemented as an iptables module (x_tables) and has 2 parts – xt_RTPENGINE and plugin to the iptables and ip6tables command-line utilities

Sequence of events for a newly established media stream is then:

  1. Kamailio as SIP proxy controls rtpengine and signals it about a newly established call.
  2. Rtpengine daemon allocates local UDP ports and sets up preliminary forward rules based on the info received from the SIP proxy.
  3. An RTP packet is received on the local port.
  4. It traverses the iptables chains and gets passed to the xt_RTPENGINE module.
  5. The module doesn’t recognize it as belonging to an established stream and thus ignores it.
  6. The packet continues normal processing and eventually ends up in the daemon’s receive queue.
  7. The daemon reads it, processes it and forwards it. It also updates some internal data.
  8. This userspace-only processing and forwarding continues for a little while, during which time information about additional streams and/or endpoints may be obtained from the SIP proxy.
  9. After a few seconds, when the daemon is satisfied with what it has learned about the media endpoints, it pushes the forwarding rules to the kernel.
  10. From this moment on, the kernel module will recognize incoming packets belonging to those streams and will forward them on its own. It will stop those packets from traversing the network stacks any further, so the daemon will not see them any more on its receive queues.
  11. In-kernel forwarding is allowed to cease to work at any given time, either accidentally (e.g. by removal of the iptablesrule) or deliberatly (the daemon will do so in case of a re-invite), in which case forwarding falls back to userspace-only operation.

Kernel Module

The kernel module supports multiple forwarding tables, identified through their ID number, bydefault 0 to 63. Each running instance of the rtpengine daemon controls one such table.

To load use modprobe xt_RTPENGINE and to unload rmmod xt_RTPENGINE. With the module loaded, a new directory will appear in /proc/, namely /proc/rtpengine/, containing pseudo-files, control ( to create and delete forwarding tables) and list ( list of currently active forwarding tables)

To manually create a forwarding table with ID 33, the following command can be used:

echo 'add 43' > /proc/rtpengine/control

iptables module

In order for the kernel module to be able to actually forward packets, an iptables rule must be set up to send packets into the module. Each such rule is associated with one forwarding table. In the simplest case, for forwarding table 33, this can be done through:

iptables -I INPUT -p udp -j RTPENGINE --id 33

To restrict the rules to the UDP port range used by rtpengine, e.g. by supplying a parameter like –dport 30000:40000. If the kernel module receives a packet that it doesn’t recognize as belonging to an active media stream, it will simply ignore it and hand it back to the network stack for normal processing.

A typical start-up sequence including in-kernel forwarding might look like this:

modprobe xt_RTPENGINE
iptables -I INPUT -p udp -j RTPENGINE --id 0
ip6tables -I INPUT -p udp -j RTPENGINE --id 0

ensure that the table we want to use doesn’t exist – usually needed after a daemon restart, otherwise will error

echo 'del 0' > /proc/rtpengine/control

start daemon

/usr/sbin/rtpengine --table=0 --interface=10.64.73.31 --interface=2001:db8::4f3:3d \
--listen-ng=127.0.0.1:2223 --tos=184 --pidfile=/run/rtpengine.pid --no-fallback

Running Multiple Instances

To run multiple instances of rtpengine on the same machine run multiple instances of the daemon using different command-line options ( local addresses and listening ports), together with multiple different kernel forwarding tables.

For example, if one local network interface has address 10.64.73.31 and another has address 192.168.65.73, then the start-up sequence might look like this:

modprobe xt_RTPENGINE

iptables -I INPUT -p udp -d y.y.y.y -j RTPENGINE --id 0
iptables -I INPUT -p udp -d x.x.x.x -j RTPENGINE --id 1
echo 'del 0' > /proc/rtpengine/control
echo 'del 1' > /proc/rtpengine/control
/usr/sbin/rtpengine --table=0 --interface=<ip> \
--listen-ng=127.0.0.1:2223 --tos=184 --pidfile=/run/rtpengine-10.pid --no-fallback
/usr/sbin/rtpengine --table=1 --interface=<ip_pvy>\
--listen-ng=127.0.0.1:2224 --tos=184 --pidfile=/run/rtpengine-192.pid --no-fallback

With this setup, the SIP proxy can choose which instance of rtpengine to talk to and thus which local interface to use by sending its control messages to either port 2223 or port 2224.

Transcoding

Currently transcoding is supported for audio streams. Can be turned off with with_transcoding=no option in makeFile.

Normally rtpengine leaves codec negotiation up to the clients involved in the call and does not interfere. In this case, if the clients fail to agree on a codec, the call will fail.

Transcoding options in the ng control protocol,  transcode or ptime. If a codec is requested via the transcode option that was not originally offered, transcoding will be engaged for that call. With transcoding active for a call, all unsupported codecs will be removed from the SDP.

Transcoding happens in userspace only, so in-kernel packet forwarding will not be available for transcoded codecs. Codecs that are supported by both sides will simply be passed through transparently (unless repacketization is active). In-kernel packet forwarding will still be available for these codecs.

Codecs supported by rtpengine can be shown with –codecs options

  • rtpengine –codecs
  • PCMA: fully supported
  • PCMU: fully supported
  • G723: fully supported
  • G722: fully supported
  • QCELP: supported for decoding only
  • G729: supported for decoding only
  • speex: fully supported
  • GSM: fully supported
  • iLBC: not supported
  • opus: fully supported
  • vorbis: codec supported but lacks RTP definition
  • ac3: codec supported but lacks RTP definition
  • eac3: codec supported but lacks RTP definition
  • ATRAC3: supported for decoding only
  • ATRAC-X: supported for decoding only
  • AMR: supported for decoding only
  • AMR-WB: supported for decoding only
  • PCM-S16LE: codec supported but lacks RTP definition
  • PCM-U8: codec supported but lacks RTP definition
  • MP3: codec supported but lacks RTP definition

ng Control Protocol

Advanced control protocol to pass SDP body from the SIP proxy to the rtpengine daemon, has the body rewritten in the daemon, and then pas back to the SIP proxy to embed into the SIP message. It is  based on the bencode standard and runs over UDP transport.

Each message passed between the SIP proxy and the media proxy contains of two parts:

  1. message cookie ( to match requests to responses, and retransmission detection) and
  2. bencoded dictionary

The dictionary of each request must contain at least one key called command and corresponding value must be a string and determines the type of message. Currently the following commands are defined:

  • ping
  • offer
  • answer
  • delete
  • query
  • start recording
  • stop recording
  • block DTMF
  • unblock DTMF
  • block media
  • unblock media
  • start forwarding
  • stop forwarding
  • play media
  • stop media

The response dictionary must contain at least one key called result. The value can be either ok (optional key warning) or error( to be accompanied by error-reason). For the ping command, the additional value pong is allowed.

rtpengine.sample.conf

[rtpengine]

table = 0
no-fallback = false
for userspace forwarding only:
table = -1

// separate multiple interfaces with semicolons:
interface = internal/12.23.34.45;external/23.34.45.54

listen-ng = 127.0.0.1:2223
listen-tcp = 25060
listen-udp = 12222

timeout = 60
silent-timeout = 3600
tos = 184
control-tos = 184
delete-delay = 30
final-timeout = 10800

foreground = false
pidfile = /run/ngcp-rtpengine-daemon.pid
num-threads = 16

port-min = 30000
port-max = 40000
max-sessions = 5000

recording-dir = /var/spool/rtpengine
recording-method = proc
recording-format = raw

redis = 127.0.0.1:6379/5
redis-write = password@x.x.x.x:6379/42
redis-num-threads = 8
no-redis-required = false
redis-expires = 86400
redis-allowed-errors = -1
redis-disable-time = 10
redis-cmd-timeout = 0
redis-connect-timeout = 1000

b2b-url = http://127.0.0.1:8090/
xmlrpc-format = 0

log-level = 6
log-stderr = false
log-facility = daemon
log-facility-cdr = local0
log-facility-rtcp = local1

graphite = 127.0.0.1:9006
graphite-interval = 60
graphite-prefix = foobar.

homer = 123.234.345.456:65432
homer-protocol = udp
homer-id = 2001

sip-source = false
dtls-passive = false

ngcp-rtpengine-daemon Service

To start the ngcp-rtpengine-daemon service

/etc/init.d/ngcp-rtpengine-daemon start
[ ok ] Starting ngcp-rtpengine-daemon (via systemctl): ngcp-rtpengine-daemon.service.

Checking status ngcp-rtpengine-daemonservice

# systemctl status ngcp-rtpengine-daemon.service

● ngcp-rtpengine-daemon.service - NGCP RTP/media Proxy Daemon
   Loaded: loaded (/lib/systemd/system/ngcp-rtpengine-daemon.service; disabled; vendor preset: enabled)
   Active: active (running) since Thu 2019-04-11 10:16:20 UTC; 24s ago
  Process: 13751 ExecStopPost=/usr/sbin/ngcp-rtpengine-iptables-setup stop (code=exited, status=0/SUCCESS)
  Process: 13797 ExecStartPre=/usr/sbin/ngcp-rtpengine-iptables-setup start (code=exited, status=0/SUCCESS)
 Main PID: 13814 (rtpengine)
    Tasks: 19
   Memory: 10.5M
      CPU: 102ms
   CGroup: /system.slice/ngcp-rtpengine-daemon.service
           └─13814 /usr/sbin/rtpengine -f -E --no-log-timestamps --pidfile /run/ngcp-rtpengine-daemon.pid --config-file /etc/rtpengine/rtpengine.conf --table 0

To start recording service

/etc/init.d/ngcp-rtpengine-recording-daemon start

RTP engine receives command offer

Received command 'offer' from :53888
Dump for 'offer' from :53888: {  
"sdp":"v=0 
 o=- 1554978148897419 1 IN IP4 pvt_ip 
 s=Bria 3 release 3.5.5 stamp 71243 
 c=IN IP4 192.168.1.23 
 t=0 0 
 m=audio 50754 RTP/AVP 0 98 101 
 a=rtpmap:98 ILBC/8000 
 a=rtpmap:101 telephone-event/8000 
 a=fmtp:101 0-15 
 a=sendrecv 
 ",
    "ICE":"remove",
    "record-call":"yes",
    "direction":[  
       "internal",
       "internal"
    ],
    "flags":[  
       "no-rtcp-attribute"
    ],
    "replace":[  
       "origin",
       "session-connection"
    ],
    "transport-protocol":"RTP/AVP",
    "call-id":"732597d6-6d96-485b-b6dc-7d93703c1405",
    "received-from":[  
       "IP4",
       ""
Creating new call
Turning on call recording.
Wrote metadata file to temporary path: /var/spool/rtpengine/tmp/
...

RTP engine receives command delete

Received command 'delete' from :57304
Dump for 'delete' from :57304: { "call-id": "732597d6-6d96-485b-b6dc-7d93703c1405", "received-from": [ "IP4", "" ], "from-tag": "cb8a1e30", "command": "delete" }
Deleting call branch 'cb8a1e30' (via-branch '')
Call branch 'cb8a1e30' (via-branch '') deleted, no more branches remaining
  Deleting entire call
 INFO: [ID="732597d6-6d96-485b-b6dc-7d93703c1405"]: Final packet stats:
 --- Tag 'cb8a1e30', created 0:05 ago for branch '', in dialogue with ''
 ------ Media #1 (audio over RTP/AVP) using unknown codec
 --------- Port   :10044 <>    :50754, SSRC 0, 0 p, 0 b, 0 e, 5 ts
 freeing send_timer
 --------- Port   :10045 <>    :50755 (RTCP), SSRC 0, 0 p, 0 b, 0 e, 5 ts
 freeing send_timer
 --- Tag '', created 0:05 ago for branch '', in dialogue with 'cb8a1e30'
 ------ Media #1 (audio over RTP/AVP) using unknown codec
--------- Port   :10032 <>          (null):0    , SSRC 0, 0 p, 0 b, 0 e, 5 ts
freeing send_timer
--------- Port   :10033 <>          (null):0     (RTCP), SSRC 0, 0 p, 0 b, 0 e, 5 ts
freeing send_timer
 rtpengine: ci=732597d6-6d96-485b-b6dc-7d93703c1405, created_from=:53888, 
 last_signal=1554978149, 
 tos=0, 
 ml0_start_time=1554978149.645290, 
 ml0_end_time=1554978154.822680, 
 ml0_duration=5.177390, 
 ml0_termination=REGULAR, 
 ml0_local_tag=cb8a1e30, 
 ml0_local_tag_type=FROM_TAG, 
...



Telephony Solutions with Kamailio

  • Features
    • RFC3261 compliant
    • Customisable and Felxible – Modular Architecture
    • Call routing and control
    • NAT traversal
    • Data store integration
    • Transport Layers supported
    • Asynchronous TCP, UDP and SCTP
    • Secure Communication ( TLS + AAA)
    • IP and DNS
    • Accounting
    • External Integration
    • Rich Communication Services ( RCS)
  • Kamailio (OpenSER) SIP Server v4.3- default configuration script
    • debug
    • authentication
    • persistent user location
    • presence
    • nat traversal
    • RTPProxy
    • PSTN gateway
    • Block 3XX redirect repliest
    • Database aliases lookup
    • speed dial
    • multi-domain support
    • XMLRPC
    • anti-flood detection
  • Routing Logic
    • WITHINDLG
    • LOCATION
    • PRESENCE
    • AUTH
    • NATDETECT
    • NATMANAGE
    • DLGURI
    • SIPOUT
    • PSTN
    • XMLRPC
    • TOVOICEMAIL
    • MANAGE_BRANCH
    • MANAGE_REPLY
    • MANAGE_FAILURE
  • kamctl
  • kamdbctl
  • Kamctlrc
  • Kamailio.cfg
  • kamcmd
  • siremis

Kamailio™ (former OpenSER) is an Open Source SIP Server released under GPL.

Kamailio primarily acts as a SIP server for VOIP and telecommunications platforms under various roles and can handle load of hight CPS ( Calls per second ) with custom call routing logic with the help of scripts.

Rich features set suiting to telephony domain that includes IMS extensions for VoLTE; ENUM; DID and least cost routing; load balancing; routing fail-over; Json and XMLRPC control interface, SNMP monitoring.

To integrate with a carrier grade telecom network as SBC / gateway / inbound/outbound proxy , it can act as IPv4-IPv6 gateway , UDP/TCP/SCTP/WS translator and even had NAT and anti DOS attack support .

Features

If Kamailio is the central to the VoIP system it can also perform accounting with rich database extensions Mysql PostgreSQL UnixODBC Berkeley DB Oracle Redis, MongoDB Cassandra etc

  • Kamailio is SIP (RFC3261) compliant

It can work as Registrar or Location server. For SIP call logic it can become a Proxy or SIP Application server. Can also act like a Redirect, Dispatcher or simply a SIP over websocket server.

  • Customisable and Felxible

It can be embedded to devices as the binary file is small size. Additional modules can be appended for more functions with the same core.

Modular architecture – core, internal libraries , module interface and ability to extend functionality with scripts such as LUA, Kamailio can be readily integrated to a VOIP ecosystem.

Kamailio Architecture , Core and Modules.

  • Call routing and control with Scripting and programming

Offers stateless and transactional stateful SIP Proxy processing ( suited for inbound gateways ) and serial and parallel forking.

Kamailio Transaction management describes branches, serial and paralle forking and TM module.

Also NAT traversal support for SIP and RTP traffic ( suited to be WebRTC server ) . Read more about kamailio DNS subsystem management , load balancing , NAT and NAThelper modules in Kamailio DNS and NAT.

Kamailio as a WebRTC SIP Server with mdules websocket , TLS , NATHelper  and JSSIP integration.

Among other features it offers load balancing with many distribution algorithms and failover support, flexible least cost routing , routing failover and replication for High Availability (HA).

Can be readily integrated with external databases, caches, notification system ( SNS , APNS , GCM ), VoIP monitors, CDR processors, API systems etc for  efficient call processing.

  • Transport Layers supported 
    • UDP, TCP, TLS and SCTP
    • IPv4 and IPv6
    • gateways via (IPv4 to IPv6, UDP to TLS, a.s.o.)
    • SCTP multi-homing and multi-streaming
    • WebSocket for WebRTC 
  • Asynchronous TCP, UDP and SCTP

Asynchronous SIP message processing and  inter-process message queues communication system

  • Secure Communication ( TLS  + AAA)
    • Digest SIP User authentication
    • Authorization via ACL or group membership
    • IP and Network authentication
    • TLS support for SIP signaling
    • transparent handling of SRTP for secure audio
    • TLS domain name extension support
    • authentication and authorization against database (MySQL, PostgreSQL, UnixODBC, BerkeleyDB, Oracle, text files), RADIUS and DIAMETER

Kamailio Security here for snaity, ACL permission , firewall , flood detection , topology hiding and digests.

  • IP and DNS
    • support for SRV and NAPTR DNS lookups
    • SRV DNS failover
    • DNSsec support
    • ENUM support
    • internal DNS caching system – avoid DNS blocking
    • IP level Blacklists
    • multi-homed and multi-domain support
    • topology hiding – hide IP addresses in SIP headers to protect your network architecture
  • Accounting

Kamailio gives event based and configurable accounting data details. Can show multi-leg call accounting ( A leg to B leg ). It can store to database, Radius or Diameter based on module used . Has a prepaid engine.

  • External Interaction

text-based management interface via FIFO file, udp, xmlrpc and unix sockets.

RPC control interface – via XMLRPC, UDP or TCP

  • Rich Communication Services (RCS)
    • SIP SIMPLE Presence Server (rich presence)
    • Presence User Agent ( SUBSCRIBE , NOTIFY and PUBLSH)
    • XCAP client capabilities and Embedded XCAP Server
    • Presence DialogInfo support – SLA/BLA
    • Instant Messaging ( IM) 
    • Embedded MSRP relay
  • Monitoring and Troubleshooting

Support for SNMP – interface to Simple Network Management Protocol.  For Debugging it has config debugger , remote control via XMLRPC and error message logging system .Provides internal statistics exported via RPC and SNMP.

  • Extensibility APIs

The supported  one are Perl  , Java SIP Servlet Application Interface  , Lua  , Managed Code (C#) , Python.

Lua Scripts for kamailio Routing – KEMI interpreter , function , routing logic, PV variables

  • Multiple Database Backends

(MySQL, PostgreSQL, SQLite, UnixODBC, BerkeleyDB, Oracle, text files) and other database types which have unixodbc drivers. ‘

It can have connections pool and different backends  be used at same time (e.g., accounting to Oracle and authorization against MySQL).

Has connectors for Memcached, Redis , MongoDB and Cassandra no-SQL backends

  • Interconnectivity

Acts as SIP to PSTN gateway and gateway to sms or xmpp and other IM services. Has Interoperability with SIP enabled devices and applications such as SIP phones (Snom, Cisco, etc.), Media Servers (Asterisk, FreeSwitch, etc). More details on Kamailio as Inbound/Outbound proxy or Session Border Controller (SBC) here

  • IMS
    • diameter support and authentication
    • I-CSCF, P-CSCF, S-CSCF
    • charging, QOS, ISC
  • Miscellaneous
    • CPL – Call Processing Language (RFC3880)
    • Internal generic caching system
    • Memcached connector
    • Redis NoSQL database connector
    • CLI – kamctl and sercmd
    • Web Management Interface: Siremis
    • SIP-T and SIP-I
    • music on hold queue
    • message body compression/decompression (gzip-deflate)

Scalability with Kamailio system

  • Kamailio can run on embedded systems, with limited resources – the performances can be up to hundreds of call setups per second
  • used as load balancer in stateless mode, Kamailio can handle over 5000 call setups per second
  • on systems with 4GB memory, Kamailio can serve a population over 300 000 online subscribers
  • system can easily scale by adding more Kamailio servers
  • Kamailio can be used in geographic distributed VoIP platforms
  • Kamailio least-cost-routing scales up to millions of routing rules
  • straightforward failover and redundancy

Start Kamalio

service kamailo start

Kamaiio Logs tailing

tail -f /var/log/kamailio

To Check if Kamailio instance is running

>ps -ax | grep “kamailio”
 57411 ?        S      0:01 /usr/sbin/kamailio -f /etc/kamailio/kamailio.cfg -P /var/run/kamailio/kamailio.pid -m 4096 -M 128 -u root -g root

Read RTP engine on kamailio SIP server which focuses on setting up sipwise rtpegine to proxy rtp traffic from kamailio app server. Also daemon and kernal modules. ,transcoding , in-kernel packet forwarding , ngcontrol protocol etc.

Installation and Configuration

Installing kamailio from git repo

clone kamailio from their github https://github.com/kamailio/kamailio

goto desired branch. The contents of clonned folder are

COPYING    ChangeLog    INSTALL    ISSUES       Makefile        README README.md       doc       etc      misc       pkg      test      utils

run ‘make cfg’ which compiles using gcc abd created ‘src’ folder with contents

make cfg

Makefile Makefile.defs Makefile.groups Makefile.modules Makefile.rules Makefile.sources Makefile.utils core main.c modules.lst Makefile.cfg Makefile.dirs Makefile.libs Makefile.radius Makefile.shared Makefile.targets config.mak lib modules

Edit modules.lst to enable db_mysql and dialplan module

vim src/modules.lst

//the list of extra modules to compile
include_modules= db_mysql

Use ‘make’ command followed by make all. Make sure to have gcc bison and flex installed

make cfg
make all

Alternativey on debian system , use readymade measures like

apt install mysql-server
apt install kamailio kamailio-mysql-modules

To validate and verify the location of kamillio use ‘which kamailio’ which returns /usr/sbin/kamailio

For Modules installation, check all avaible modules with command ‘apt search kamailio’and to install a new module such as websockt module use ‘apt install kamailio-websocket-modules’

Database access : After installaing kamailio , edit the kamailio.cfg file in /etc/kamailio to set the reachabe SIP domain, database engine, username/password etc to connect to databaseand enable the kamdbctl script to run and create users and tables, etc.

SIP_DOMAIN=kamailio.org

SIP_DOMAIN=17.3.4.5

chrooted directory

$CHROOT_DIR=”/path/to/chrooted/directory”

database type: MYSQL, PGSQL, ORACLE, DB_BERKELEY, DBTEXT, or SQLITE by default none is loaded

DBENGINE=MYSQL

Run kamdbctl to create users and database now

kamdbctl create

the database created is name kamailio and its tables are

+---------------------+
 | Tables_in_kamailio  |
 +---------------------+
 | acc                 |
 | acc_cdrs            |
 | active_watchers     |
 | address             |
 | aliases             |
 | carrier_name        |
 | carrierfailureroute |
 | carrierroute        |
 | cpl                 |
 | dbaliases           |
 | dialog              |
 | dialog_vars         |
 | dialplan            |
 | dispatcher          |
 | domain              |
 | domain_attrs        |
 | domain_name         |
 | domainpolicy        |
 | globalblacklist     |
 | grp                 |
 | htable              |
 | imc_members         |
 | imc_rooms           |
 | lcr_gw              |
 | lcr_rule            |
 | lcr_rule_target     |
 | location            |
 | location_attrs      |
 | missed_calls        |
 | mohqcalls           |
 | mohqueues           |
 | mtree               |
 | mtrees              |
 | pdt                 |
 | pl_pipes            |
 | presentity          |
 | pua                 |
 | purplemap           |
 | re_grp              |
 | rls_presentity      |
 | rls_watchers        |
 | rtpproxy            |
 | sca_subscriptions   |
 | silo                |
 | sip_trace           |
 | speed_dial          |
 | subscriber          |
 | trusted             |
 | uacreg              |
 | uid_credentials     |
 | uid_domain          |
 | uid_domain_attrs    |
 | uid_global_attrs    |
 | uid_uri             |
 | uid_uri_attrs       |
 | uid_user_attrs      |
 | uri                 |
 | userblacklist       |
 | usr_preferences     |
 | version             |
 | watchers            |
 | xcap                |
 +---------------------+

Kamctlrc

The Kamailio configuration file for the control tools. Can set variables used in the kamctl and kamdbctl setup scripts. Per default all variables here are commented out, the control tools will use their internal default values. This file lets to edit  SIP domain, the database engine, username/password/ to connect to database, etc.

## your SIP domain
 SIP_DOMAIN=1.1.1.1
## chrooted directory
# $CHROOT_DIR="/path/to/chrooted/directory"
## database type: MYSQL, PGSQL, ORACLE, DB_BERKELEY, DBTEXT, or SQLITE
# by default none is loaded

# If you want to setup a database with kamdbctl, you must at least specify this parameter.

DBENGINE=MYSQL
## database host
# DBHOST=localhost
# DBPORT=3306
## database name (for ORACLE this is TNS name)
# DBNAME=kamailio
# database path used by dbtext, db_berkeley or sqlite
# DB_PATH="/usr/local/etc/kamailio/dbtext"
database read/write user
# DBRWUSER="kamailio"
## password for database read/write user
# DBRWPW="kamailiorw"

database read only user

# DBROUSER="kamailioro"
## password for database read only user
# DBROPW="kamailioro"
## database access host (from where is kamctl used)
# DBACCESSHOST=192.168.0.1

database super user (for ORACLE this is ‘scheme-creator’ user)

# DBROOTUSER="root"
## password for database super user
## - important: this is insecure, targeting the use only for automatic testing
## - known to work for: mysql
# DBROOTPW="dbrootpw"
## database character set (used by MySQL when creating database)
#CHARSET="latin1"
## user name column
# USERCOL="username"
# SQL definitions

# If you change this definitions here, then you must change them
# in db/schema/entities.xml too.
# FIXME
# FOREVER="2030-05-28 21:32:15"
# DEFAULT_Q="1.0"
# Program to calculate a message-digest fingerprint
# MD5="md5sum"
# awk tool
# AWK="awk"
# gdb tool
# GDB="gdb"

# If you use a system with a grep and egrep that is not 100% gnu grep compatible,
# e.g. solaris, install the gnu grep (ggrep) and specify this below.

grep tool
# GREP="grep"
# egrep tool
# EGREP="egrep"
# sed tool
# SED="sed"
# tail tool
# LAST_LINE="tail -n 1"
# expr tool
# EXPR="expr"

Describe what additional tables to install. Valid values for the variables below are yes/no/ask. With ask (default) it will interactively ask the user for an answer, while yes/no allow for automated, unassisted installs.

#If to install tables for the modules in the EXTRA_MODULES variable.

# INSTALL_EXTRA_TABLES=ask
# If to install presence related tables.
# INSTALL_PRESENCE_TABLES=ask
# If to install uid modules related tables.
# INSTALL_DBUID_TABLES=ask

 Define what module tables should be installed.

If you use the postgres database and want to change the installed tables, then you must also adjust the STANDARD_TABLES or EXTRA_TABLES variable accordingly in the kamdbctl.base script.

standard modules

# STANDARD_MODULES="
standard acc lcr domain group permissions registrar usrloc msilo
alias_db uri_db speeddial avpops auth_db pdt dialog dispatcher
dialplan"

extra modules

# EXTRA_MODULES="imc cpl siptrace domainpolicy carrierroute userblacklist htable purple sca"
type of aliases used: DB - database aliases; UL - usrloc aliases
- default: none , ALIASES_TYPE="DB"
control engine: RPCFIFO
 - default RPCFIFO
CTLENGINE="RPCFIFO"

## path to FIFO file for engine RPCFIFO
# RPCFIFOPATH="/var/run/kamailio/kamailio_rpc_fifo"

## check ACL names; default on (1); off (0)
# VERIFY_ACL=1

## ACL names-if VERIFY_ACL is set,only the ACL names from below list are accepted
# ACL_GROUPS="local ld int voicemail free-pstn"

## check if user exists (used by some commands such as acl);
## - default on (1); off (0)

# VERIFY_USER=1

## verbose - debug purposes - default '0'
# VERBOSE=1

## do (1) or don't (0) store plaintext passwords
## in the subscriber table - default '1'

# STORE_PLAINTEXT_PW=0

Kamailio START Options

PID file path – default is: /var/run/kamailio/kamailio.pid

# PID_FILE=/var/run/kamailio/kamailio.pid

Extra start options – default is: not set

# example: start Kamailio with 64MB share memory: STARTOPTIONS="-m 64"
# STARTOPTIONS=

Kamailio.cfg

config files are used to customize and deploy SIP services since each and every SIP packet is route based on policies specified in conf file ( routing blocks ). Location when installed from source – /usr/local/etc/kamailio/kamailio.cfg , when installed from package – /etc/kamailio/kamailio.cfg

The features in config file :-

  • User authentication

Kamailio doesn’t have user authentication by default , so to enable it one must

#!define WITH_MYSQL
#!define WITH_AUTH

kamdbctl tool is to be used for creating and managing the database.

kamdbctl create

Kamctl is used for adding subscriber information and password.

kamctl add altanai1 123
mysql: [Warning] Using a password on the command line interface can be insecure.
MySQL password for user 'kamailio@localhost': 
mysql: [Warning] Using a password on the command line interface can be insecure.
new user 'altanai1' added

More details in Tools section below .

  • IP authorization
  • accounting
  • registrar and location servicesTo have persisant location enabled so that records are not lost once kamailio are restarted , we need to save it to database and reload when restarting
#!define WITH_USRLOCDB
  • attacks detection and blocking (anti-flood protection)
  • NAT traversal

requires RTP proxy for RTP relay. NAT traversal support can be set by

#!define WITH_NAT
  • short dialing on server
  • multiple identities (aliases) for subscribers
  • multi-domain support
  • routing to a PSTN gateway
  • routing to a voicemail server
  • TLS encryption
  • instant messaging (pager mode with MESSAGE requests)
  • presence services

Kamailio (OpenSER) SIP Server v4.3- default configuration script

Several features can be enabled using ‘#!define WITH_FEATURE’ directives:

debugger params

define WITH_DEBUG
...
#!ifdef WITH_DEBUGs
modparam("debugger", "cfgtrace", 1)
#!endif

To run in debug mode: 

#!ifdef WITH_DEBUG
 debug=4
 log_stderror=yes
#!else
 debug=2
 log_stderror=no
#!endif

memdbg=5
memlog=5
log_facility=LOG_LOCAL0
fork=yes
children=4

To enable mysql: 

define WITH_MYSQL
...
#!ifdef WITH_MYSQL
loadmodule "db_mysql.so"
#!endif

To enable authentication

enable mysql
define WITH_AUTH

To enable IP authentication execute: enable mysql ,  enable authentication ,  define WITH_IPAUTH and  add IP addresses with group id ‘1’ to ‘address’ table.

To enable persistent user location :

enable mysql
define WITH_USRLOCDB

To enable presence server :

enable mysql
define WITH_PRESENCE

To enable nat traversal :

define WITH_NAT

RTPProxy

Install RTPProxy: http://www.rtpproxy.org

start RTPProxy:

rtpproxy -l your_public_ip -s udp:localhost:7722

option for NAT SIP OPTIONS keepalives: WITH_NATSIPPING

PSTN gateway

To enable PSTN gateway routing :

define WITH_PSTN

set the value of pstn.gw_ip

check route[PSTN] for regexp routing condition.

#!ifdef WITH_PSTN
pstn.gw_ip = "" desc "PSTN GW Address"
pstn.gw_port = "" desc "PSTN GW Port"
#!endif

Database aliases lookup

To enable database aliases lookup :

enable mysql
define WITH_ALIASDB

To enable speed dial lookup execute:

enable mysql
define WITH_SPEEDDIAL

To enable multi-domain support execute:

enable mysql
define WITH_MULTIDOMAIN
...
 #!ifdef WITH_MULTIDOMAIN# - the value for 'use_domain' parameter
 #!define MULTIDOMAIN 1
 #!else
 #!define MULTIDOMAIN 0
 #!endif

To enable TLS support execute:

adjust CFGDIR/tls.cfg as needed
define WITH_TLS

To enable XMLRPC support :

define WITH_XMLRPC

adjust route[XMLRPC] for access policy

To enable anti-flood detection execute:

adjust pike and htable=>ipban settings as needed (default is block if more than 16 requests in 2 seconds and ban for 300 seconds)

define WITH_ANTIFLOOD
...
      
route[REQINIT] { 
  #!ifdef WITH_ANTIFLOOD 
  # flood detection from same IP and traffic ban      
  if(src_ip!=myself) {                
    if($sht(ipban=&gt;$si)!=$null) {             
       # ip is already blocked             
       xdbg("request from blocked IP - $rm from $fu (IP:$si:$sp)\n");             
       exit;        
     }        
     if (!pike_check_req()) {             
       xlog("L_ALERT","ALERT: pike blocking $rm from $fu (IP:$si:$sp)\n");             
       $sht(ipban=&gt;$si) = 1;             
       exit;        
     }     
  }     

  if($ua =~ "friendly-scanner") {        
    sl_send_reply("200", "OK");         
    exit;     
  } 
  #!endif 
.. 
}

To block 3XX redirect replies execute:

define WITH_BLOCK3XX

To enable VoiceMail routing :

define WITH_VOICEMAIL

set the value of voicemail.srv_ip and adjust the value of voicemail.srv_port

ifdef WITH_VOICEMAIL
# VoiceMail Routing on offline, busy or no answer
# - by default Voicemail server IP is empty to avoid misrouting
voicemail.srv_ip = "" desc "VoiceMail IP Address"
voicemail.srv_port = "5060" desc "VoiceMail Port"
#!endif

To enhance accounting execute:

enable mysql
define WITH_ACCDB

add following columns to database

define WITH_MYSQL
define WITH_AUTH
define WITH_USRLOCDB

#!ifdef ACCDB_COMMENT
ALTER TABLE acc ADD COLUMN src_user VARCHAR(64) NOT NULL DEFAULT '';
ALTER TABLE acc ADD COLUMN src_domain VARCHAR(128) NOT NULL DEFAULT '';
ALTER TABLE acc ADD COLUMN src_ip varchar(64) NOT NULL default '';
ALTER TABLE acc ADD COLUMN dst_ouser VARCHAR(64) NOT NULL DEFAULT '';
ALTER TABLE acc ADD COLUMN dst_user VARCHAR(64) NOT NULL DEFAULT '';
ALTER TABLE acc ADD COLUMN dst_domain VARCHAR(128) NOT NULL DEFAULT '';
ALTER TABLE missed_calls ADD COLUMN src_user VARCHAR(64) NOT NULL DEFAULT '';
ALTER TABLE missed_calls ADD COLUMN src_domain VARCHAR(128) NOT NULL DEFAULT '';
ALTER TABLE missed_calls ADD COLUMN src_ip varchar(64) NOT NULL default '';
ALTER TABLE missed_calls ADD COLUMN dst_ouser VARCHAR(64) NOT NULL DEFAULT '';
ALTER TABLE missed_calls ADD COLUMN dst_user VARCHAR(64) NOT NULL DEFAULT '';
ALTER TABLE missed_calls ADD COLUMN dst_domain VARCHAR(128) NOT NULL DEFAULT '';
#!endif

enhanced DB accounting

#!ifdef WITH_ACCDB
modparam("acc", "db_flag", FLT_ACC)
modparam("acc", "db_missed_flag", FLT_ACCMISSED)
modparam("acc", "db_url", DBURL)
modparam("acc", "db_extra",
"src_user=$fU;src_domain=$fd;src_ip=$si;"
"dst_ouser=$tU;dst_user=$rU;dst_domain=$rd")
#!endif

Include Local Config If Exists

import_file “kamailio-local.cfg”

Value defines – IDs used later in config #!ifdef WITH_MYSQL # – database URL – used to connect to database server by modules such # as: auth_db, acc, usrloc, a.s.o.

 #!ifndef DBURL 
 #!define DBURL "mysql://kamailio:kamailiorw@localhost/kamailio" 
 #!define DBURL "mysql://kamailio:kamailiorw@localhost/kamailio" 
 # !endif



 # FLT_ - per transaction (message) flags 
 # FLB_ - per branch flags
 #!define FLT_ACC 1
 #!define FLT_ACCMISSED 2
 #!define FLT_ACCFAILED 3
 #!define FLT_NATS 5
 #!define FLB_NATB 6
 #!define FLB_NATSIPPING 

 !substdef "!MY_IP_ADDR!!g"
 #!substdef "!MY_DOMAIN!!g" 
 #!substdef "!MY_WS_PORT!8080!g"
 #!substdef "!MY_WSS_PORT!4443!g"
 #!substdef "!MY_WS_ADDR!tcp:MY_IP_ADDR:MY_WS_PORT!g"
 #!substdef "!MY_WSS_ADDR!tls:MY_IP_ADDR:MY_WSS_PORT!g"&amp;amp;lt;/pre&amp;amp;gt;
 
 #!define WITH_WEBSOCKETS
disable TCP (default on)
#disable_tcp=yes
enable_sctp = 0

disable the auto discovery of local aliases based on reverse DNS on IPs (default on)
#auto_aliases=no

add local domain aliases
#alias=”sip.mydomain.com”

//port to listen
port=5060
#!ifdef WITH_TLS
enable_tls=yes
#!endif

Life time of TCP connection when there is no traffic – a bit higher than registration expires to cope with UA behind NAT

Modules Section

set paths to location of modules (to sources or installation folders)

#!ifdef WITH_SRCPATH
mpath="modules/"
#!else
mpath="/usr/lib/x86_64-linux-gnu/kamailio/modules/"
#!endif

Load modules

loadmodule "mi_fifo.so"
loadmodule "kex.so"
loadmodule "corex.so"
loadmodule "tm.so"
loadmodule "tmx.so"
loadmodule "sl.so"
loadmodule "rr.so"
loadmodule "pv.so"
loadmodule "maxfwd.so"
loadmodule "usrloc.so"
loadmodule "registrar.so"
loadmodule "textops.so"
loadmodule "siputils.so"
loadmodule "xlog.so"
loadmodule "sanity.so"
loadmodule "ctl.so"
loadmodule "cfg_rpc.so"
loadmodule "mi_rpc.so"
loadmodule "acc.so"

#!ifdef WITH_AUTH
loadmodule "auth.so"
loadmodule "auth_db.so"
#!ifdef WITH_IPAUTH
loadmodule "permissions.so"
#!endif
#!endif

#!ifdef WITH_ALIASDB
loadmodule "alias_db.so"
#!endif

#!ifdef WITH_SPEEDDIAL
loadmodule "speeddial.so"
#!endif

#!ifdef WITH_MULTIDOMAIN
loadmodule "domain.so"
#!endif

#!ifdef WITH_PRESENCE
loadmodule "presence.so"
loadmodule "presence_xml.so"
#!endif

#!ifdef WITH_NAT
loadmodule "nathelper.so"
loadmodule "rtpproxy.so"
#!endif

#!ifdef WITH_TLS
loadmodule "tls.so"
#!endif

#!ifdef WITH_ANTIFLOOD
loadmodule "htable.so"
loadmodule "pike.so"
#!endif

#!ifdef WITH_XMLRPC
loadmodule "xmlrpc.so"
#!endif

#!ifdef WITH_DEBUG
loadmodule "debugger.so"
#!endif

#!ifdef WITH_WEBSOCKETS
loadmodule "xhttp.so"
#loadmodule "websocket.so"
loadmodule "nathelper.so"
#!endif

modules params for mi_fifo,ctl, tm and rr

----- mi_fifo params -----
#modparam("mi_fifo", "fifo_name", "/var/run/kamailio/kamailio_fifo")

----- ctl params -----
#modparam("ctl", "binrpc", "unix:/var/run/kamailio/kamailio_ctl")

----- tm params -----
# auto-discard branches from previous serial forking leg
modparam("tm", "failure_reply_mode", 3)
# default retransmission timeout: 30sec
modparam("tm", "fr_timer", 30000)
# default invite retransmission timeout after 1xx: 120sec
modparam("tm", "fr_inv_timer", 120000)

----- rr params -----
# set next param to 1 to add value to ;lr param (helps with some UAs)
modparam("rr", "enable_full_lr", 0)
# do not append from tag to the RR (no need for this script)
modparam("rr", "append_fromtag", 0)

registrar params

modparam("registrar", "method_filtering", 1)
/* uncomment the next line to disable parallel forking via location */
# modparam("registrar", "append_branches", 0)
/* uncomment the next line not to allow more than 10 contacts per AOR */
#modparam("registrar", "max_contacts", 10)
# max value for expires of registrations
modparam("registrar", "max_expires", 3600)
# set it to 1 to enable GRUU
modparam("registrar", "gruu_enabled", 0)

usrloc params – enable DB persistency for location entries

#!ifdef WITH_USRLOCDB
modparam("usrloc", "db_url", DBURL)
modparam("usrloc", "db_mode", 2)
modparam("usrloc", "use_domain", MULTIDOMAIN)
#!endif

auth_db params

#!ifdef WITH_AUTH
modparam("auth_db", "db_url", DBURL)
modparam("auth_db", "calculate_ha1", yes)
modparam("auth_db", "password_column", "password")
modparam("auth_db", "load_credentials", "")
modparam("auth_db", "use_domain", MULTIDOMAIN)
#!endif

permissions params

#!ifdef WITH_IPAUTH
modparam("permissions", "db_url", DBURL)
modparam("permissions", "db_mode", 1)
#!endif

alias_db params

#!ifdef WITH_ALIASDB
modparam("alias_db", "db_url", DBURL)
modparam("alias_db", "use_domain", MULTIDOMAIN)
#!endif

speeddial params

#!ifdef WITH_SPEEDDIAL
modparam("speeddial", "db_url", DBURL)
modparam("speeddial", "use_domain", MULTIDOMAIN)
#!endif

domain params

#!ifdef WITH_MULTIDOMAIN
modparam("domain", "db_url", DBURL)
modparam("domain", "register_myself", 1)
#!endif

presence params

#!ifdef WITH_PRESENCE
modparam("presence", "db_url", DBURL)
!endif

presence_xml params

modparam("presence_xml", "db_url", DBURL)
modparam("presence_xml", "force_active", 1)

WITH_NAT

rtpproxy params

modparam("rtpproxy", "rtpproxy_sock", "udp:127.0.0.1:7722")

nathelper params

 modparam("nathelper", "natping_interval", 30)
 modparam("nathelper", "ping_nated_only", 1)
 modparam("nathelper", "sipping_bflag", FLB_NATSIPPING)
 modparam("nathelper", "sipping_from", "sip:pinger@kamailio.org") 

params needed for NAT traversal in other modules

modparam("nathelper|registrar", "received_avp", "$avp(RECEIVED)")
modparam("usrloc", "nat_bflag", FLB_NATB)

tls params

#!ifdef WITH_TLS
modparam("tls", "config", "/etc/kamailio/tls.cfg")
#!endif

pike params

#!ifdef WITH_ANTIFLOOD
modparam("pike", "sampling_time_unit", 2)
modparam("pike", "reqs_density_per_unit", 16)
modparam("pike", "remove_latency", 4)
#!endif

htable params

ip ban htable with autoexpire after 5 minutes

modparam("htable", "htable", "ipban=&gt;size=8;autoexpire=300;")

xmlrpc params

modparam("xmlrpc", "route", "XMLRPC");
modparam("xmlrpc", "url_match", "^/RPC")

nathelper params

#!ifdef WITH_WEBSOCKETS
modparam("nathelper|registrar", "received_avp", "$avp(RECEIVED)")
#!endif

Note: leaving NAT pings turned off here as nathelper is <em>only</em> being used for&nbsp;WebSocket connections. NAT pings are not needed as WebSockets have&nbsp;their own keep-alives.

Routing Logic

Main SIP request routing logic processing of any incoming SIP request starts with this route . Read more on Kamailio Call routing and Control

request_route { 
    # per request initial checks 
    route(REQINIT);

    #!ifdef WITH_WEBSOCKETS 
        if (nat_uac_test(64)) {     
            force_rport();     
            if (is_method("REGISTER")) {          
                fix_nated_register();     
            } else {         
                fix_nated_contact();         
                if (!add_contact_alias()) {  
                   xlog("L_ERR", "Error aliasing contact \n");             
                   sl_send_reply("400", "Bad Request");                    
                   exit;         
                }     
            } 
        } 
    #!endif

    # NAT detection 
    route(NATDETECT); 

    # CANCEL processing 
    if (is_method("CANCEL")) {     
        if (t_check_trans()) {         
            route(RELAY);     
        }     
        exit; 
    } 

    # handle requests within SIP dialogs 
    route(WITHINDLG); 

    ### only initial requests (no To tag) 
    # handle retransmissions 
    if(t_precheck_trans()) { 
        t_check_trans(); 
        exit; 
    } 
    t_check_trans(); 

    # authentication 
    route(AUTH); 

    # record routing for dialog forming requests (in case they are routed) - remove preloaded route headers 
    remove_hf("Route"); 

    if (is_method("INVITE|SUBSCRIBE")) 
        record_route(); 

    # account only INVITEs 
    if (is_method("INVITE")) {     
        setflag(FLT_ACC); # do accounting 
    }

    # dispatch requests to foreign domains 
    route(SIPOUT); 

    ### requests for my local domains 
    # handle presence related requests 
    route(PRESENCE); 

    # handle registrations 
    route(REGISTRAR); 

    if ($rU==$null) {     
        # request with no Username in RURI     
        sl_send_reply("484","Address Incomplete");     
        exit; 
    } 
    
    # dispatch destinations to PSTN 
    route(PSTN); 

    # user location service 
    route(LOCATION); 
}

Wrapper for relaying requests

enable additional event routes for forwarded requests – serial forking, RTP relaying handling, a.s.o.

route[RELAY] {    
    if (is_method("INVITE|BYE|SUBSCRIBE|UPDATE")) {         
        if(!t_is_set("branch_route")) 
            t_on_branch("MANAGE_BRANCH");    
    }   
      
    if (is_method("INVITE|SUBSCRIBE|UPDATE")) {         
        
        if(!t_is_set("onreply_route"))
            t_on_reply("MANAGE_REPLY");    
        }   

        if (is_method("INVITE")) {        
            if(!t_is_set("failure_route")) 
                t_on_failure("MANAGE_FAILURE");     
        }    
        
        if (!t_relay()) {         
            sl_reply_error();     
        } 
    exit; 
}

Per SIP request initial checks

route[REQINIT] { 

if (!mf_process_maxfwd_header("10")) {     
   sl_send_reply("483","Too Many Hops");     
   exit; 
} 

if(is_method("OPTIONS") &amp;&amp; uri==myself &amp;&amp;; $rU==$null) {     
   sl_send_reply("200","Keepalive");     
   exit; 
} 

if(!sanity_check("1511", "7")) {     
  xlog("Malformed SIP message from $si:$sp\n");     
  exit; 
} 
}

Handle requests within SIP dialogs

sequential request withing a dialog should take the path determined by record-routing

route[WITHINDLG] {
    if (!has_totag()) return;
    if (has_totag()) {
        if (loose_route()) {
            #!ifdef WITH_WEBSOCKETS
            if ($du == "") {
                if (!handle_ruri_alias()) {
                    xlog("L_ERR", "Bad alias <$ru>\n");
                    sl_send_reply("400", "Bad Request");
                    exit;
                }
            }
            #!endif
         }
     exit;
     }

    if (loose_route()) {
        route(DLGURI);
        if (is_method("BYE")) {
            setflag(FLT_ACC); # do accounting ...
            setflag(FLT_ACCFAILED); # ... even if the transaction fails
        }
        else if ( is_method("ACK") ) {
            # ACK is forwarded statelessy
            route(NATMANAGE);
        }
        else if ( is_method("NOTIFY") ) {
            # Add Record-Route for in-dialog NOTIFY as per RFC 6665.
            record_route();
        }
        route(RELAY);
        exit;
    }

    if (is_method("SUBSCRIBE") && uri == myself) {
        # in-dialog subscribe requests
        route(PRESENCE);
        exit;
    }

if ( is_method("ACK") ) {
    if ( t_check_trans() ) {
        # no loose-route, but stateful ACK;
        # must be an ACK after a 487
        # or e.g. 404 from upstream server
        route(RELAY);
        exit;
    } else {
        # ACK without matching transaction ... ignore and discard
        exit;
    }
}

sl_send_reply("404","Not here");
exit;
}

Handle SIP registrations

tbd

User location service

route[LOCATION] {

#!ifdef WITH_SPEEDDIAL
# search for short dialing - 2-digit extension
if($rU=~"^[0-9][0-9]$")
    if(sd_lookup("speed_dial"))
    route(SIPOUT);
#!endif

#!ifdef WITH_ALIASDB
# search in DB-based aliases
    if(alias_db_lookup("dbaliases"))
    route(SIPOUT);
#!endif

$avp(oexten) = $rU;
if (!lookup("location")) {
    $var(rc) = $rc;
    route(TOVOICEMAIL);
    t_newtran();
    switch ($var(rc)) {
        case -1:
        case -3:
           send_reply("404", "Not Found");
        exit;
        case -2:
           send_reply("405", "Method Not Allowed");
        exit;
    }
}

# when routing via usrloc, log the missed calls also
if (is_method("INVITE")) {
    setflag(FLT_ACCMISSED);
}

route(RELAY);
exit;
}

Presence processing

route[PRESENCE] {

if(!is_method("PUBLISH|SUBSCRIBE"))
return;

if(is_method("SUBSCRIBE") && $hdr(Event)=="message-summary") {
    route(TOVOICEMAIL);
    # returns here if no voicemail server is configured
    sl_send_reply("404", "No voicemail service");
    exit;
}

#!ifdef WITH_PRESENCE
if (!t_newtran()) {
    sl_reply_error();
    exit;
}

if(is_method("PUBLISH")) {
    handle_publish();
    t_release();
} else if(is_method("SUBSCRIBE")) {
    handle_subscribe();
    t_release();
}
exit;
#!endif

# if presence enabled, this part will not be executed
if (is_method("PUBLISH") || $rU==$null) {
    sl_send_reply("404", "Not here");
    exit;
}
return;
}

IP authorization and user authentication

route[AUTH] {
#!ifdef WITH_AUTH

#!ifdef WITH_IPAUTH
if((!is_method("REGISTER")) && allow_source_address()) {
    # source IP allowed
    return;
}
#!endif

if (is_method("REGISTER") || from_uri==myself)
{
    # authenticate requests
    if (!auth_check("$fd", "subscriber", "1")) {
        auth_challenge("$fd", "0");
        exit;
    }

    # user authenticated - remove auth header
    if(!is_method("REGISTER|PUBLISH"))
        consume_credentials();
    }

# if caller is not local subscriber, then check if it calls
# a local destination, otherwise deny, not an open relay here
    if (from_uri!=myself && uri!=myself) {
        sl_send_reply("403","Not relaying");
        exit;
    }

#!endif
return;
}

Caller NAT detection

route[NATDETECT] {
    #!ifdef WITH_NAT
    force_rport();

    if (nat_uac_test("19")) {
        if (is_method("REGISTER")) {
            fix_nated_register();
        } else {
            if(is_first_hop())
                set_contact_alias();
        }
        setflag(FLT_NATS);
    }
    #!endif
    return;
}

RTPProxy control and signaling updates for NAT traversal

route[NATMANAGE] {

#!ifdef WITH_NAT
if (is_request()) {
    if(has_totag()) {
        if(check_route_param("nat=yes")) {
            setbflag(FLB_NATB);
        }
     }
}
if (!(isflagset(FLT_NATS) || isbflagset(FLB_NATB)))
return;

rtpproxy_manage("co");

if (is_request()) {
    if (!has_totag()) {
        if(t_is_branch_route()) {
            add_rr_param(";nat=yes");
        } 
    }
}

if (is_reply()) {
    if(isbflagset(FLB_NATB)) {
        if(is_first_hop())
        set_contact_alias();
    }
}

#!endif
return;
}

URI update for dialog requests

route[DLGURI] {
    #!ifdef WITH_NAT
    if(!isdsturiset()) {
        handle_ruri_alias();
    }
    #!endif
    return;
}

Routing to foreign domains

route[SIPOUT] {
    if (uri==myself) return;
    append_hf("P-hint: outbound\r\n");
    route(RELAY);
    exit;
}

PSTN GW routing

route[PSTN] {
  #!ifdef WITH_PSTN
  # check if PSTN GW IP is defined
  if (strempty($sel(cfg_get.pstn.gw_ip))) {
    xlog("SCRIPT: PSTN routing enabled but pstn.gw_ip not defined\n");
    return;
  }

  # route to PSTN dialed numbers starting with '+' or '00'(international format)
  if(!($rU=~"^(\+|00)[1-9][0-9]{3,20}$"))
  return;

  # only local users allowed to call
  if(from_uri!=myself) {
    sl_send_reply("403", "Not Allowed");
    exit;
  }

  if (strempty($sel(cfg_get.pstn.gw_port))) {
     $ru = "sip:" + $rU + "@" + $sel(cfg_get.pstn.gw_ip);
  } else {
     $ru = "sip:" + $rU + "@" + $sel(cfg_get.pstn.gw_ip) + ":"
     + $sel(cfg_get.pstn.gw_port);
  }

  route(RELAY);
  exit;
  #!endif
  return;
}

XMLRPC routing

#!ifdef WITH_XMLRPC
route[XMLRPC] {
if ((method=="POST" || method=="GET") && (src_ip==127.0.0.1)) {
# close connection only for xmlrpclib user agents
    if ($hdr(User-Agent) =~ "xmlrpclib")
        set_reply_close();
        set_reply_no_connect();
        dispatch_rpc();
    exit;
}
send_reply("403", "Forbidden");
exit;
}
#!endif

Routing to voicemail server

route[TOVOICEMAIL] {
#!ifdef WITH_VOICEMAIL
if(!is_method("INVITE|SUBSCRIBE"))
return;

# check if VoiceMail server IP is defined
if (strempty($sel(cfg_get.voicemail.srv_ip))) {
    xlog("SCRIPT: VoiceMail routing enabled but IP not defined\n");
    return;
}
if(is_method("INVITE")) {
    if($avp(oexten)==$null)
        return;
    $ru = "sip:" + $avp(oexten) + "@" + $sel(cfg_get.voicemail.srv_ip)+ ":" + $sel(cfg_get.voicemail.srv_port);
} else {
    if($rU==$null)
        return;
    $ru = "sip:" + $rU + "@" + $sel(cfg_get.voicemail.srv_ip)
+ ":" + $sel(cfg_get.voicemail.srv_port);
}
route(RELAY);
exit;
#!endif

return;
}

Manage outgoing branches

branch_route[MANAGE_BRANCH] {
    xdbg("new branch [$T_branch_idx] to $ru\n");
    route(NATMANAGE);
}

Manage incoming replies

onreply_route[MANAGE_REPLY] {
    xdbg("incoming reply\n");
    if(status=~"[12][0-9][0-9]")
        route(NATMANAGE);
}

Manage failure routing cases

failure_route[MANAGE_FAILURE] {
    route(NATMANAGE);

    if (t_is_canceled()) {
        exit;
    }

    #!ifdef WITH_BLOCK3XX
    # block call redirect based on 3xx replies.
    if (t_check_status("3[0-9][0-9]")) {
        t_reply("404","Not found");
        exit;
    }
    #!endif

    #!ifdef WITH_VOICEMAIL
    # serial forking - route to voicemail on busy or no answer (timeout)
    if (t_check_status("486|408")) {
        $du = $null;
        route(TOVOICEMAIL);
        exit;
    }
    #!endif
}

Supports pseudo-variables to access and manage parts of the SIP messages and attributes specific to users and server. Transformations to modify existing pseudo-variables, accessing only the wanted parts of the information.

Already has over 1000 parameters, variables and functions exported to config file. Supports runtime update framework – to avoid restarting the SIP server when needing to change the config parameters.

kamctl

Manage kamailio from command line, providing lots of operations, such as adding/removing/updating SIP users, controlling the ACL for users, managing the records for LCR or load balancing, viewing registered users and internal statistics, etc. When needed to interact with Kamailio, it does it via FIFO file created by mi_fifo module.

kamdbctl

Helps to configure and database needed by kamailio . First we need to select a database engine in the kamctlrc file by DBENGINE parameter .

Valid values are: MYSQL, PGSQL, ORACLE, DB_BERKELEY, DBTEXT.

The tool can be used to create and manage the database structure needed by Kamailio, therefore it should be immediately after Kamailio installation, in case you plan to run Kamailio with a database backend.

kamcmd

send RPC commands to Kamailio from command line, requires  ctl module

siremis

web management interface for Kamailio, written in PHP , AJAX , web 2.0 using MVC architecture

  • system and database administration tools for Kamailio SIP Server
  • subscriber, database aliases and speed dial management
  • location table view
  • dispatcher (load balancer), prefix-domain translation and least cost routing (lcr) management
  • access control lists (user groups) and permissions management
  • accounting records and missed calls vies
  • manage call data records (generated from acc records)
  • hash table, dial plan table and user preferences table management
  • offline message storage, presence service and sip trace views
  • communication with Kamailio SIP Server via XMLRPC ,  JSONRPC
  • communication with FreeSWITCH via event socket
  • create and display charts from statistic data stored by Kamailio
  • user location statistics charts
  • SIP traffic load charts
  • memory usage charts
  • accounting records charts and summary table
  • SQL-based CDR generation and rating billing engine

Covered in this article

  • Internal architecture
  • Configuration language
  • least cost routing
  • load balancing
  • traffic dispatching
  • DID routing
  • prefix based routing
  • SIP trunks and peering
  • traffic shaping
  • topology hiding
  • flood detection
  • scanning attacks prevention
  • anti-fraud policies
  • SQL and noSQL connectors
  • enum and DNS based routing
  • authentication and authorization
  • secure communication (TLS)
  • registration and location services
  • accounting and call data records
  • call control – redirect, forward, baring
  • redundancy and scalability
  • high availability and failover
  • websockets and webrtc

References :

  • [1] Henning Westerholt – Kamailio project-1&1 Internet AG ( 2009 )

Proxying Media Streams via Kamailio’s RTP Proxy


SIP/VOIP transformation towards IMS (Total IP)


The telecommunications industry has been going through a significant transformation over the past few years. At the outset incumbent operators used to focus on mainly basic voice services and still remained profitable due to the limited number of players in the space and requirement of huge amounts as initial investment.

However, with the advent of competitive vendors, rise in consumer base, and introduction of cost effective IP based technologies a major revolution has come about. This has enabled operators to come out of their traditional business models to maintain and enhance subscriber base by providing better and cheaper voice, multimedia and data services in order to grab the biggest possible share in this multi- billion dollar industry.

The evolution in Telecom industry has been accelerating all the time. The Next-Generation Operators wants to keep pace with the rapidly changing technology by, adapting to market needs and looking at the system and business process from multiple perspectives concurrently. Communication Service Providers (CSPs) need to consider several factors in mind before proposing any solution. They need to deploy solutions which are highly automated, highly flexible, caters to customer needs coupled with ultra low operating costs.

Upgrading a softswitrch solutions to IMS

The Softswitch is decomposed into two logical components of a subscriber-facing unit and a PSTN-facing unit. 

  • Subscriber facing unit in Softswitch is upgraded to AGCF (Access Gateway Control Function) 
  • PSTN facing unit is upgraded to MGCF (Media Gateway Controller Function) to interwork with IMS as shown.

By separating the Softswitch into these components, the network can be more easily scaled for better overall network efficiencies. More AGCFs can be added as required, allowing the network to scale with an increase in subscribers. Similarly, More PSTN trunks can be added as traffic increases. Once PSTN and subscriber control functions are separated, the IMS elements, CSCF and BGCF functions can be introduced. BGCF is the interface for interconnecting IMS with legacy PSTN networks.

New SIP-based services can now be rapidly rolled out by deploying new Application Servers (AS) and its integrations to other SBC for UCC( unified communication and colloboartion ) systems. IMS has 3GPP specified ISC interface, which is a SIP-based interface for interfacing-to-application servers. Using these constructs, multiple application servers from multiple vendors can be interconnected over the IMS ISC interface.

Intelligent Networks( IN)

Telecom networks (2014) are made up of integrated service digital network (ISDN), the public switched telephone network (PSTN) ,the Public Land Mobility Network (PLMN) and many others. Intelligent networks (IN) ensures that call control is handed over to a control platform. The control platform determines how the establishment of this call shall continue. Applying IN to any of these networks has in common that call establishment is intercepted at a designated node in the network

By hosting new services on the new platform and combining new and old services CSP‟s aim to provide service bundles that would generate new revenue streams. This process is largely dependant on IMS ( IP Multimedia Subsystem ) architecture .

Transformation towards IMS (Total IP)
Transformation towards IMS (Total IP)

Optimization in operator landscape evolve as result of synergistic technologies that come together to address the innovation and cost optimization needs of operator for better user experience. In following sections different technological evolutions that are affecting overall operator ecosystems have been discussed with focus towards Service Layer.

Fixed/mobile convergence(FMC) with IMS

“Fixed Mobile Convergence is a transition point in the telecommunications industry that will finally remove the distinctions between fixed and mobile networks, providing a superior experience to customers by creating seamless services using a combination of fixed broadband and local access wireless technologies to meet their needs in homes, offices, other buildings and on the go.”

 Fixed-Mobile Convergence Alliance (FMCA)  2004

System can communicate over the cellular network, or act as a new endpoint on the IP network. Home Subscriber Server (HSS) manages subscriber data uniformly between the cellular and IP worlds. The Handoff Server runs on top of the ISC interface, and provides a seamless experience when subscribers move from the cellular network to a Wi-Fi network. The AGCF remains the functional centre of the network, but with the introduction of the HSS, has added the Cx and Sh interfaces defined by the IMS.

Legacy to IP transformation

This section broadly covered the aspects of migration from legacy IN solution to new age JAINSLEE framework based one. Applies to Legacy IN hosting voice based services mostly  such as VPN, Access Screening ,Number Portability, SIP-Trunking,Call Gapping.

Most operator environments have seen a rise in the number of service delivery platforms. Also complexity of telecom networks have increased manifold hence CSPs are facing multiple challenges. Increased efforts and costs are required for maintaining all the SDP platforms. These platforms are generally of different vendors and cater to different technologies thereby greatly increase chances of limiting the scalability and flexibility of the operator landscape. More effort required for sustaining the life cycle of the platform and challenges in integrating non compatible SDPs due to proprietary design have been stumbling blocks in the progress of CSPs across the world.

To overcome these challenges there is trend in the market to move towards SDP consolidation wherein instead of maintaining several SDPs with their proprietary design CSPs prefer maintaining a single or less number of SDPs having standardized interfaces.

SDP consolidation
SDP consolidation (1)
SDP consolidation (2)

As illustrated in the above figure there is a transition that is taking place in the industry towards consolidation of service delivery session control. This would provide a cost effective sustenance of existing applications and the rapid creation and deployment of new services leading to increased revenue recognition by CSPs.

  • Agile Development
  • Innovative services
  • open SOA based architectures
  • IN/NGN Platform and Services
  • Reuse of existing investments in legacy service platforms
  • low cost of new service development
  • faster time to market
  • Monetize investment in Network Infrastructure uplift – SIP trunking, VoLTE etc.

Services that should be covered  in the Scope of Migration from fixed line to IP telephony are:

  • Virtual Private Network (VPN) : An Intelligent Network (IN) service, which offers the functions of a private telephone network. The basic idea behind this service is that business customers are offered the benefits of a (physical) private network, but spared from owning and maintaining it.
  • Access Screening(ASC): An IN service, which gives the operators the possibility to screen (allow/barring) the incoming traffic and decide the call routing, especially when the subscribers choose an alternate route/carrier/access network (also called Equal Access) for long distance calls on a call by call basis or pre-selected.
  • Number Portability(NP) : An IN service allows subscribers to retain their subscriber number while changing their service provider, location, equipment or type of subscribed telephony service. Both geographic numbers and non-geographic numbers are supported by the NP service.

 

WebRTC based Unified Communication platform

Using WebRTC Solution for Delivering In Context Voice which provides new monetizing benefits to the Enterprise customers of Service Providers. This includes following components:

  • WebRTC Gateway for implementation for inter-connect with SIP Legacy
  • Enhancement of WebRTC Client with new features like Cloud Address Book, Conferencing & Social Networking hooks.
  • Cloud based solutions
INtoJAISNLEE

Challenges in Migration to IMS  (Total IP )

Since long I have been advocating the benefits of migration to IMS  from a current fixed line / legacy/ proprietary VOIP / SS7 based system . However I decided to write this post on the challenges in migration to IMS system from a telecom provider’s view.  Though I could think of many , I have jot down the major 4 . they are as follows :

Data Migration challenges

  • Establishing a common data model definition
  • Data migration seamlessly
  • Configuration management
  • Extracting data from multiple sources and vendors , that includes legacy systems
  • Extracting data due to its large scale and volume

Training

  • Creating an effective knowledge share and transfer for live operations
  • Training in fallback plans, standards and policies .

Customer impact

  • Minimized customer outage
  • Enhance customer experience by delivering quality services on schedule
  • Ensuring security of customer’s confidential data
  • Transfer of customer services without any impact.

Testing in replicated environment

  • Physical pre-transfer test
  • Reducing cycle time
  • Verification and validation at every change in data environment
  • Detect production issues early in the test -lifecycle

Fallback plans

  • Pilot program and real network simulation for ensuring preparedness
  • Tracking changes in new network

 


Kamailio Transaction management and Transaction Module tm

Kamailio is basically only a transaction stateful proxy, without any dialog support built in. Here the TM module enables stateful processing of SIP transactions ( by maintaining state machine). State is a requirement for many complex logic such as accounting, forking, DNS resolution. 

Although most of the Kamailio module related description is covered here, I wanted to keep a separate space to describe and explain how Kamailio handles transactions and in particular, the Transaction Module.

Note: This article has been updated many times to match v5.1 since v3.0 from when it was written, if u see and outdated content or deprecated functions, please point them out to me in the comments. If you are new to Kamailio, this post is probably not a good starting point for you, instead read more on Kamailio. It is a powerful open-source SIP server here and has a widespread application in telephony.

Kamailio can manage stateless replying as well as stateful processing – SIP transaction management. The difference between the two is below

StatefulStateless
stateful processing is per SIP transaction

Each SIP transaction will be kept in memory so that any replies, failures, or retransmissions can be recognized
Forwarding each msg in the dialog without any context.
Application understands the transactions , for example
– recognize if a new INVITE message is a resend
– know that 200 OK reponse belongs to the initial INVITE which it will be able to handle in an onreply_route[x] block.
it doesnt
know that the call is on-going.
However it can use callId to match INVITE and BYE.
Uses : manage call state , routing , call control like forward on busy, voicemailUses : Load distribution , proxying

Kamailio’s Transaction management

t_relay, t_relay_to_udp and t_relay_to_tcp are main functions to setup transaction state, absorb retransmissions from upstream, generate downstream retransmissions and correlate replies to requests in Kamailio.

Lifecycle of Transaction

Transactions lifecycle are controlled by various factors which includes reliable ( TCP) or non reliable transport, invite or non-invite transaction types etc. Transaction are terminated either by final response or when timers are fired, which control it.

ACK is considered part of INVITE trasnaction when non 2xx / negative final resposne is received , When 2xx final / positive response is recievd than ACK is not considered part of the transaction.

Memory Management in Transactions

Transaction Module copies clones of received SIP messages in shared memory. Non-TM functions operate over the received message in private memory. Therefore core operations ( like record_route) should not be called before settings the transaction state ( t_realy ) for state-fully processing a message.

An INVITE transaction will be kept in memory for maximum: max_inv_lifetime + fr_timer + wt_timer.
While A non-INVITE transaction will be kept in memory for a maximum: max_noninv_lifetime + wt_timer.

Branches

A single SIP INVITE request may be forked to multiple destinations, all of which together is called destination sets and Individual elements within the destination sets are called branches. A transaction can have more than one branch. For example, during DNA failover, each failed DNS SRV destination can introduce a new branch.

Serial, Parallel and Combined Forking 

By default Kamailio performs parallel forking sending msg to all destinations and waiting for a response, however it can also do serial ie send requests one by one and wait for response /timeout before sending next. 

By use of priorities ( q value 0 – 1.0) , Kamailio can also intermix the forking technique ie describing priority oder for serial and same level for parallel . The destination uri are loaded using unctions t_load_contacts() and t_next_contacts().

Parallel forking snippet

request_route {
  seturi("sip:a@example.com");
  append_branch("sip:b@example.com");
  append_branch("sip:c@example.com");
  append_branch("sip:d@example.com");

  t_relay();
  break;
}

Mixed forking snippet

modparam("tm", "contacts_avp", "tm_contacts");
modparam("tm", "contact_flows_avp", "tm_contact_flows");

request_route {
  seturi("sip:a@example.com"); // lowest 0 
  append_branch("sip:b@example.com", "0.5"); // shoudl be in parallel with C
  append_branch("sip:c@example.com", "0.5"); // shoudl be in parallel with B
  append_branch("sip:d@example.com", "1.0"); // highest priority , should be tried first

  t_load_contacts();   // load all branches as per q values, store them in AVP configured in modparam 
  t_next_contacts();   // takes AVP and extracts higher q value branch

  t_relay();
  break;
}

Code to terminate when no more branches are found ( -1 returned) and return the message upstream

 failure_route["serial"]
 {
   if (!t_next_contacts()) {
     exit;
   }
   t_on_failure("serial");
   t_relay();
 }

TM Module

t_relay, t_relay_to_udp and t_relay_to_tcp are main functions to setup transaction state, absorb retransmissions from upstream, generate downstream retransmissions and correlate replies to requests.

Memory

TM copies clones of received SIP messages in shared memory. non-TM functions operate over the received message in private memory. Therefore core operations ( like record_route) should ne called before settings the trasnaction state ( t_realy ) for statefully processing a message.

An INVITE transaction will be kept in memory for maximum: max_inv_lifetime + fr_timer + wt_timer.
While A non-INVITE transaction will be kept in memory for a maximum: max_noninv_lifetime + wt_timer.

Parameters

Various parameters are used to fine tune how trsnactions are handled and timedout in kamailio. Note all timers are set in miliseconds notation.

  • fr_timer (integer) – timer hit when no final reply for a request or ACK for a negative INVITE reply arrives. Default 30000 ms (30 seconds).
  • fr_inv_timer (integer) – timer hit when no final reply for an INVITE arrives after a provisional message was received on branch. Default 120000 ms (120 seconds).
  • restart_fr_on_each_reply (integer) – restart fr_inv_timer fir INVITE transaction for each provisional reply. Otherwise it will be sreatred only for fisrt and then increasing provisonal replies. Turn it off in cases when dealing with bad UAs that continuously retransmit 180s, not allowing the transaction to timeout.
  • max_inv_lifetime (integer) – Maximum time an INVITE transaction is allowed to be active in a tansaction. It starts from the time trnsaction was created and after this timer is hit , transaction is moved to either wait state or in the final response retransmission state. Default 180000 ms (180 seconds )
  • max_noninv_lifetime (integer) – Maximum time a non-INVITE transaction is allowed to be active. default 32000 ms (32 seconds )
  • wt_timer (integer) – Time for which a transaction stays in memory to absorb delayed messages after it completed.
  • delete_timer (integer) – Time after which a to-be-deleted transaction currently ref-ed by a process will be tried to be deleted again. This is now obsolte and now transaction is deleted the moment it’s not referenced anymore.

Retry transmission timers

  • retr_timer1 (integer) – Initial retransmission period
  • retr_timer2 (integer) – Maximum retransmission period started increasingly from starts with retr_timer1 and stays constant after this
  • noisy_ctimer (integer) – if set, INVITE transactions that time-out (FR INV timer) will be always replied. Otherwise they will be quitely dropped without any 408 branch timeout resposne
  • auto_inv_100 (integer) – automatically send and 100 reply to INVITEs.
  • auto_inv_100_reason (string) – Set reason text of the automatically sent 100 to an INVITE.
  • unix_tx_timeout (integer) – nix socket transmission timeout,
  • aggregate_challenges (integer) – if more than one branch received a 401 or 407 as final response, then all the WWW-Authenticate and Proxy-Authenticate headers from all the 401 and 407 replies will be aggregated in a new final response.

Blacklist

  • blst_503 (integer) – reparse_invite=1.
  • blst_503_def_timeout (integer) – blacklist interval if no “Retry-After” header is present
  • blst_503_min_timeout / blst_503_max_timeout (integer) – minimum and maximun blacklist interval respectively
  • blst_methods_add (unsigned integer) – Bitmap of method types that trigger blacklisting on transaction timeouts and by default INVITE triggers blacklisting only
  • blst_methods_lookup (unsigned integer) – Bitmap of method types that are looked-up in the blacklist before being forwarded statefully. For default only applied to BYE.

Reparse

  • reparse_invite (integer) – set if CANCEL and negative ACK requests are to be constructed from the INVITE message ( same record-set etc as INVITE ) which was sent out instead of building them from the received request.
  • reparse_on_dns_failover (integer) – SIP message after a DNS failover is constructed from the outgoing message buffer of the failed branch instead of from the received request.
  • ac_extra_hdrs (string) – Header fields prefixed by this parameter value are included in the CANCEL and negative ACK messages if they were present in the outgoing INVITE. Can be only used with reparse_invite=1.
  • on_sl_reply (string) – Sets reply route block, to which control is passed when a reply is received that has no associated transaction.
modparam("tm", "on_sl_reply", "stateless_replies")
...
onreply_route["stateless_replies"] {
    // return 0 if do not allow stateless replies to be forwarded
    return 1; // will pass to core for stateless forwading
}
  • xavp_contact (string) – name of XAVP storing the attributes per contact.
  • contacts_avp (string) – name of an XAVP that stores names of destination sets. Used by t_load_contacts() and t_next_contacts() for forking branches
  • contact_flows_avp (string) – name of an XAVP that were skipped
  • fr_timer_avp (string) – override teh value of fr_timer on per transactio basis , outdated
  • cancel_b_method (integer) – method to CANCEL an unreplied transaction branch. Params :
    • 0 will immediately stop the request (INVITE) retransmission on the branch so that unrpelied branches will be terminated
    • 1 will keep retransmitting the request on unreplied branches.
    • 2 end and retransmit CANCEL even on unreplied branches, stopping the request retransmissions.
  • unmatched_cancel (string) – sets how to forward CANCELs that do not match any transaction. Params :
    • 0 statefully
    • 1 statelessly
    • 2 dropping them
  • ruri_matching (integer) – try to match the request URI when doing SIP 1.0 transaction matching as older SIP didnt have via cookies as in RFC 3261
  • via1_matching (integer) – match the topmost “Via” header when doing SIP 1.0 transaction matching
  • callid_matching (integer) – match the callid when doing transaction matching.
  • pass_provisional_replies (integer)
  • default_code (integer) – Default response code sent by t_reply() ( 500 )
  • default_reason (string) – Default SIP reason phrase sent by t_reply() ( “Server Internal Error” )
  • disable_6xx_block (integer)- treat all the 6xx replies like normal replies. However according to RFC receiving a 6xx will cancel all the running parallel branches, will stop DNS failover and forking.
  • local_ack_mode (integer) – where locally generated ACKs for 2xx replies to local transactions are sent. Params :
    • 0 – the ACK destination is choosen according next hop in contact and the route set and then DNS resolution is used on it
    • 1 – the ACK is sent to the same address as the corresponding INVITE branch
    • 2 – the ACK is sent to the source of the 2xx reply.
  • failure_reply_mode (integer) – how branches are managed and replies are selected for failure_route handling. Params :
    • 0 – all branches are kept
    • 1 – all branches are discarded
    • 2 – only the branches of previous leg of serial forking are discarded
    • 3 – all previous branches are discarded
    • if you dont want to drop all branches then use t_drop_replies() to sleectively drop
  • faked_reply_prio (integer) – how branch selection is done.
  • local_cancel_reason (boolean) – add reason headers for CANCELs generated due to receiving a final reply.
  • e2e_cancel_reason (boolean) – add reason headers for CANCELs generated due to receiving a CANCEL
  • remap_503_500 (boolean) – conversion of 503 response code to 500. RFC requirnment.
  • failure_exec_mode (boolean) – Add local failed branches in timer to be considered for failure routing blocks.
  • dns_reuse_rcv_socket (boolean) – reuse of the receive socket for additional branches added by DNS failover.
  • event_callback (str) – function in the kemi configuration file (embedded scripting language such as Lua, Python, …) to be executed instead of event_route[tm:local-request] block. The function recives a string param with name of the event.
modparam("tm", "event_callback", "ksr_tm_event")
...
function ksr_tm_event(evname)
    KSR.info("===== TM module triggered event: " .. evname .. "\n");
    return 1;
end
  • relay_100 (str) – whether or not a SIP 100 response is proxied. not valid behavior when operating in stateful mode and only useful when in stateless mode
  • rich_redirect (int) – to add branch info in 3xx class reply. Params :
    0 – no extra info is added (default)
    1 – include branch flags as contact header parameter
    2 – include path as contact uri Route header

Functions

These functions are operational blocks and route handlers for trsnactions handling in kamailio

  • t_relay([host, port]) – Relay a message statefully.
    Exmaple to show if t_relay fails, atleast send a reply to UAC statelessly to not keep it waiting
if (!t_relay()) 
{ 
    sl_reply_error(); 
    break; 
};
  • t_relay_to_udp([ip, port]) / t_relay_to_tcp([ip, port]) – same as above, relay a message statefully but using specific protocol
if (some_conditon)
    t_relay_to_udp("1.2.3.4", "5060"); # sent to 1.2.3.4:5060 over udp
else
    t_relay_to_tcp(); # relay to msg. uri, but over tcp
  • t_relay_to_tls([ip, port])
  • t_relay_to_sctp([ip, port])
  • t_on_failure(failure_route) – on route block for failure management on a branch when a negative reply is recived to transaction. here uri is reset to value which it had on relaying.
  • t_on_branch_failure(branch_failure_route) – controls when negative response come for a transacion. here uri is reset to value which it had on relaying.
  • t_on_reply(onreply_route) – gets control when a reply from transaction is received
  • t_on_branch(branch_route) – control is passed after forking (when a new branch is created)
  • t_newtran() – Creates a new transaction
  • t_reply(code, reason_phrase) – Sends a stateful reply after a transaction has been established.
  • t_send_reply(code, reason)
  • t_lookup_request() – Checks if a transaction exists
  • t_retransmit_reply()
  • t_release() – Remove transaction from memory
  • t_forward_nonack([ip, port]) – forward a non-ACK request statefully
  • t_forward_nonack_udp(ip, port) / t_forward_nonack_tcp(ip, port)
  • t_forward_nonack_tls(ip, port)
  • t_forward_nonack_sctp(ip, port)
  • t_set_fr(fr_inv_timeout [, fr_timeout]) – Sets the fr_inv_timeout
  • t_reset_fr()
  • t_set_max_lifetime(inv_lifetime, noninv_lifetime) – Sets the maximum lifetime for the current INVITE or non-INVITE transaction, or for transactions created during the same script invocation
  • t_reset_max_lifetime()
  • t_set_retr(retr_t1_interval, retr_t2_interval) – Sets the retr_t1_interval and retr_t2_interval for the current transaction
  • t_reset_retr()
  • t_set_auto_inv_100(0|1) – switch automatically sending 100 replies to INVITEs on/off on a per transaction basis
  • t_branch_timeout() – Returns true if the failure route is executed for a branch that did timeout.
  • t_branch_replied()
  • t_any_timeout()
  • t_any_replied()
  • t_grep_status(“code”)
  • t_is_canceled()
  • t_is_expired()
  • t_relay_cancel()
  • t_lookup_cancel([1])
  • t_drop_replies([mode])
  • t_save_lumps()
  • t_load_contacts()
  • t_next_contacts()
  • t_next_contact_flow()
  • t_check_status(re)
  • t_check_trans() – check if a message belongs or is related to a transaction.
  • t_set_disable_6xx(0|1)
  • t_set_disable_failover(0|1)
  • t_set_disable_internal_reply(0|1)
  • t_replicate([params]) – Replicate the SIP request to a specific address.
  • t_relay_to(proxy, flags) – KSR.tm.t_relay()
  • t_set_no_e2e_cancel_reason(0|1)
  • t_is_set(target) – KEMI – KSR.tm.t_is_set() Return true if the attribute specified by ‘target’ is set for transaction. Target can be branch_route , failure_route and onreply_route.
if not(KSR.tm.t_is_set("branch_route")>0) then
    core.set_branch_route("ksr_branch_manage");
end

if not(KSR.tm.t_is_set("onreply_route")>0) then
    core.set_reply_route("ksr_onreply_manage");
end

if not(KSR.tm.t_is_set("failure_route")>0) and (req_method == "INVITE") then
   core.set_failure_route("ksr_failure_manage");
end
  • t_use_uac_headers()
  • t_is_retr_async_reply()
  • t_uac_send(method, ruri, nexthop, socket, headers, body)
  • t_get_status_code() – Return the status code for transaction or -1 in case of error or no status code was set.

Snippet to demo stateful handling of trsansactions

Yhis program is designed to accept all Register with 200 OK and create a new transaction. Does a check for username altanai. After the check cutom message hello is replied and any other username is printed a different rejection reply.

# ------------------ module loading ----------------------------------
loadmodule "tm.so"

route{
    # for testing purposes, simply okay all REGISTERs
    if (method=="REGISTER") {
        log("REGISTER");
        sl_send_reply("200", "ok");
        break;
    };

    # create transaction state with t_newtran(); abort if error occurred
    if (t_newtran()){
        log("New Transaction created"); 
    }
    else {
        sl_reply_error();
        break;
    };

    log(1, "New Transaction Arrived\n");

    # add a check for matching username to print a cutom message with t_reply()
    if (uri=~"altanai@") {
        if (!t_reply("409", "Well , hello altanai !")) {
            sl_reply_error();
        };
    } else {
        if (!t_reply("699", "Do not proceed with this one")) {
            sl_reply_error();
        };
    };
}

Raw RPC cmds

1. kamctl rpc tm.list

{  

   "jsonrpc":"2.0",
   "result":[  
      {  
         "cell":"0x7f0698d06488",
         "tindex":50969,
         "tlabel":163886326,
         "method":"INVITE",
         "from":"From: ;tag=dddab54e\r\n",
         "to":"To: \r\n",
         "callid":"Call-ID: NjkyYjJlNzJkNzQ1OTYyZjE2MDM2NjFlYWZkNjY4OWE\r\n",
         "cseq":"CSeq: 1",
         "uas_request":"yes",
         "tflags":65,
         "outgoings":2,
         "ref_count":1,
         "lifetime":29578635
      }
   ],
   "id":3922
}

2. kamctl rpc tm.stats

before call

{  
   "jsonrpc":"2.0",
   "result":{  
      "current":0,
      "waiting":0,
      "total":3,
      "total_local":0,
      "rpl_received":6,
      "rpl_generated":6,
      "rpl_sent":6,
      "6xx":0,
      "5xx":3,
      "4xx":0,
      "3xx":0,
      "2xx":0,
      "created":3,
      "freed":3,
      "delayed_free":0
   },
   "id":4119
}

during call

{  

   "jsonrpc":"2.0",
   "result":{  
      "current":1,
      "waiting":0,
      "total":4,
      "total_local":0,
      "rpl_received":7,
      "rpl_generated":7,
      "rpl_sent":7,
      "6xx":0,
      "5xx":3,
      "4xx":0,
      "3xx":0,
      "2xx":0,
      "created":4,
      "freed":3,
      "delayed_free":0
   },
   "id":4217
}

during call wait

{  
   "jsonrpc":"2.0",
   "result":{  
      "current":1,
      "waiting":1,
      "total":4,
      "total_local":0,
      "rpl_received":8,
      "rpl_generated":8,
      "rpl_sent":8,
      "6xx":0,
      "5xx":4,
      "4xx":0,
      "3xx":0,
      "2xx":0,
      "created":4,
      "freed":3,
      "delayed_free":0
   },
   "id":4275
}

after call is completed

{  
   "jsonrpc":"2.0",
   "result":{  
      "current":0,
      "waiting":0,
      "total":4,
      "total_local":0,
      "rpl_received":8,
      "rpl_generated":8,
      "rpl_sent":8,
      "6xx":0,
      "5xx":4,
      "4xx":0,
      "3xx":0,
      "2xx":0,
      "created":4,
      "freed":4,
      "delayed_free":0
   },
   "id":4333
}

SIP VoIP system architecture basics


A VOIP/CPaaS solution is designed to accommodate the signalling and media both along with integration leads to various external endpoints such as various SIP phones ( desktop, softphones, webRTC ), telecom carriers, different VoIP networks providers, enterprise applications ( Skype, Microsoft Lync ), Trunks etc.

A sufficiently capable SIP platform should have

  1. Audio calls ( optionally video ) service using SIP gateways
  2. Media services (such as recording , conferencing, voicemail, and IVR )
  3. Messaging and presence ( could be using SIP SIMPLE, SMS , messahing service from third parties)
  4. Developing SIP based applications : Programmable services through standardized APIs and development of new modules
  5. NAT and DNS near-end and far-end NAT traversal for signalling and media flows
  6. Telemetry for Sessions , Registry, Location and lookup service
  7. CDR Processing and Billing : Backend for CDR and accounts ( can use Redis, Kafka , MySQL, PostgreSQL, Oracle, Radius, LDAP, Diameter)
  8. Serial and parallel forking, load balancing , proxying
  9. Cross platform and integration to External Telecommunication provider landscape
    • Interconnectivity with other IP multimedia systems, VoLTE ( optional interconnection with other types of communications networks as GSM or PSTN/ISDN).
    • support for VoIP signalling protocols (SIP, H,323, SCCP, MGCP, IAX) and telephony signalling protocols ( ISDN/SS7, FXS/FXO, Sigtran ) either internally via pluggable modules or externally via gateways .
Performnace factors :Security considerations :
High availability using redundant servers in standby
Load balancing
IPv4 and IPv6 network layer support
TCP , UDP , SCTP transport layer protocol support
DNS lookups and hop by hop connectvity
authentication, authorization, and accounting (AAA)
Digest authentication and credentials fetched from backend
Media Encryption
TLS and SRTP support
Topology hidding to prevent disclosing IP form internal components in via and route headers
Firewalls , blacklist, filters , peak detectors to prevent Dos and Ddos attacks

The article only outlines SIP system architecture  from 3 viewpoints :

  1. Infrastructure standpoint
  2. Vore voice engineering perspective
  3. External components required to run and system

Infrastructure Requirements

  • Data Centers with BCP ( Business Continuity Planning ) and DR ( Disaster Recovery )
  • Servers and Clusters for faster and parallel calculating
  • Virtualization
    VMs to make a distributed computing environment with HA ( high availability ) and DRS ( Distributed Resource Scheduling )
  • Storage
    SAN with built-in redundancy for the resiliency of data.
    WORM compliant NAS for storing voice archives over a retention period.
  • Racks, power supplies, battery backups, cages etc.
  • Networking
    DMZs ( Demilitarized Zones)  which are interfacing areas between internal servers in the green zone and outside network
    VLANs for segregation between tenants.
    Connectivity through the public Internet as well as through VPN or dedicated optical fibre network for security.
  • Firewall configuration
  • Load Balancer ( Layer 7 )
  • Reverse Proxies for the security of internal IPs and port
  • Security controls In compliance with ISO/IEC 27000 family – Information security management systems
  • PKI Infrastructure to manage digital certificates
  • Key management with HSM ( hardware security module )
  • truster CA ( Certificate Authority ) to issue publicly signed certificate for TLS ( Https, wss etc)
  • OWASP ( Open Web Application Security Project )  rules compliance

Integral Components of a VOIP SIP based architecture

  • Call Controller
  • Media Manager
  • Recording
  • Softclients
  • logs and PCAP archives
  • CDR generators
  • Session Borer Controllers ( SBCs)

A SIP server can be moulded to take up any role based on the libraries and programs that run on it such as gateway server, call manager, load balancer etc. This in turn defines its placement in overall VoIP communication architecture. For example
– stateless proxy servers are placed on the border,
– application and B2BUA server at the core

sip entities
SIP platform components

SIP Gateways

A SIP gateway is an application that interfaces a SIP network to a network utilising another signalling protocol. In terms of the SIP protocol, a gateway is just a special type of user agent, where the user agent acts on behalf of another protocol rather than a human. A gateway terminates the signalling path and can also terminate the media path .

sip gaeways
To PSTN for telephony inter-working
To H.323 for IP Telephony inter-working
Client – originates message
Server – responds to or forwards message

Logical SIP entities are:

  • User Agent Client (UAC): Initiates SIP requests  ….
  • User Agent Server (UAS): Returns SIP responses ….
  • Network Servers ….

Registrar Server

A registrar server accepts SIP REGISTER requests; all other requests receive a 501 Not Implemented response. The contact information from the request is then made available to other SIP servers within the same administrative domain, such as proxies and redirect servers. In a registration request, the To header field contains the name of the resource being registered, and the Contact header fields contain the contact or device URIs.

regsitrar server

Proxy Server

A SIP proxy server receives a SIP request from a user agent or another proxy and acts on behalf of the user agent in forwarding or responding to the request. Just as a router forwards IP packets at the IP layer, a SIP proxy forwards SIP messages at the application layer.

Typically proxy server ( inbound or outbound) have no media capabilities and ignore the SDP . They are mostly bypassed once dialog is established but can add a record-route .
A proxy server usually also has access to a database or a location service to aid it in processing the request (determining the next hop).

proxy server

 1. Stateless Proxy Server
A proxy server can be either stateless or stateful. A stateless proxy server processes each SIP request or response based solely on the message contents. Once the message has been parsed, processed, and forwarded or responded to, no information (such as dialog information) about the message is stored. A stateless proxy never retransmits a message, and does not use any SIP timers

2. Stateful Proxy Server
A stateful proxy server keeps track of requests and responses received in the past, and uses that information in processing future requests and responses. For example, a stateful proxy server starts a timer when a request is forwarded. If no response to the request is received within the timer period, the proxy will retransmit the request, relieving the user agent of this task.

  3 . Forking Proxy Server
A proxy server that receives an INVITE request, then forwards it to a number of locations at the same time, or forks the request. This forking proxy server keeps track of each of the outstanding requests and the response. This is useful if the location service or database lookup returns multiple possible locations for the called party that need to be tried.

Redirect Server

A redirect server is a type of SIP server that responds to, but does not forward, requests. Like a proxy server, a redirect server uses a database or location service to lookup a user. The location information, however, is sent back to the caller in a redirection class response (3xx), which, after the ACK, concludes the transaction. Contact header in response indicates where request should be tried .

redirect server

Application Server

The heart of all call routing setup. It loads and executes scripts for call handling at runtime and maintains transaction states and dialogs for all ongoing calls . Usually the one to rewrite SIP packets adding media relay servers, NAT . Also connects external services like Accounting , CDR , stats to calls .

Adding Media Management

Media processing is usually provided by media servers in accordance to the SIP signalling. Bridges, call recording, Voicemail, audio conferencing, and interactive voice response (IVR) are commomly used. Read more about Media Architecture here

RFC 6230 Media Control Channel Framework decribes framework and protocol for application deployment where the application programming logic and media processing are distributed.

Any one such service could be a combination of many smaller services within such as Voicemail is a combitional of prompt playback, runtime controls, Dual-Tone Multi-Frequency (DTMF) collection, and media recording. RFC 6231 Interactive Voice Response (IVR) Control Package for the Media Control Channel Framework.

DTMF( Dual tone Multi Frequency )

delivery options:

  • Inband –  With Inband digits are passed along just like the rest of your voice as normal audio tones with no special coding or markers using the same codec as your voice does and are generated by your phone.
  • Outband  – Incoming stream delivers DTMF signals out-of-audio using either SIP-INFO or RFC-2833 mechanism, independently of codecs – in this case, the DTMF signals are sent separately from the actual audio stream.

TTS ( Text to Speech )

 Alexa Text-to-Speech (TTS) + Amazon Polly

Ivona – multiple language text to speech converter with ssml scripts such as below

      <speak>
          <p>
              <s><prosody rate="slow">IVONA</prosody> means highest quality speech
              synthesis in various languages.</s>
              <s>It offers both male and female radio quality voices <break/> at a
              sampling rate of 22 kHz <break/> which makes the IVONA voices a
              perfect tool for professional use or individual needs.</s>
          </p>
      </speak>

check ivona status

service ivona-tts-http status
 tail -f /var/log/tts.log

Developing SIP based applications

Basic SIP methods

SIP defines basic methods such as INVITE, ACK and BYE which can pretty much handle simple call routing with some more advanced processoes too like call forwarding/redirection, call hold with optional Music on hold, call parking, forking, barge etc.

Extending SIP headers

Newer SIP headers defined by more updated SIP RFC’s contina INFO, PRACK, PUBLISH, SUBSCRIBY, NOTIFY, MESSAGE, REFER, UPDATE. But more methods or headers can be added to baseline SIP packets for customization specific to a particular service provider. In case where a unrecognized SIP header is found on a SIP proxy which it either does not suppirt or doesnt understand, it will simply forward it to the specified endpoint.

Call routing Scripts

Interfaces for programming SIP call routing include :
– Call Processing Language—SIP CPL,
– Common Gateway Interface—SIP CGI,
– SIP Servlets,
– Java API for Integrated Networks—JAIN APIs etc .

Some known SIP stacks :

SailFin – SIP servlet container uses GlassFish open source enterprise Application Server platform (GPLv2), obsolete since merger from Sun Java to Oracle.

Mobicents – supports both JSLEE 1.1 and SIP Servlets 1.1 (GPLv2)

Cipango – extension of SIP Servlets to the Jetty HTTP Servlet engine thus compliant with both SIP Servlets 1.1 and HTTP Servlets 2.5 standards.

WeSIP – SIP and HTTP ( J2EE) converged application server build on OpenSER SIP platform

Additionally SIP stacks are supported on almost all popular SIP programming lanaguges which can be imported as lib and used for building call routing scripts to be mounted on SIP servers or endpoints such as :

PJSIP in C

JSSIP Javascript

Sofia in kamailio , Freswitch

Some popular SIP server also have proprietary scripting language such as –
Asterisk Gateway Interface (AGI) , application interface for extending the dialplan with your functionality in the language you choose – PHP, Perl, C, Java, Unix Shell and others

SIP platform Development

  • audio calls ( optionally video )
  • media services such as conferencing, voicemail, and IVR,
  • messaging as IM and presence based on SIMPLE,
  • programmable services through standardized APIs and development of new modules
  • near-end and far-end NAT traversal for signalling and media flows
  • interconnectivity with other IP multimedia systems, VoLTE ( optional interconnection with other types of communications networks as GSM or PSTN/ISDN)
  • Registry, location and lookup service
  • Serial and parallel forking

A sufficiently capable SIP platform shoudl consist of following features :

Performance factors :

  • High availability using redundant servers in standby
  • Load balancing
  • IPv4 and IPv6 support

Security considerations :

  • digest authentication and credentials fetched from backend
  • Media Encryption
  • TLS and SRTP support
  • Topology hiding to prevent disclosng IP form internal components in via and route headers
  • Firewalls , blacklist, filters , peak detectors to prevent Dos and Ddos attacks .

Collecting and Processing PCAPS

  • VoIP monitor – network packet sniffer with commercial frontend for SIP RTP RTCP SKINNY(SCCP) MGCP WebRTC VoIP protocols

it uses a passive network sniffer (like tcpdump or wireshark) to analyse packets in realtime and transforms all SIP calls with associated RTP streams into database CDR record which is sent over the TCP to MySQL server (remote or local). If enabled saving SIP / RTP packets the sniffer stores each VoIP call into separate files in native pcap format (to local storage).

voip monitor
  • sngrep
  • tcpdump
  • custom made pcap capture and uploader

NAT and DNS

To adapt SIP to modern IP networks with inter network traversal ICE, far and near-end NAT traversal solutions are used. Network Address traversal is crtical to traffic flow between private public network and from behind firewalls and policy controlled networks
One can use any of the VOVIDA-based STUN server, mySTUN , TurnServer, reStund , CoTURN , NATH (PJSIP NAT Helper), ReTURN, or ice4j

Near-end NAT traversal

STUN (session traversal utilities for NAT) – UA itself detect presence of a NAT and learn the public IP address and port assigned using Nating. Then it replaces device local private IP address with it in the SIP and SDP headers. Implemented via STUN, TURN, and ICE.
limitations are that STUN doesnt work for symmetric NAT (single connection has a different mapping with a different/randomly generated port) and also with situations when there are multiple addresses of a end point.

TURN (traversal using relay around NAT) or STUN relay – UA learns the public IP address of the TURN server and asks it to relay incoming packets. Limitatiosn since it handled all incoming and outgong traffic, it must scale to meet traffic requirments and should not become the bottle neck junction or single point of failure.

ICE (interactive connectivity establishment) – UA gathers “candidates of communication” with priorities offered by the remote party. After this client pairs local candidates with received peer candidates and performs offer-answer negotiating by trying connectivity of all pairs, therefore maximising success. The types of candidates :
– host candidate who represents clients’ IP addresses,
– server reflexive candidate for the address that has been resolved from STUN
– and a relayed candidate for the address which has been allocated from a TURN relay by the client.

Far-end NAT traversal

UA is not concerned about NAT at all and communicated using its local IP port. The border controller implies a NAT handling components such as an application layer gateway (ALG) or universal plug and play (UPnP) etc which resolves the private and public network address mapping by act as a back to back user agent (B2BUA).
Far end NAT can also be enabled by deploying a public SIP server which performs media relay (RTP Proxy/Media proxy).

Limitations of this approach
(-) security risks as they are operating in the public network
(-) enabling reverse traffic from UAS to UAC behind NAT.

A keep-alive mechanism is used to keep NAT translations of communications between SIP endpoint and its serving SIP servers opened , so that this NAT translation can be reused for routing. It contains client-to-server “ping” keep-alive and corresponding server-to-client “pong” messages. The 2 keep-alive mechanisms: a CRLF keep-alive and a STUN keep-alive message exchange.

The 3 types of SIP URIs,

  • address of record (AOR)
  • fully qualified domain name (FQDN)
  • globally routable user agent (UA) URI
    SIP uniform resource identifiers (URIs) are identified based on DNS resolution since the URI after @ symbol contains hostname , port and protocl for the next hop.

Adding record route headers for locating the correct SIP server for a SIP message can be done by :
– DNS service record (DNS SRV)
– naming authority pointer (NAPTR) DNS resource record

Steps for SIP endpoints locating SIP server

  1. From SIP packet get the NAPTR record to get the protocl to be used
  2. Inspect SRV record to fetch port to use
  3. Inspect A/AAA record to get IPv4 or IPv6 addresses
    ref : RFC 3263 – Locating SIP Servers
    Can use BIND9 server for DNS resolution supports NAPTR/SRV, ENUM, DNSSEC, multidomains, and private trees or public trees.

CDR Processing and Billing

CDR store call detail records along with proof of call with tiemstamps, orignation, destination, duaration, rate etc. At the end of month or any other term, the aggregated CDR are cumulatively processed to generate the bill for a user. This heavy data stream needs to be accurately processed and this can be achived by using data-pipelines like AWS kinesis or Kafka eventstore.

The prime requirnment for the system is to handle enormous amount of call records data in relatime , cater to a number of producers and consumers.

For security the data is obfuscated into blob using base 64 encoding.

For good consistency only a single shard should be rsponsible to process one user account’s bill.

Data Streams for billing service

AWS Kinesis – Kinesis Data Streams is sued for for rapid and continuous data intake and aggregation. The type of data used can include IT infrastructure log data, application logs, social media, market data feeds, and web clickstream data. It supports data sharding (ie number of call records grouped) and uses a partition Key ( string MD5 hash) to determine which shard the record goes to. 

(+) This system can handle high volume of data in realtime and produce call uuid specfic reults which can be consumed by consumers waiting for the processed results

(-) If not consumed with a pre-specified time duration the processed results expire and are irretrivable . Self implement publisher to store teh processed reults from kisesis stream to data stores like Redis / RDBMS or other storge locations like s3 , dynamo DB. If pieline crashes during operation , data is lost

(-) Data stream should have low latency igesting contnous data from producer and presenting data to consumer.

Call Rate and Accounting

Generally data streams proecssing are used for crtical and voluminious service usage like for
– metering/billing
– server activity,
– website clicks,
– geo-location of devices, people, and physical goods

Call Rates are very crticial for billing and charging the calls . Any updates from the customer or carriers or individuals need to propagate automatically and quickly to avoid discrpencies and neagtive margins. CDRs need to be processed sequentially and incrementally on a record-by-record basis or over sliding time windows, and used for a wide variety of analytics including correlations, aggregations, filtering, and sampling.

To acheieve this the follow setup is ideal to use the new input rate sheet values via web UI console or POST API and propagate it quickly to main DB via AWS SQS which is a queing service and AWS lamda which is a serverless trigger based system . This ensures that any new input rates are updates in realtime and maintin fallback values in s3 bucket too

Call Rate and Accounting using task pipes , lambda serverless and qiueing service. Uses s3 buckets , AWS lambda, AWS SQS and AWS RDS.
Call Rate and Accounting using task pipes , lambda serverless and qiueing service

Cross platform and integration to External Telecommunication provider landscape

It is an advantage to plan for ahead for connection with IMS such as openIMS, support for Voip signalling protocols (SIP, H,323, SCCP, MGCP, IAX) and telephony signalling protocls ( ISDN/SS7, FXS/FXO, Sigtran ) either internally via pluggable modules or externally via gateways or for SIP trunking integration via OTT providers/ cloud telephony.

Adhere to Standard

The obvious starting milestone before making a full-scale carrier-grade, SIP-based VoIP system is to start by building a PBX for intra-enterprise communication. There are readily available solutions to make an IP telephony PBX Kamailio, FreeSWITCH, asterisk, Elastix, SipXecs. It is important to use the standard protocol and widely acceptable media formats and codecs to ensure interoperability and reduce compute and delay involved in protocol or media transcoding.

Database Integration

Need backend , cache , databse integration to npt only store routing rules with temporary varaible values but also aNeed backend, cache, database integration to not only store routing rules with temporary variable values but also account details, call records details, access control lists etc. Should therefore extend integration with text-based DB, Redis, MySQL, PostgreSQL, OpenLDAP, and OpenRadius.

Consistency of Call Records and duplicated charging records at various endpoints

In current Voip scenarios a call may be passing thorugh various telco providers , ISP and cloud telephony serviIn current VoIP scenarios, a call may be passing through various telco providers, ISP and cloud telephony service providers where each system maintains its own call records and billing. This in my opinion is duplication and can be avoided by sharing a consistent data store possible in the blockchain. This is an experimental idea that I have further explored in this article


There are other external components to setup a VOIP solution apart from Core voice Servers and gateways like the ones listed below, I will try to either add a detailed overall architecture diagram here or write about them in an seprate article. Keep watching this space for updates

  • Payment Gateways
  • Billing and Invoice
  • Fraud Prevention
  • Contacts Integration
  • Call Analytics
  • API services
  • Admin Module
  • Number Management ( DIDs ) and porting
  • Call Tracking
  • Single Sign On and User Account Management with Oauth and SAML
  • Dashboards and Reporting
  • Alert Management
  • Continuous Deployment
  • Automated Validation
  • Queue System
  • External cache

References :

SIP solutioning and architectures is a subsequent article after SIP introduction, which can be found here.

Read about VoIP/ OTT / Telecom Solution startup’s strategy for Building a scalable flexible SIP platform which includes :

  • Scalable and Flexible SIP platform building
  • Cluster SIP telephony Server for High Availability
  • Failure Recovery
  • Multi-tier cluster architecture
  • Role Abstraction / Micro-Service based architecture
  • Distributed Event management and Event-Driven architecture
  • Containerization
  • Autoscaling Cloud Servers
  • Open standards and Data Privacy
  • Flexibility for inter-working – NextGen911 , IMS , PSTN
  • security and Operational Efficiencies

SIP ( Session Initiation Protocol )

Update : At the time of writing this article on SIP and related VOIP technologies I was newbie in VOIP domain, probably just out college. However over the past decade, looking at the steady traffic to these articles, I have tried updating the same with new RFC standards and market trends. This is an updated version (2019).


The Session Initiation Protocol (SIP) is a multimedia signalling protocol that has evolved the defacto communication standard for IP telephony. Today it forms the primary protocol for many Real Time Communication platforms which are integrated with telecom carriers and provide Cloud and IP based Services for applications such as robo/mass calls for advertising, API based calls like OTP generator, IVR announcements with DTMF input like customer care centre etc. Infact it would be not far from truth to say that converged platform we find today are a result of SIP integrating with the IP world.

Converged platforms integrates audio, video, data, presence, instant messaging, voicemails and conference services into a single network . SIP is the key component to build an advanced converged IP communication platform or rich multimedia Real time communication service.

SIP can be used to create programmable APIs and complex call routing VoIP scripts such as PBX , SBC etc.

Bears the support of many high quality open source and freeware SIP client , servers , proxies , tool such as Kamailio, Astersk, Freeswitch, Sipp, JAINSIP etc. Also supported on most standardised VoIP hardware and network such as Cisco, Microsoft, Avaya, and Radvision.

It is standardized by Internet Engineering Task Force (IETF) such as RFC 3261 which describes SIP v2 . Architecturally SIP request response (404 , 301) format is very similar to HTTP and its addressing schemes have a resemblance to SMTP (sip:altanai@company.com).

SIP – Application layer protocol

We know the ISO OSI layers  which servers as a standard model for data communications .

  1. Physical Layer : Ethernet , USB , IEEE 802.11  WiFi, Bluetooth  , BLE
  2. Data Link Layer : ARP ( Address Resolution Protocol ) ,  PPP ( point to point protocol ) , MAC ( Media Access control ) , ATM , Frame Relay
  3. Network Layer :  IP (IPv4 / IPv6), ICMP, IPsec
  4. Transport : TCP , UDP , SCTP
  5. Session : PPTP ( Point to point tunnelling protocol) , NFS, SOCKS
  6. Presentation : Codecs such as JPEG , GIFF , SSL
  7. Application : Application level like Call -manager/ softphone  as HTTP , FTP , DNS , SIP  , RTSP , RTP , DNS

SIP is an application layer protocol

SIP and SDP as Application layer protocols

SIP ( Session Initiation Protocol) negotiates session between 2 parties. It primarily exchanges headers that are used for making a call session such as example of outgoing telephone call from SIP session invite.

Session Initiation Protocol (INVITE)
Request-Line: INVITE sip:altanai@telecomcompany.com;transport=tcp SIP/2.0
Method: INVITE
Request-URI: altanai@telecomcompany.com;transport=tcp
        Request-URI User Part: altanai
        Request-URI Host Part: telecomcompany.com
        [Resent Packet: False]

Message Header

Via: SIP/2.0/TCP 1.2.3.4:5080;rport;branch=z9hG4bKceX7a2H2866cN
        Transport: TCP
        Sent-by Address: 1.2.3.4
        Sent-by port: 5080
        RPort: rport
        Branch: z9hG4bKceX7a2H2866cN

Max-Forwards: 41

From: "+16014801797" <sip:+16014801797@1.2.3.4>;tag=7HKgjNQ6y2FSj
        SIP Display info: "+16014801797"
        SIP from address: sip:+16014801797@1.2.3.4
                SIP from address User Part: +16014801797
                E.164 number (MSISDN): 16014801797
                        Country Code: Americas (1)
                SIP from address Host Part: 1.2.3.4
        SIP from tag: 7HKgjNQ6y2FSj

To: <sip:altanai@telecomcompany.com;transport=tcp>
        SIP to address: sip:altanai@telecomcompany.com;transport=tcp
        SIP to address User Part: altanai
        SIP to address Host Part: telecomcompany.com
        SIP To URI parameter: transport=tcp

Call-ID: e10306be-0cfd-4b38-af3c-b2ada0827cef
CSeq: 126144925 INVITE
Contact: <sip:mod_sofia@1.2.3.4:5080;transport=tcp>
User-Agent: phone1
Allow: INVITE, ACK, BYE, CANCEL, OPTIONS, MESSAGE, INFO, UPDATE, REFER, NOTIFY
Supported: path, replaces
Allow-Events: talk, hold, conference, refer
Privacy: none
Content-Type: application/sdp
Content-Disposition: session
Content-Length: 249
SIP Display info: "+16014801797"
SIP PAI Address: sip:+16014801797@1.2.3.4

The SIP philosophy :

  • reuse Internet addressing (URLs, DNS, proxies)
  • utilize rich Internet feature set
  • reuse HTTP coding
  • text based
  • makes no assumptions about underlying protocol: TCP, UDP, X.25, frame, ATM, etc
  • support of multicast

SIP URI can either be in format of sip:altanai@telecomcompnay.com (RFC 2543 ) or sips:altanai@telecomcompany.com ( secure with TLS over TCP RFX 3261) . Additionally SIP URI resolution can either be

  • DNS SRV based such as altanai@telecomcompnay.com with SIP servers locating record for domain “telecomcompnay.com ” or
  • FQDN ( Fully qualified domain name ) / contact / ip address based such as altanai@2.2.2.2 or altanai@us-west1-prod-server . Both of which do not need any resolution for routing.

Tags are pseudo-random numbers inserted in To or From headers to uniquely identify a call leg

Max forwards  is a count decremented by each proxy
that forwards the request.When count goes to zero, request is discarded and 483 Too Many Hops response is sent.Used for stateless loop detection.

Content-Type indicates the type of message body attachment. In this case application /SDP but  others could be text/plain, application/cpl+xml, etc.)

Content-Length indicates the octet (byte) count of the message body

Contact direct route to contact the sender, composed of SIPURI with a user name and IP or FQDN. USed for later requests to directly reach the destination such as ACK after INVITE

via gives the last SIP hop as IP, transport, and transaction-specific parameters along with branch that identifies the transaction
each proxy adds an additional via header. fianlly via header is used to route back the responses . This ensures the user agents after the initial request dont have to rely on DNS and location tables to route the messages.

Firewalls can sometimes block SIP packets, change TCP to UDP or change IP address of the packets. Record-Route can be used, ensures Firewall proxy stays in path. Clients and Servers copy Record-Route and put in Route header for all messages.

Message body is separated from SIP header fields by a blank line (CRLF).

sip arch
SIP Message Body

SIP Request methods

  • INVITE : Initiates negotiation to establish a session ( dialog). Usually contains SDP payload.
  • Another invite during an existing session ( dialog) is called an RE-INVITE.  RE-INVITE can be used for hold / resume a call and change session parameters and codecs in mid of a call
  • ACK : Acknowledge an INVITE request by completing the 3 way handshake.
  • If an INVITE did not contain media contain then ACK must contain it .
  • BYE : Ends a session ( dialog).
  • CANCEL : Cancels a session( dialog)  before it establishes  .
  • REGISTER : Registers a user location (host name, IP) on a registrar SIP server.
  • OPTIONS : Communicates information about the capabilities of the calling and receiving SIP phones ( methods , extensions , codecs etc )
  • PRACK : Provisional Acknowledgement for provisional response as 183 ( session in progress). PRACK only application to 101- 199 responses .
  • SUBSCRIBE : Subscribes for Notification from the notifier. Can use Expire=0 to unsubscribe.
  • NOTIFY : Notifies the subscriber of a new event.
  • PUBLISH : Publishes an event to the Server.
  • INFO : Sends mid session information.
  • REFER : Asks the recipient to issue call transfer.
  • MESSAGE : Transports Instant Messages.
  • UPDATE : Modifies the state of a session ( dialog).

SIP responses :

  • 1xx = Informational SIP Responses

100 Trying
180 Ringing
183 Session Progress

  • 2xx = Success Responses

200 OK – Shows that the request was successful

  • 3xx = Redirection Responses
  • 4xx = Request Failures

401 Unauthorized
404 Not Found
405 Method Not Allowed
407 Proxy Authentication Required
408 Request Timeout
480 Temporarily Unavailable
481 Call/Transaction Does Not Exist
486 Busy Here
487 Request Terminated
488 Not Acceptable Here
482 Loop Detected
483 Too Many Hops

  • 5xx = Server Errors

500 Server Internal Error
503 Service Unavailable

  • 6xx = Global Failures

600 Busy Everywhere
603 Decline
604 Does Not Exist Anywhere
606 Not Acceptable

SIP callflow diagram for a Call Setup and termination using RTP for media and RTCP for control.

Read about SIP messages indepth here 

SDP ( Session Description Protocol)

SIP can bear many kinds of MIME attachments , one such is SDP. SDP contains session metadata used for establishing the session. It defines media information and capabilities such as codecs and formats , timestamps , termination points like address , ports. Additionally it can also convey other details like bandwidth and contact for the node acting as proxy for the session.

Read Indepth about SIP messages and SDP https://telecom.altanai.com/2014/01/03/sip-in-depth/(opens in a new tab)

Sample SDP payload for Invite SIP above :

Session Description Protocol Version (v): 0
Owner/Creator, Session Id (o): FreeSWITCH 1532932581 1532932582 IN IP4 1.2.3.4
        Owner Username: FreeSWITCH
        Session ID: 1532932581
        Session Version: 1532932582
        Owner Network Type: IN
        Owner Address Type: IP4
        Owner Address: 1.2.3.4
Session Name (s): FreeSWITCH
Connection Information (c): IN IP4 1.2.3.4
        Connection Network Type: IN
        Connection Address Type: IP4
        Connection Address: 1.2.3.4
Time Description, active time (t): 0 0
        Session Start Time: 0
        Session Stop Time: 0
Media Description, name and address (m): audio 29398 RTP/AVP 0 101
        Media Type: audio
        Media Port: 29398
        Media Protocol: RTP/AVP
        Media Format: ITU-T G.711 PCMU
        Media Format: DynamicRTP-Type-101
Media Attribute (a): rtpmap:0 PCMU/8000
        Media Attribute Fieldname: rtpmap
        Media Format: 0
        MIME Type: PCMU
        Sample Rate: 8000
Media Attribute (a): rtpmap:101 telephone-event/8000
        Media Attribute Fieldname: rtpmap
        Media Format: 101
        MIME Type: telephone-event
        Sample Rate: 8000
Media Attribute (a): fmtp:101 0-16
        Media Attribute Fieldname: fmtp
        Media Format: 101 [telephone-event]
        Media format specific parameters: 0-16
Media Attribute (a): silenceSupp:off - - - -
        Media Attribute Fieldname: silenceSupp
        Media Attribute Value: off - - - -
Media Attribute (a): ptime:20
        Media Attribute Fieldname: ptime
        Media Attribute Value: 20

v=0  indicates the start of the SDP content.

o=FreeSWITCH 1532932581 1532932582 IN IP4 1.2.3.4 , is session origin and owner’s name

c=IN IP4 1.2.3.4 is connection data specifing the IP address of session.  

m= is Media type – audio, port – 29398, RTP/AVP Profile – 0 and 101

Attribute profile – 0, codec – PCMU, sampling rate – 8000 Hz and Attribute profile – 101, telephone-event

SIP transaction

A SIP transaction occurs between a UAC and a UAS in form of 1 request , its provisional and final response.

All transactions are independent of each other. Each transaction are uniquely identified by the branch id on the via header and the cseq.

Via: SIP/2.0/UDP <server ip>:5060;branch=z9hG4bKcb16.c47db56d6d8eb62677a0f0dc733cd73d.0
...
CSeq: 1 INVITE

SIP transaction consists of a single request and any responses to that request, which include zero or more provisional responses and one or more final responses.

A transaction consists of a Request, any non-final (1xx) Responses received, and a final Response (2xx, 3xx, 4xx, 5xx, or 6xx).
ACK is not considered part of this transaction and is a new transaction.
Request whose responses to that are non succesfull such as INVITE response with 100, 405 then,
ACK is part of the transaction.
Hence , ror positive replies (2XX), a new transaction is created for ACK with new CONTACT header and it can be sent straight to the UAS bypassing the proxy. For negative replies, ACK stays part of INVITE transaction hence request is sent to the same proxy as INVITE.

Examples

for ACK given below , tid=-d8754z-deea18278a05ce16-1—d8754z-

T 2017/06/06 06:56:03.656614 :37126 -> :5060 [AP]
 ACK sip:9876543210@:5080;transport=tcp SIP/2.0.
 Via: SIP/2.0/TCP :38834;branch=z9hG4bK-d8754z-deea18278a05ce16-1---d8754z-;rport.
 Max-Forwards: 70.
 To: :5080>;tag=fdc0b562c1d44395f53d16b622397a3f-589d.
 From: >;tag=b5327b03.
 Call-ID: MTllYjkyZjczMjhjM2I5OGE4MTgzZDUxODVjYmM0YzY.
 CSeq: 1 ACK.
 Content-Length: 0.

For CANCEL given below , tid=-d8754z-04665556a3f8c928-1—d8754z-

T 2017/06/06 06:53:09.643301 :37126 -> :5060 [AP]
 CANCEL sip:9876543210@:5080;transport=tcp SIP/2.0.
 Via: SIP/2.0/TCP :38834;branch=z9hG4bK-d8754z-04665556a3f8c928-1---d8754z-;rport.
 Max-Forwards: 70.
 To: :5080>.
 From: >;tag=c0869612.
 Call-ID: NTJhMGU1ZTA1NTAyZTYzZmUzMWQ0NjQ2MjIwYTE0MmI.
 CSeq: 1 CANCEL.
 User-Agent: Bria 3 release 3.5.5 stamp 71243.
 Content-Length: 0.

SIP entities that have notion of transactions are called stateful.

Branch

The branch parameter is a transaction identifier. Responses relating a request can be correlated because they will contain the same transaction identifier.

Dialog

The p2p relationship between 2 sip endpoints , containing sequence of transactions, is called the dialog . The initiator of the session that generates the establishing INVITE generates the unique Call-ID and From tag. In the response to the INVITE, the user agent answering the request will generate the To tag. The combination of the local tag (contained in the From header field), remote tag (contained in the To header field), and the Call-ID uniquely identifies the established session, known as a dialog. This dialog identifier is used by both parties to identify this call because there could be multiple calls set up between them.

  • A dialog is uniquely identified by: Call-ID header , remote-tag and local-tag.
  • DialogId is different for both ends since local and remote for both ends are different.

Example : Notice the to and from tag ids in INVITE and its 200 ok. The dialog id for invite is , 97576NjQ5MTBlNjVjNDQ0MzFmOTEyZGEzYWJjZjQxYjcyYzc70edc66c. Since it is the first INVITE, it doesnt bear the To tag.

INVITE sip:1234567890@ SIP/2.0
   Via: SIP/2.0/UDP :59583;branch=z9hG4bK-524287-1---22728813bce01a15;rport
   Max-Forwards: 70
   Contact: :59583>
   To: >
   From: >;tag=70edc66c
   Call-ID: 97576NjQ5MTBlNjVjNDQ0MzFmOTEyZGEzYWJjZjQxYjcyYzc
   CSeq: 1 INVITE
   Allow: OPTIONS, SUBSCRIBE, NOTIFY, INVITE, ACK, CANCEL, BYE, REFER, INFO
   Content-Type: application/sdp
   Supported: replaces
   User-Agent: X-Lite release 5.5.0 stamp 97576
   Content-Length: 210

   v=0
   o=- 1559804173873191 1 IN IP4 
   s=X-Lite release 5.5.0 stamp 97576
   c=IN IP4 
   t=0 0
   m=audio 49750 RTP/AVP 8 101
   a=rtpmap:101 telephone-event/8000
   a=fmtp:101 0-15
   a=sendrecv

The dialog id, with reversed to and from tag in 200 ok response 97576NjQ5MTBlNjVjNDQ0MzFmOTEyZGEzYWJjZjQxYjcyYzcStNBKgjjXS84r70edc66c

SIP/2.0 200 OK
   Via: SIP/2.0/UDP :59583;branch=z9hG4bK-524287-1---22728813bce01a15;rport=10973;received=
   From: >;tag=70edc66c
   To: >;tag=StNBKgjjXS84r
   Call-ID: 97576NjQ5MTBlNjVjNDQ0MzFmOTEyZGEzYWJjZjQxYjcyYzc
   CSeq: 1 INVITE
   Contact: :5060;transport=udp>
   User-Agent: FreeSWITCH-mod_sofia/1.9.0-742-8f1b7e0~64bit
   Accept: application/sdp
   Allow: INVITE, ACK, BYE, CANCEL, OPTIONS, MESSAGE, INFO, UPDATE, REGISTER, REFER, NOTIFY, PUBLISH, SUBSCRIBE
   Supported: timer, path, replaces
   Allow-Events: talk, hold, conference, presence, as-feature-event, dialog, line-seize, call-info, sla, include-session-description, presence.winfo, message-summary, refer
   Session-Expires: 120;refresher=uas
   Content-Type: application/sdp
   Content-Disposition: session
   Content-Length: 222
   Remote-Party-ID: "1234567890" >;party=calling;privacy=off;screen=no

   v=0
   o=FreeSWITCH 1559778909 1559778910 IN IP4 
   s=FreeSWITCH
   c=IN IP4 
   t=0 0
   m=audio 25266 RTP/AVP 8 101
   a=rtpmap:8 PCMA/8000
   a=rtpmap:101 telephone-event/8000
   a=fmtp:101 0-16
   a=ptime:20

Matching in-dialog transactions/requests

The combination of the To tag, From tag, and Call-ID completely defines a peer-to-peer SIP relationship between endpoints and is referred to as a dialog.

Record Routing

All requests sent within a dialog are by default sent directly from one user agent to the other. Only requests outside a dialog traverse SIP proxies. This approach makes SIP network more scalable because only a small number of SIP messages hit the proxies.

However few request need to explicitly state that they need to stay on path of proxies such as for accounting during termination of when NAT process is being carried out then. For these we need to insert a Record-Route header field into SIP messages which contain address of the proxy. Messages sent within a dialog will then traverse all SIP proxies that put a Record-Route header field into the message.

The server copies the Record-Route header field unchanged into the response. (Record-Route is only relevant for 2xx responses) i.e. the end point recipient will also mirror the proxies for the response.

record routingwithout Record Routingrecord routing (1)with record routing

Strict Routing

Rewrite the Request-URI ie Request-URI always contained URI of the next hop so it is necessary to save the original Request-URI as the last Route header field.  Defined in RFC2543.

Loose routing

Request-URI is no more overwritten, it always contains URI of the destination user agent, therby keeping target seprated from route. ( ;lr). If there are any Route header field in a message, then the message is sent to the URI from the topmost Route header field. Defined in RFC 326.

SIP Authorization

Authentication , security , confidentiality and integrity form the basic requirement for any communication system . To protect against hacking a user account and Denial of service attacks , SIP uses HTTP digest authentication mechanism with nonces and challenges along with 407 Proxy Authorization required and 401 unauthorised .  The sender has to resend the request with MD5 hash of nonce and password ( password id never send in clear ). Thus preventing man-in-middle attacks.

Challenge / Response Scheme :

  • Sends REGISTER   and receives 401 / 407 Challenge + nonce 
  • Again sends REGISTER + MD-5 hash (pw + nonce) get a 200 OK

REGISTER using HTTP Digest for authentication using TLS transport, challenge is in form

CSeq: 1 REGISTER
WWW-Authenticate: Digest realm="atlanta.example.com", qop="auth",
nonce="ea9c8e88df84f1cec4341ae6cbe5a359", opaque="", stale=FALSE, algorithm=MD5

Here qop is Quality Of Protection param indicating quality of protection that the client has applied to the message. qop=1 (enabled) will help you to avoid replay attacks.

Here qop is Quality Of Protection param indicating

challenge response by UA to UAS

Authorization: Digest username="bob", realm="atlanta.example.com"
nonce="ea9c8e88df84f1cec4341ae6cbe5a359", opaque="",
uri="sips:ss2.biloxi.example.com",
response="dfe56131d1958046689d83306477ecc"

Cancellation of Registration – UA sends REGISTER request with Expires: 0 Contact: * , to apply to all . Since user is already authenticated , it is not challenged again .

To prevent spoofing ie impersonating as server , SIP provides server authentication too. Required by ITSP’s  ( Internet telephony service providers ) .

End to end encryption is achieved thorough TS and SRTP.

More on SIP Security here .

Mobility and Location Service

According to RFC 3263 Session Initiation Protocol (SIP): Locating SIP Servers , if the proxy finds that the request is for an outside domain , it will take help of a DNS server to resolve to IP address of target domain and forward the request. Then target domain proxy used REGISTRAR’s discovery services to find if user is present in the host via location table entry . If found then request reaches the user .

To provide session mobility SIP endpoints send Register request to their respective registrar as they move and update their location. As User changes terminals , they registers themselves to the appropriate server
– Location server tracks the location of user
– Redirect servers prioritise the possible locations of the user
– Users keep same services as located at home server, while mobile
Call is processed by home servers using RECORD-ROUTE

NAT ( Network Address Translator)

Network Address Translator, defined by RFC 3022 to conserve network space as most packets are exchanged inside a private network itself.

All internet users whether they are using Wifi , 3G/LTE,  home AP, any other telecom data packet network  by TSP or ISP , are assigned a private IP address , which is unreachable from out side world .Addresses are assigned by Internet Assigned Numbers Authority (IANA). Private address blocks are in format of 10.0.0.0/8, 172.16.0.0/12, 192.168.0.0/16.

Therefore when they access the Internet , this address is converted into a  globally unique public IP address through a NAT for external communication

Screen Shot 2018-08-18 at 4.33.06 PM

SIP Issues around NAT

NATs modify IP addresses (Layer 3)- SIP/SDP are Layer 7 protocols – transparent to NAT

SIP Via:, From: and Contact: headers use not-routable private addresses
SDP states that originator wishes to receive media at not-routable private addresses
If destination on the public internet tries to send SIP or RTP traffic to those private address
Traffic will be dumped by first router

Solution are to use  either Application level gateway (ALG) or STUN or Universal Plug and Pray (UPnP)

To rewrite all SIP/SDP source addresses

  • SIP Via:, From: and Contact: headers use public NAT address
  • SDP addresses use NAT public address
  • Use SIP over TCP

Use draft-ietf-sip-symmetric-response-00 and “Symmetric” SIP/RTP
Use same UDP port number for incoming/outgoing
Hold ports open for call duration
Send UDP packet typically every 30 seconds
SIP over UDP uses 30 second re-INVITE, REGISTER or OPTIONs
RTP sends at much higher frequency by default

NAPT ( Network Address Port Translator )Can map multiple private IP addresses and ports to one public IP address and ports

To adapt SIP to modern IP networks with inter network traversal ICE, far and near-end NAT traversal solutions are used. Network Address traversal is crtical to traffic flow between private public network and from behind firewalls and policy controlled networks

Opensource STUN / TURN prooviders : google STUN server, mySTUN , TurnServer, reStund , CoTURN , NATH (PJSIP NAT Helper), ReTURN, or ice4j

Near-end NAT traversalFar-end NAT traversal
STUN (session traversal utilities for NAT) – UA itself detect presence of a NAT and learn the public IP address and port assigned using NAting. Then it replaces device local private IP address with it in the SIP and SDP headers. Implemented va STUN, TURN, and ICE.
(-) doesnt work for symmetric NAT (single connection has a different mapping with a different/randomly generated port)
(-) doesnt work when there are multiple addresses of a end point.

TURN (traversal using relay around NAT) or STUN relay – UA learns the public IP address of the TURN server and asks it to relay incoming packets.
(-) since it handled all incoming and outgong traffic , it must scale to meet traffic requirments and should not become the bottleneck junction or single point of failure.
UA is not concerned about NAT at all and communicated using its local IP port. The border controller implies a NAT handling compoenets such as an application layer gateway (ALG) or universal plug and play (UPnP) etc which resolves the private and public network address mapping by act as a back to back user agent (B2BUA).
ICE (interactive connectivity establishment) – UA gathers “candidates of communication” with priorities offered by the remote party. After this client pairs local candidates with received peer candidates and performs offer-answer negotiating by trying connectivity of all pairs, therefore maximising success. The types of candidates
– host candidate who represents clients’ IP addresses,
– server reflexive candidate for the address that has been resolved from STUN
– relayed candidate for the address which has been allocated from a TURN relay by the client.
Far end NAT can also be enabled by deploying a public SIP server which performs media relay (RTP Proxy/Media proxy).

(-) security risks : operating in public network enabling reverse traffic from UAS to UAC behind NAT.

A keep-alive mechanism is used to keep NAT translations of communications between SIP endpoint and its serving SIP servers opened , so that this NAT translation can be reused for routing. It contains client-to-server “ping” keep-alive and corresponding server-to-client “pong” messages. The 2 keep-alive mechanisms: a CRLF keep-alive and a STUN keep-alive message exchange.

SIP Flows

Screen Shot 2018-08-16 at 10.11.14 PM
Components of SIP based VoIP Solution

Registration

Localization Server  –Used by the Proxy Server and Redirect Server to obtain the location of the called user (one or more addresses)

Registration Server- Accept registration requests from the client applications . Generally, the service is offered by the Proxy Server or Redirect Server

DNS Server – Used to locate the Proxy Server or Redirect Server using NAPTR or SRV records

The 3 types of SIP URIs,

  • address of record (AOR)
  • fully qualified domain name (FQDN)
  • globally routable user agent (UA) URI
  • SIP uniform resource identifiers (URIs) are identified based on DNS resolution since the URI after @ symbol contains hostname , port and protocol for the next hop.

Adding record route headers for locating the correct SIP server for a SIP message can be done by :
DNS service record (DNS SRV)
naming authority pointer (NAPTR) DNS resource record

Steps for SIP endpoints locating SIP server

  1. From SIP packet get the NAPTR record to get the protocl to be used
  2. Inspect SRV record to fetch port to use
  3. Inspect A/AAA record to get IPv4 or IPv6 addresses
    ref : RFC 3263 – Locating SIP Servers
    Can use BIND9 server for DNS resolution supports NAPTR/SRV, ENUM, DNSSEC, multidomains, and private trees or public trees.

Screen Shot 2018-08-18 at 12.46.14 PM

Call Redirection

Sending Call invite but as Redirect Server responded with 302 moved temporary , a new destination address is returned. The invite is forwarded to another proxy server which connects the sip endpoints again after consultation with Redirect server .

Screen Shot 2018-08-18 at 10.37.38 AM

In this stage of we see the call getting connected to sip endpoint via 2 proxy servers . The redirect server doesnt get into path once the initial sip request is send.

Screen Shot 2018-08-18 at 11.12.17 AM

After communication the endpoints send BYE to terminate the session

Screen Shot 2018-08-18 at 11.13.59 AM

Forking

This callflow deals with the use-case when a user maybe registered from multiple SIP phones ( perhaps one home phone , one car and one office desk etc ) and wants to receive a ring on all registered phone ie fork a call to multiple endpoints .

Screen Shot 2018-08-18 at 11.17.19 AM

In the above diagram we can see a forked invite going to both the sip phones . Both of them reply with 100 trying and 180 ringing, but only 1 gets answered by the user .

Screen Shot 2018-08-18 at 11.17.26 AM

After one endpoint sends 200 ok and connects with session , the other receiver a cancel from the sip server .

Screen Shot 2018-08-18 at 11.17.33 AM

Click to Dial

A web or desktop application which has HTTP can fire a API call which is interpreted by the controller or SIP server  and call is fired.

Screen Shot 2018-08-18 at 1.23.36 PM

The API can contain params for to and from sip addresses as well as any authentication  token that is required for api authentication and validation .

Source code for some of the SIP application described above are on github at https://github.com/altanai/sip-servlets

SIMPLE (SIP for Instant Messaging and Presence Leveraging Extensions )

  • several vendors who intend to implement SIMPLE
  • provides for presence and buddy lists
  • Instant Messaging in the enterprise
  • telephony enabled user lists

Using SIP based Call routing algorithms and flows , one can build carrier grade communication solution . SIP solutions can hook up with existing telecom networks and service providers to be backward compatible . Also has untapped unlimited potential to integrate with any external IP application or service to provide converged , customised control both for signalling and media planes.

References :

  1. SIP servlets samples : https://github.com/altanai/sip-servlets
  2. SIP by Henning Schulzrinne Dept. of Computer Science Columbia University New York
  3. International Institute of Telecommunications 2000-2004
  4. Introduction to SIP by Patrick Ferriter from ZULTYS
  5. Internet Draft, IETF, RFC 2543
  6. NTU – Internet Telephony based on SIP

RFC 3665 – Session Initiation Protocol (SIP) Basic Call Flow Examples

It contains SIP implementation examples such as

  • SIP Registration – Successful New Registration, Update of Contact List, Request for Current Contact List, Cancellation of Registration, Unsuccessful Registration
  • SIP Session Establishment
  • Session Establishment Through Two Proxies,
  • Session with Multiple Proxy Authentication,
  • Successful Session with Proxy Failure,
  • Session Through a SIP ALG,
  • Session via Redirect and Proxy Servers with SDP in ACK,
  • Session with re-INVITE (IP Address Change),
  • Unsuccessful No Answer, Unsuccessful Busy, Unsuccessful No Response from User Agent, Unsuccessful Temporarily Unavailable,
  • Security Considerations

RFC 5359 – Session Initiation Protocol Service Examples

It contains the description for services like

  • Call Hold, Consultation Hold, Music on Hold,
  • Transfer – Unattended, Transfer – Attended, Transfer – Instant Messaging,
  • Call Forwarding Unconditional, Call Forwarding – Busy, Call Forwarding – No Answer,
  • 3-Way Conference – Third Party Is Added, 3-Way Conference – Third Party Joins, Find-Me ,
  • Call Management (Incoming Call Screening) , Call Management (Outgoing Call Screening) ,
  • Call Park , Call Pickup , Automatic Redial , Click to Dial.

Telecommunications convergence – VoIP, PBX and IMS


There has been rapid evolution of telecom platform over the last few decades. Starting from the the mobile phone network-enabled universal communication agnostic to actual location to present day high bandwidth high data rate entertainment/ streaming like applications. The affordable, personal communication system has converged to enterperise level secure communication systems that cater to low latency and highly secure end to end encrypted scenarios.

IP communication

Some of the positive aspects of using IP communication over traditional communication systems are :

Higher ROI( Return of investment)

ROI is a big factor for SME before making the switch to IP telephony inplace of traditional established system like landline phone and cables. However it is for a fact that once the VOIP comm system is setup , it most certainly reduces call costs by 70%.

Third party Interations

It is often a necessaity to integrate communication system with CRM (content realationship management ) systems or Sales management systems or other lead gtracking systesm which are driven from communications with possible clients or investors ( called leads). Since most web portals are on IP protocl as HTTP, VOIP fits very well, with the click to call on webpage itself among other features such as directory integration , notofoication , call scripts etc.

VAS ( Value Added Service)

Value Added Services , refer to services build on top of existing underlying mobile communication call and sms. These could be innovation usecases build using -IVR / DTMF such as cricket score, astrology updates or call recoring , find-me-follow-me applicatoion for multiple devices , voicemail/ visual voice mail , re-routing to home phone or assiatnt phone, called ID etc. In short it can add intelligence to the way calls are managed .

As bandwidth has increased, so has the proliferation of VoIP systems. From the user’s perspective, modern mobile devices deliver the converged, multi-media communication and entertainment experience.

VOIP

VOIP , short for Voice over IP , is called so beacuse it not only converts your voice calls in analog voice into digital packets but also channels voice data through IP networks such as LAN , WAN , Internet etc using the Internet Protocol (IP) .

  • VOIP system on LAN ( Local Area Network ) can use it as its backbone system to establish communication between endpoints . For example : Office communication system within the same enterprise/building.
  • Similarity  VOIP over WAN ( Wide Area Network ) use the help  of IP PBX and VoIP service provider to enable communication across Internet . For example : OTT providers and internet calls.
  • By using the services of telecom providers in support with above plan it is also possible to land a VOIP call onto a real phone over GSM / PSTN via gateways.

For a provider of IP telephony system, number of factors come into picture such as :

  1. Bandwidth : Low bandwidth has always been a big concern for IP calls especially due to packet loss and thus high noise. While a LAN connection ensures good experience, calls over internet or VOIP PBX are not necessarily as neat. Network switching between different Internet service providers causes congestion and lags too.
  2. Inter-operability : Connecting remote works / employees to the VOIP network requires interoperablity between their hand held device like android , ios , tablets , smart watch or other types od communication devices such as hardphone, desktop-systems , kiosk , surveillance cams etc is a challenging considering the underlying OS and networking support.
  3. Traffic: Maximum simultaneous call or peak traffic rate can create bottlenecks in communication channel or worse still result in high bandwidth usage. For example as p2p conf call between 5 parties will create a mesh network between each participant resulting in 4 outgoing and 4 incoming channels.
  4. QoS (Quality of service ) :
    • Call drops ,
    • prioritization of important calls ,
    • Security preventing the attacks and hacks ,
    • keeping information secure by encryption end to end
  5. AAA : managing Authentication, Authorization and accounting
  6. Reuse existing Hardware :
    • Replacing old hardware or installing softphone apps on mobiles etc .
    • Reuse old servers . Manage setup between datacentres and cloud deployments
    • Administravtive hurdles between different counteries and geographies for using hardware
  7. Scaling
    • How quickly can it scale up or scale down ?
    • Will the communication system grow horizontly or vertically ?
    • How to ensure that the growing system can accommodate new users , physical office location , remote centers , call centres etc ?
  8. Codecs : Under low bandwidth condition it is a good idea to switch to low resolution ( in case of video ) and low bandwidth codec ( in case of audio ) .

Other factors such as privacy , accounability , Lawful interception ( legal requirnments in many enterprises ) , Auditing , SLA ( Service Level Agreements) to ensure the system is up 99.99 % of time and agrreeing to pay compensation if system is down for longer duration than 0.01 % of time so on.

Hosting the PBX

Unified communication Solutions as SaaS or IaaS refer to on-premise or cloud-hosted IP PBX Solutions. Comparison of both is as follows

On -premiseCloud Based
The solution is usually of the SaaS nature ( software as a service ) which is hosted by the consumer / business unit itself . The service provider offers his infrastructure to the consumer as a service and bills monthly / yearly etc .
Hosting the solution system on premise and setting up the infrastructure means more customization and flexibility but it also means more investment and maintenance . On the other hand hosting the solution on cloud is often a quick setup with relatively lower upfront payment. The billing is either carried out per per user basis or based on consumption . The data is synced to cloud servers for storage and can be fetched from there when required such as cloud synced Call-logs or contact-book .

Convergence Vision 

We already know some of the latest trends of industry with respect to telecom convergence such as :

FMC

Fixed Mobile Convergence (FMC) stands for integrating user’s fixed desk phone with his mobile phone. Call continuity is a VAS( Value added service ) which lets him to switch calls between different call devices even softphones , mid call also. It has multi-faced advantages such as not missing any call on account of being out of office , having the same call preferences on each device such as blocked numbers , IVR settings etc .

UC

Unified Communication(UC) refers to the accessibility of all communication and collaboration services from the users call agent ( phone / soft-phone ) . These services can include file transfer , chat , conference , call settings , blocking , white-listing , fax , cloud sync , call logs , called ID , favorites , recording .
Read more about Unified communication and collaboration here .

BYOD

Bring your own device (BYOD) is one of the hottest trends in industry almost across all domains where user is expected or is given to option to bring his personal laptop for official use . It is the responsibility of enterprise comm system to seamlessly integrate it with in-office communication system and provide the same privileges and security to business critical applications as preset in configuration settings . It increases the flexibility and productivity while keeping the infrastructure cost down.

IMS provided Network Interoperability and Access Independence

ims-access-network-independence
Image Source unknown. Represents the convergence of IMS subsystem with various access types

IMS based tele-coommunication convergence described in figure below

  • clients get direct connectivity to IP PBX in offices or hotels
  • home users connect through cable wires or Wifi/WiMax
  • non SIP based legacy endpoints connect via signalling and media gateways

The access endpoints connecte to a single managed core IP network which intercoonectes with IMS core . The back end system not only manages calls and sessions but also registration  ,  billing , operations and adminstartion.

IMS convergence vision
picture courtesy – unknown

 Intelligent Network   —>    Next Generation IMS System 

The signalling protocols migration like from signalling system 7 (SS7) to session initial protocol (SIP) have been taking place in Telco-Industry. Similarly nodes of legacy network like signal transfer point (STP) of legacy network are being migrated to call session control function (CSCF) of IMS  that allows the rapid development and deployment of enhanced, revenue-generating multimedia services for fixed, mobile and cable operators.

IMS architecture enables operators to seamlessly run a plethora of next-generation converged services over their fixed, mobile and cable networks, achieve a faster time-to-market for new services and have fewer performance bottlenecks.

converged telecommunications

Business benefits of IMS 

  1. Delivering Services: Delivering services and applications on a “wherever, however, whenever” basis.
  2. Multimedia services: Enabling service providers to offer multimedia services across both next-gen, packet-switched networks and traditional circuit-switched networks.
  3. Protocol stack: IMS architecture provides pipes and protocols onto which service providers can attach no. of applications very conveniently.
  4. Open Source standard: IMS architecture is based on open standard which makes it possible for different vendors of hardware and software to integrate with each other seamlessly.

As a subscriber, one of the main benefits of the IMS architecture is the capacity of the network to deliver the same set of services whatever the access network used.

convergence

This is made possible thanks to the centralization of the service execution process. A specific call server of the control plan (called Serving Call Session Control Function, S-CSCF) is responsible for invoking the application servers based on criteria provisioned in the central database. The S-CSCF gets these criteria (called Initial Filter Criteria) during the user’s registration in the IMS network.

Circuit Switched Voice –> Packet based VOIP 

Voice over IP revolutionized in the Telecommunication space.It also makes your communication experience much richer and nicer with a series of enhanced features and extended possibilities. The no. of user migrating from traditional circuit switched network to IP has been quite substantial in recent years. CSP are embracing VOIP technology as a potential revenue generator and investing huge chunk of money to create value propositions for themselves in VOIP.

In conclusion here are the top business benefits of adopting a converged and unified IP telephony solution such as IMS and SIP are

  • Cost Savings : Saving money is the number-one reason most businesses and households make the switch to a VoIP system, VoIP systems don’t require a phone cabinet or on-site routing equipment- just phones.
  • Features: VoIP also allows users to take advantage of advanced features only available on internet-based phone systems. Features like online call monitoring, and online phone system access to add or configure extensions are also available with VoIP systems.
  • Flexibility: VoIP allows people to go mobile and call directly from their cell phone and be charged at low VoIP rates
  • Tracking Options: Since VoIP is an internet-based system, user can track and manage their system from their computer. Most VoIP systems allow user to track call volume and call time fairly easily- a feature that can be especially helpful for businesses that bill clients hourly or for time spent on the phone.