EEP duplicates and IP datagram and encapsulates and sends for remote relatime monitoring for SIP specific alerts and notifications . HEP is popular among many SIP servers including Freeswitch , Opensips, Kamailio, RTP engine as an external module .
- intended for passive duplicated for remote collection
- can be used for audit storage and analysis
- does not alter the orignal datagram or headers
HOMER is Packet and Event capture system popular fpr VOIP/RTC Monitoring based on HEP/EEP (Extensible Encapsulation protocol)
SIP Server Integration
Homer and homer encapsulation protocl (HEP) integration with sip server brings the capabilities to SIP/SDP payload retention with precise timestamping better monitor and detect anomilies in call tarffic and events correlation of session ,logs , reports also the power to bring charts and statictics for SIP and RTP/RTCP packets etc. We read about sipcapture and sip trace modules in project sipcapture_siptrace_hep.
Both Kamailio and Opensips HEP Integration are structurally simmilar. In kamailio SIPCAPTURE [2] module enables support for –
● Monitoring/mirroring port
● IPIP encapsulation (ETHHDR+IPHDR+IPHDR+UDPHDR)
● HEP encapsulation protocol mode (HEP v1, v2, v3)
Figure Opensips Capturing ( credits http://www.opensips.org)
Figure showing Opensips integartion with external capturing agent via proxy agent ( which can be HOMER)
To achieve that, load and configure the SipCapture module in the routing script.
Snippets fro Kamailio Homer docker installation as a collector
git clone https://github.com/sipcapture/homer-docker.git cd homer-docker docker-compose build docker-compose up
Outsnippets from screen while the installation takes place
Creating network "homer-docker_default" with the default driver Creating volume "homer-docker_homer-data-semaphore" with default driver Creating volume "homer-docker_homer-data-mysql" with default driver Creating volume "homer-docker_homer-data-dashboard" with default driver Pulling mysql (mysql:5.6)... 5.6: Pulling from library/mysql ... Creating mysql ... done Creating homer-webapp ... done Creating homer-cron ... done Creating homer-kamailio ... done Creating bootstrap-mysql ... done Attaching to mysql, homer-webapp, bootstrap-mysql, homer-cron, homer-kamailio .... homer-webapp | Homer web app, waiting for MySQL homer-cron | Homer cron container, waiting for MySQL homer-kamailio | Kamailio, waiting for MySQL bootstrap-mysql | Mysql is now running. bootstrap-mysql | Beginning initial data load.... bootstrap-mysql | Creating Databases... bootstrap-mysql | Creating Tables... ..... omer-kamailio | Kamailio container detected MySQL is running & bootstrapped homer-kamailio | 0(22) INFO: <core> [core/sctp_core.c:75]: sctp_core_check_support(): SCTP API not enabled - if you want to use it, load sctp module homer-kamailio | 0(22) WARNING: <core> [core/socket_info.c:1315]: fix_hostname(): could not rev. resolve 0.0.0.0 homer-kamailio | config file ok, exiting... homer-kamailio | loading modules under config path: //usr/lib/x86_64-linux-gnu/kamailio/modules/ homer-kamailio | Listening on homer-kamailio | udp: 0.0.0.0:9060 homer-kamailio | Aliases: homer-kamailio | homer-kamailio | 0(23) INFO: <core> [core/sctp_core.c:75]: sctp_core_check_support(): SCTP API not enabled - if you want to use it, load sctp module homer-kamailio | 0(23) WARNING: <core> [core/socket_info.c:1315]: fix_hostname(): could not rev. resolve 0.0.0.0 homer-kamailio | loading modules under config path: //usr/lib/x86_64-linux-gnu/kamailio/modules/ homer-kamailio | Listening on homer-kamailio | udp: 0.0.0.0:9060 homer-kamailio | Aliases: homer-kamailio | homer-kamailio | 0(23) INFO: sipcapture [sipcapture.c:480]: parse_table_names(): INFO: table name:sip_capture ... homer-webapp | Homer web app container detected MySQL is running & bootstrapped homer-webapp | Module php5 already enabled
Capture tools
Dialoge module
storing dialogs in mysql DB , requires initialising mysql
#!define WITH_MYSQL
...
#!ifdef WITH_MYSQL
loadmodule "db_mysql.so"
#!endif
...
#!ifdef WITH_MYSQL
# - database URL - used to connect to database server by modules such
# as: auth_db, acc, usrloc, a.s.o.
#!ifndef DBURL
#!define DBURL "mysql://root:kamailio@localhost/kamailio"
#!endif
#!endif
loadmodule "dialog.so"
# ----- dialog params ------
modparam("dialog", "dlg_flag", 10)
modparam("dialog", "track_cseq_updates", 0)
modparam("dialog", "dlg_match_mode", 2)
modparam("dialog", "timeout_avp", "$avp(i:10)")
modparam("dialog", "enable_stats", 1)
modparam("dialog", "db_url", DBURL)
modparam("dialog", "db_mode", 1)
modparam("dialog", "db_update_period", 120)
modparam("dialog", "table_name", "dialog")
seting db_mode – synchronisation of dialog information from memory to an underlying database has following options
0 – NO_DB – the memory content is not flushed into DB;
1 – REALTIME – any dialog information changes will be reflected into the database immediately.
2 – DELAYED – the dialog information changes will be flushed into DB periodically, based on a timer routine.
3 – SHUTDOWN – the dialog information will be flushed into DB only at shutdown – no runtime updates.
note :
- use the same hash_size while using diff kamailio to restore dialogs
database table for dialogue
- install mysql
- define root ( with db create permissions ) and user ( with database read wrote ) permission in kamctlrc
vi /usr/local/etc/kamailio/kamctlrc
- Dialogue table schema *
name type size default null key extra attributes description
id unsigned int 10 no primary autoincrement unique ID
hash_entry unsigned int 10 no Number of the hash entry in the dialog hash table
hash_id unsigned int 10 no The ID on the hash entry
callid string 255 no Call-ID of the dialog
from_uri string 128 no URI of the FROM header (as per INVITE)
from_tag string 64 no identify a dialog, which is the combination of the Call-ID along with two tags, one from participant in the dialog.
to_uri string 128 no URI of the TO header (as per INVITE)
to_tag string 64 no identify a dialog, which is the combination of the Call-ID along with two tags, one from participant in the dialog.
caller_cseq string 20 no Last Cseq number on the caller side.
callee_cseq string 20 no Last Cseq number on the caller side.
caller_route_set string 512 yes Route set on the caller side.
callee_route_set string 512 yes Route set on on the caller side.
caller_contact string 128 no Caller's contact uri.
callee_contact string 128 no Callee's contact uri.
caller_sock string 64 no Local socket used to communicate with caller
callee_sock string 64 no Local socket used to communicate with callee
state unsigned int 10 no The state of the dialog.
start_time unsigned int 10 no The timestamp (unix time) when the dialog was confirmed.
timeout unsigned int 10 0 no The timestamp (unix time) when the dialog will expire.
sflags unsigned int 10 0 no The flags to set for dialog and accesible from config file.
iflags unsigned int 10 0 no The internal flags for dialog.
toroute_name string 32 yes The name of route to be executed at dialog timeout.
req_uri string 128 no The URI of initial request in dialog
xdata string 512 yes Extra data associated to the dialog (e.g., serialized profiles).
Siptrace module
SIPtrace module offer a possibility to store incoming and outgoing SIP messages in a database and/or duplicate to the capturing server (using HEP, the Homer encapsulation protocol, or plain SIP mode).
loadmodule "siptrace.so"
modparam("siptrace", "duplicate_uri", "sip:127.0.0.1:9060")
modparam("siptrace", "hep_mode_on", 1)
modparam("siptrace", "trace_to_database", 0)
modparam("siptrace", "trace_flag", 22)
modparam("siptrace", "trace_on", 1)
integrating iut with request route to start duplicating the sip messages
sip_trace();
setflag(22);
- trace_mode * 1 – uses core events triggered when receiving or sending SIP traffic to mirror traffic to a SIP capture server using HEP 0 – no automatic mirroring of SIP traffic via HEP.
duplicate
address in form of a SIP URI where to send a duplicate of traced message. It uses UDP all the time.
modparam("siptrace", "duplicate_uri", "sip:127.0.0.1:9060")
to check the duplicate messages arriving
ngrep -W byline -d any port 9060 -q
RPC commands
Can ruen sip trace on or off
kamcmd> siptrace.status on
Enabled
and to check
kamcmd> siptrace.status check
Enabled
Store sip_trace in database
modparam("siptrace", "trace_to_database", 1)
modparam("siptrace", "db_url", DBURL)
modparam("siptrace", "table", "sip_trace")
where the sip_trace tabel description is
+-------------+------------------+------+-----+---------------------+----------------+
| Field | Type | Null | Key | Default | Extra |
+-------------+------------------+------+-----+---------------------+----------------+
| id | int(10) unsigned | NO | PRI | NULL | auto_increment |
| time_stamp | datetime | NO | MUL | 2000-01-01 00:00:01 | |
| time_us | int(10) unsigned | NO | | 0 | |
| callid | varchar(255) | NO | MUL | | |
| traced_user | varchar(128) | NO | MUL | | |
| msg | mediumtext | NO | | NULL | |
| method | varchar(50) | NO | | | |
| status | varchar(128) | NO | | | |
| fromip | varchar(50) | NO | MUL | | |
| toip | varchar(50) | NO | | | |
| fromtag | varchar(64) | NO | | | |
| totag | varchar(64) | NO | | | |
| direction | varchar(4) | NO | | | |
+-------------+------------------+------+-----+---------------------+----------------+
sample databse storage for sip traces
select * from sip_trace;
| id | time_stamp | time_us | callid | traced_user | msg | method | status | fromip | toip | fromtag | totag | direction |
+----+---------------------+---------+---------------------------------------------+-------------+-----------------------------------
| 1 | 2019-07-18 09:00:18 | 417484 | MTlhY2VmNDdjN2QxZGM5ZDFhMWRhZThhZDU4YjE0MGM | | INVITE sip:altanai@sip_addr;transport=udp SIP/2.0
Via: SIP/2.0/UDP local_addr:25584;branch=z9hG4bK-d8754z-1f5a337092a84122-1---d8754z-;rport
Max-Forwards: 70
Contact: <sip:derek@call_addr:7086;transport=udp>
To: <sip:altanai@sip_addr>
From: <sip:derek@sip_addr>;tag=de523549
Call-ID: MTlhY2VmNDdjN2QxZGM5ZDFhMWRhZThhZDU4YjE0MGM
CSeq: 1 INVITE
Allow: INVITE, ACK, CANCEL, OPTIONS, BYE, REFER, NOTIFY, MESSAGE, SUBSCRIBE, INFO
Content-Type: application/sdp
Supported: replaces
User-Agent: Bria 3 release 3.5.5 stamp 71243
Content-Length: 214
v=0
o=- 1563440415743829 1 IN IP4 local_addr
s=Bria 3 release 3.5.5 stamp 71243
c=IN IP4 local_addr
t=0 0
m=audio 59814 RTP/AVP 9 8 0 101
a=rtpmap:101 telephone-event/8000
a=fmtp:101 0-15
a=sendrecv | INVITE | | udp:caller_addr:27982 | udp:sip_pvt_addr:5060 | de523549 | | in |
| 2 | 2019-07-18 09:00:18 | 421675 | MTlhY2VmNDdjN2QxZGM5ZDFhMWRhZThhZDU4YjE0MGM | | SIP/2.0 100 trying -- your call is important to us
Via: SIP/2.0/UDP local_addr:25584;branch=z9hG4bK-d8754z-1f5a337092a84122-1---d8754z-;rport=27982;received=caller_addr
To: <sip:altanai@sip_addr>
From: <sip:derek@sip_addr>;tag=de523549
Call-ID: MTlhY2VmNDdjN2QxZGM5ZDFhMWRhZThhZDU4YjE0MGM
CSeq: 1 INVITE
Server: kamailio (5.2.3 (x86_64/linux))
Content-Length: 0 | ACK | | udp:caller_addr:27982 | udp:local_addr:5060 | de523549 | b2d8ad3f | in |
...
+----+---------------------+---------+---------------------------------------------+-------------+-----------------------------------
Heplify
Multi-Protocol Go HEP Capture Agent made https://github.com/sipcapture/heplify
wget https://dl.google.com/go/go1.11.2.linux-amd64.tar.gz
sudo tar -C /usr/local -xzf go1.11.2.linux-amd64.tar.gz
move package to /usr/local/go
mv go
Either add go bin to ~/.profile
export PATH=$PATH:/usr/local/go/bin
and apply
source ~/.profile
or set GO ROOT , and GOPATH
export GOROOT=/usr/local/go
export GOPATH=$HOME/heplify
export PATH=$GOPATH/bin:$GOROOT/bin:$PATH
installation of dependencies
go get
clone heplify repo and make
make
CAPTAGENT
New OSS Capture-Agent framework with capture suitable for SIP, XMPP and more. With internal method filtering , encryption and authetication this does look very promising howevr since I have perosnally not tried it yet , I will leave this space TBD for future
sngrep
https://github.com/irontec/sngrep
Other include Sipgrep , HEPipe and nProbe
HEPop
Multi-Protocol HEP Server & Switch in NodeJS. stand-alone HEP Capture Server designed for HOMER7 capable of emitting indexed datasets and tagged timeseries to multiple backends
https://github.com/sipcapture/HEPop
node hepop.js -c /app/myconfig.js
PCAP monitoring -> Homer Server -> Notification and Fraud Prevention
A realtime monitoring and alerting setup fom homer can best safeguard on VoIP specific attacks and suspecious activity by early warning . Some list of attacks such as DDOS , SIP SQL injections , parser , remote manipulation hijacking as cell as resource enumeration are common ifor a cloud telephony provider.
Adiitionally homer provide session quality using varables that include [1]
SD = Session Defects
[SUM(500,503,504)]
ISA = Ineffective Session Attempts
[SUM(408,500,503)]
AHR = Average HOP Requests
ASR = Answer Seizure Ratio
[(‘200’ / (INVITES – AUTH – SUM(3XX))) * 100]
NER = Network Efficiency Ratio
[(‘200’ + (‘486′,’487′,’603’) / (INVITES -AUTH-(SUM(30x)) * 100]
HOMER Web Interface or Custom Dashboard
Some more visualization for inter team communication such as NOC team can include
Homer Integration with influx DB
time series Reltiem DB install
wget https://dl.influxdata.com/influxdb/releases/influxdb_1.7.7_amd64.deb
sudo dpkg -i influxdb_1.7.7_amd64.deb
start
>influxd
8888888 .d888 888 8888888b. 888888b.
888 d88P" 888 888 "Y88b 888 "88b
888 888 888 888 888 888 .88P
888 88888b. 888888 888 888 888 888 888 888 888 8888888K.
888 888 "88b 888 888 888 888 Y8bd8P' 888 888 888 "Y88b
888 888 888 888 888 888 888 X88K 888 888 888 888
888 888 888 888 888 Y88b 888 .d8""8b. 888 .d88P 888 d88P
8888888 888 888 888 888 "Y88888 888 888 8888888P" 8888888P"
2019-07-19T07:03:04.603494Z info InfluxDB starting {"log_id": "0GjGVvbW000", "version": "1.7.7", "branch": "1.7", "commit": "f8fdf652f348fc9980997fe1c972e2b79ddd13b0"}
2019-07-19T07:03:04.603756Z info Go runtime {"log_id": "0GjGVvbW000", "version": "go1.11", "maxprocs": 1}
2019-07-19T07:03:04.707567Z info Using data dir {"log_id": "0GjGVvbW000", "service": "store", "path": "/var/lib/influxdb/data"}
For Kamailio integration follow github instructions on https://github.com/altanai/kamailioexamples
References :
[1] https://www.kamailio.org/events/2013-KamailioWorld/13-Alexandr.Dubovikov-Homer-SIP-Capture.pdf
[2] HEP/EEP – https://github.com/sipcapture/hep
[3] kamailio sipdump module – https://www.kamailio.org/docs/modules/devel/modules/sipdump.html
[4] https://github.com/sipcapture/HEPop
[5] HOMER Big Data – https://github.com/sipcapture/homer/wiki/Homer-Bigdata
Telecom R & D 