GStreamer ( LGPL )ia a media handling library written in C for applicatioan such as streaming , recording, playback , mixing and editing attributes etc. Even enhnaced applicaiosn such as tsrancoding , media ormat conversion , streaming servers for embeeded devices ( read more about Gstreamer in RPi in my srticle here).
It encompases various codecs, filters and is modular with plugins developement to enhance its capabilities. Media Streaming application developers use it as part of their framework at either the broadcaster’s end or as media player.
gst-launch-1.0 videotestsrc ! videoconvert ! autovideosink
More detailed reading :
GStreamer-1.8.1 rtsp server and client on ubuntu – Install and configuration for a RTSP Streaming server and Client https://telecom.altanai.com/2016/05/20/gstreamer-1-8-1-rtsp-server-and-client-on-ubuntu/
crtmpserver + ffmpeg –
https://telecom.altanai.com/2016/06/19/crtmpserver-ffmpeg
Streaming / broadcasting Live Video call to non webrtc supported browsers and media players
attempts of streaming / broadcasting Live Video WebRTC call to non WebRTC supported browsers and media players such as VLC , ffplay , default video player in Linux etc .
continue : Streaming / broadcasting Live Video call to non webrtc supported browsers and media players
httontinuation to the attempts / outcomes and problems in building a WebRTC to RTP media framework that successfully stream / broadcast WebRTC content to non webrtc supported browsers ( safari / IE ) / media players ( VLC )
TO continue with basics of gstreamer keep reading
To list all packages of Gstreamer
pkg-config --list-all | grep gstreamer
- gstreamer-gl-1.0 GStreamer OpenGL Plugins Libraries – Streaming media framework, OpenGL plugins libraries
- gstreamer-bad-video-1.0GStreamer bad video library – Bad video library for GStreamer elements
- gstreamer-tag-1.0 GStreamer Tag Library – Tag base classes and helper functions
- gstreamer-bad-base-1.0 GStreamer bad base classes – Bad base classes for GStreamer elements
- gstreamer-net-1.0GStreamer networking library – Network-enabled GStreamer plug-ins and clocking
- gstreamer-sdp-1.0 GStreamer SDP Library – SDP helper functions
- gstreamer-1.0 GStreamer – Streaming media framework
- gstreamer-bad-audio-1.0 GStreamer bad audio library, uninstalled – Bad audio library for GStreamer elements, Not Installedgstreamer-allocators-1.0 GStreamer Allocators Library – Allocators implementation
- gstreamer-player-1.0 GStreamer Player – GStreamer Player convenience library
- gstreamer-insertbin-1.0 GStreamer Insert Bin – Bin to automatically and insertally link elements
- gstreamer-plugins-base-1.0 GStreamer Base Plugins Libraries – Streaming media framework, base plugins libraries
- gstreamer-vaapi-glx-1.0 GStreamer VA-API (GLX) Plugins Libraries – Streaming media framework, VA-API (GLX) plugins librariesgstreamer-codecparsers-1.0 GStreamer codec parsers – Bitstream parsers for GStreamer elementsgstreamer-base-1.0 GStreamer base classes – Base classes for GStreamer elements
- gstreamer-app-1.0 GStreamer Application Library – Helper functions and base classes for application integration
- gstreamer-vaapi-drm-1.0 GStreamer VA-API (DRM) Plugins Libraries – Streaming media framework, VA-API (DRM) plugins librariesgstreamer-check-1.0 GStreamer check unit testing – Unit testing helper library for GStreamer modules
- gstreamer-vaapi-1.0 GStreamer VA-API Plugins Libraries – Streaming media framework, VA-API plugins libraries
- gstreamer-controller-1.0 GStreamer controller – Dynamic parameter control for GStreamer elements
- gstreamer-video-1.0 GStreamer Video Library – Video base classes and helper functions
- gstreamer-vaapi-wayland-1.0 GStreamer VA-API (Wayland) Plugins Libraries – Streaming media framework, VA-API (Wayland) plugins libraries
- gstreamer-fft-1.0 GStreamer FFT Library – FFT implementation
- gstreamer-mpegts-1.0 GStreamer MPEG-TS – GStreamer MPEG-TS support
- gstreamer-pbutils-1.0 GStreamer Base Utils Library – General utility functions
- gstreamer-vaapi-x11-1.0 GStreamer VA-API (X11) Plugins Libraries – Streaming media framework, VA-API (X11) plugins libraries
- gstreamer-rtp-1.0 GStreamer RTP Library – RTP base classes and helper functions
- gstreamer-rtsp-1.0 GStreamer RTSP Library – RTSP base classes and helper functions
- gstreamer-riff-1.0 GStreamer RIFF Library – RIFF helper functions
- gstreamer-audio-1.0 GStreamer Audio library – Audio helper functions and base classes
- gstreamer-plugins-bad-1.0 GStreamer Bad Plugin libraries – Streaming media framework, bad plugins libraries
- gstreamer-rtsp-server-1.0 gst-rtsp-server – GStreamer based RTSP server
At the time of writing this article Gstreamer an much early version in 1.X , which was newer than its then stable version 0.x. Since then the library has updated many fold. summarising release highlights for major versions as the blog was updated over time .
Project : Making and IP survillance system using gstreamer and Janus
To build a turn-key easily deployable surveillance solution
Features :
- Paring of Android Mobile with box
- Live streaming from Box to Android
- Video Recording inside the box
- Auto parsing of recorded video around motion detection
- Event listeners
- 2 way audio
- Inbuild Media Control Unit
- Efficient use of bandwidth
- Secure session while live-streaming
Modules
- Authentication ( OTP / username- password)
- Livestreaming on Opus / vp8
- Session Security and keepalives for live-streaming sessions
- Sync local videos to cloud storage
- Record and playback with timeline and events
- Parsing and restructuring video ( transcoding may also be required )
- Coturn server for NAT and ICE
- Web platform on box ( user interface )+ NoSQL
- Web platform on Cloud server ( Admin interface )+ NoSQL
- REST APIs for third party add-ons ( Node based )
- Android demo app for receiving the live stream and feeds
Varrying experiments and working gstreamer commands
Local Network Stream
To create /dev/video0
modprobe bcm2835-v4l2
To stream on rtspserver using rpicamsrc using h264 parse
./gst-rtsp-server-1.4.4/examples/test-launch --gst-debug=2 '(rpicamsrc num-buffers=5000 ! 'video/x-h264,width=1080,height=720,framerate=30/1' ! h264parse ! rtph264pay name=pay0 pt=96 )'
./test-launch “( tcpclientsrc host=127.0.0.1 port=5000 ! gdpdepay ! rtph264pay name=pay0 pt=96 )”
pipe raspivid to tcpserversink
raspivid -t 0 -w 800 -h 600 -fps 25 -g 5 -b 4000000 -vf -n -o - | gst-launch-1.0 -v fdsrc ! h264parse ! gdppay ! tcpserversink host=127.0.0.1 port=5000;
Stream Video over local Network with 15 fps
raspivid -n -ih -t 0 -rot 0 -w 1280 -h 720 -fps 15 -b 1000000 -o - | nc -l -p 5001
streaming video over local network with 30FPS and higher bitrate
raspivid -n -t 0 -rot 0 -w 1920 -h 1080 -fps 30 -b 5000000 -o - | nc -l -p 5001
Recording
Audio record to file
Using arecord :
arecord -D plughw:1 -c1 -r 48000 -f S16_LE -t wav -v file.wav;
Using pulse :
pulseAudio -D
gst-launch-1.0 -v pulsesrc device=hw:1 volume=8.0 ! audio/x-raw,format=S16LE ! audioconvert ! voaacenc bitrate=48000 ! aacparse ! flvmux ! filesink location = "testaudio.flv";
Video record to file ( mpg)
gst-launch-1.0 -e rpicamsrc bitrate=500000 ! 'video/x-h264,width=640,height=480’ ! mux. avimux name=mux ! filesink location=testvideo2.mpg;
Video record to file ( flv )
gst-launch-1.0 -e rpicamsrc bitrate=500000 ! video/x-h264,width=320,height=240,framerate=10/1 ! h264parse ! flvmux ! filesink location="testvieo.flv";
Video record to file ( h264)
gst-launch-1.0 -e rpicamsrc bitrate=500000 ! filesink location=”raw3.h264″;
Video record to file ( mp4)
gst-launch-1.0 -e rpicamsrc bitrate=500000 ! video/x-h264,width=320,height=240,framerate=10/1 ! h264parse ! mp4mux ! filesink location=video.mp4;
Audio + Video record to file ( flv)
gst-launch-1.0 -e /
rpicamsrc bitrate=500000 ! /
video/x-h264,width=320,height=240,framerate=10/1 ! h264parse ! muxout. /
pulsesrc volume=8.0 ! /
queue ! audioconvert ! voaacenc bitrate=65536 ! aacparse ! muxout. /
flvmux name=muxout streamable=true ! filesink location ='test44.flv';
Audio + Video record to file ( flv) using pulsesrc
gst-launch-1.0 -v --gst-debug-level=3 pulsesrc device="alsa_input.platform-asoc-simple-card.0.analog-stereo" volume=5.0 mute=FALSE ! audio/x-raw,format=S16LE,rate=48000,channels=1 ! audioresample ! audioconvert ! voaacenc ! aacparse ! flvmux ! filesink location="voicetest.flv";
Audio + Video record to file (mp4)
gst-launch-1.0 -e / rpicamsrc bitrate=500000 ! / video/x-h264,width=320,height=240,framerate=10/1 !s h264parse ! muxout. / pulsesrc volume=4.0 ! / queue ! audioconvert ! voaacenc ! muxout. / flvmux name=muxout streamable=true ! filesink location = 'test224.mp4';
Streaming
stream raw Audio over RTMP to srtmpsink
gst-launch-1.0 pulsesrc device=hw:1 volume=8.0 ! /
audio/x-raw,format=S24LE ! audioconvert ! voaacenc bitrate=48000 ! aacparse ! flvmux ! rtmpsink location = “rtmp://192.168.0.3:1935/live/test”;
stream AACpparse Audio over RTMP to srtmpsink
gst-launch-1.0 -v --gst-debug-level=3 pulsesrc device="alsa_input.platform-asoc-simple-card.0.analog-stereo" volume=5.0 mute=FALSE ! audio/x-raw,format=S16LE,rate=48000,channels=1 ! audioresample ! audioconvert ! voaacenc ! aacparse ! flvmux ! rtmpsink location="rtmp://www.altani.com:1935/voice/1/test";
stream Video over RTMP
gst-launch-1.0 -e rpicamsrc bitrate=500000 ! /
video/x-h264,width=320,height=240,framerate=6/1 ! h264parse ! /
flvmux ! rtmpsink location = ‘rtmp://52.66.125.31:1935/live/test live=1’;
stream Audio + video over RTMP from rpicamsrc , framerate 10
gst-launch-1.0 rpicamsrc bitrate=500000 ! video/x-h264,width=320,height=240,framerate=10/1 ! h264parse ! muxout. pulsesrc volume=8.0 ! queue ! audioconvert ! voaacenc bitrate=65536 ! aacparse ! muxout. flvmux name=muxout streamable=true ! rtmpsink location ='rtmp://www.altanai.com/live/test44';
stream Audio + video over RTMP from rpicamsrc , framerate 30
gst-launch-1.0 rpicamsrc bitrate=500000 ! video/x-h264,width=1280,height=720,framerate=30/1 ! h264parse ! muxout. pulsesrc ! queue ! audioconvert ! voaacenc bitrate=65536 ! aacparse ! muxout. flvmux name=muxout ! queue ! rtmpsink location ='rtmp://www.altanai.com/live/test44';
VOD ( video On Demand )
Stream h264 file over RTMP
gst-launch-1.0 -e filesrc location="raw3.h264" ! video/x-h264 ! h264p
arse ! flvmux ! rtmpsink location = 'rtmp://www.altanai.com/live/test';
Stream flv file over RTMP
gst-launch-1.0 -e filesrc location=”testvieo.flv” ! /
video/x-h264,width=320,height=240,framerate=10/1 ! h264parse ! /
flvmux ! rtmpsink location = 'rtmp://192.168.0.3:1935/live/test';
Github Repo for Livestreaming
https://github.com/altanai/Livestreaming
Contains code for Android and ios Publishers , players on various platforms including HLS and Flash , streamings servers , Wowza playing mosules , webrtc broadcast
Gstreamer 1.8.0 – 24 March 2016
Features Hardware-accelerated zero-copy video decoding on Android
New video capture source for Android using the android.hardware.Camera API
Windows Media reverse playback support (ASF/WMV/WMA)
tracing system provides support for more sophisticated debugging tools
high-level GstPlayer playback convenience API
Initial support for the new Vulkan API
Improved Opus audio codec support: Support for more than two channels; MPEG-TS demuxer/muxer can handle Opus; sample-accurate encoding/decoding/transmuxing with Ogg, Matroska, ISOBMFF (Quicktime/MP4), and MPEG-TS as container; new codec utility functions for Opus header and caps handling in pbutils library. The Opus encoder/decoder elements were also moved to gst-plugins-base (from -bad), and the opus RTP depayloader/payloader to -good.
Asset proxy support in the GStreamer Editing Services
GStreamer 1.16.0 – 19 April 2019.
GStreamer WebRTC stack gained support for data channels for peer-to-peer communication based on SCTP, BUNDLE support, as well as support for multiple TURN servers.
AV1 video codec support for Matroska and QuickTime/MP4 containers and more configuration options and supported input formats for the AOMedia AV1 encoder
Closed Captions and other Ancillary Data in video
planar (non-interleaved) raw audio
GstVideoAggregator, compositor and OpenGL mixer elements are now in -base
New alternate fields interlace mode where each buffer carries a single field
WebM and Matroska ContentEncryption support in the Matroska demuxer
new WebKit WPE-based web browser source element
Video4Linux: HEVC encoding and decoding, JPEG encoding, and improved dmabuf import/export
Hardware-accelerated Nvidia video decoder gained support for VP8/VP9 decoding, whilst the encoder gained support for H.265/HEVC encoding.
Improvements to the Intel Media SDK based hardware-accelerated video decoder and encoder plugin (msdk): dmabuf import/export for zero-copy integration with other components; VP9 decoding; 10-bit HEVC encoding; video post-processing (vpp) support including deinterlacing; and the video decoder now handles dynamic resolution changes.
ASS/SSA subtitle overlay renderer can now handle multiple subtitles that overlap in time and will show them on screen simultaneously
Meson build feature-complete (with the exception of plugin docs) and it is now the recommended build system on all platforms. The Autotools build is scheduled to be removed in the next cycle.
GStreamer Rust bindings and Rust plugins module
GStreamer Editing Services allows directly playing back serialized edit list with playbin or (uri)decodebin
References :
Telecom R & D 