zhaohuwei
/
QG4
1
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity
Browse Source

Merge pull request #8596 from andrewvoznytsa/pr-small-video-tweaks 

Small video tweaks
QGC4.4
Gus Grubba 5 years ago committed by GitHub
parent
98f34318f9 8cc078b4d7
commit
8512a64170
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
19 changed files with 850 additions and 450 deletions
Whitespace
Show all changes Ignore whitespace when comparing lines Ignore changes in amount of whitespace Ignore changes in whitespace at EOL
Split View
Diff Options
Show Stats Download Patch File Download Diff File
  1. 2
      .gitmodules
  2. 52
      VideoReceiverApp/README.md
  3. 93
      VideoReceiverApp/main.cpp
  4. 10
      custom-example/res/CustomCameraControl.qml
  5. 6
      custom-example/src/CustomPlugin.cc
  6. 4
      custom-example/src/CustomPlugin.h
  7. 4
      custom-example/src/CustomVideoManager.cc
  8. 2
      custom-example/src/CustomVideoManager.h
  9. 10
      src/FlightDisplay/FlightDisplayViewVideo.qml
  10. 9
      src/FlightMap/Widgets/VideoPageWidget.qml
  11. 408
      src/VideoManager/VideoManager.cc
  12. 83
      src/VideoManager/VideoManager.h
  13. 21
      src/VideoReceiver/GStreamer.cc
  14. 493
      src/VideoReceiver/GstVideoReceiver.cc
  15. 25
      src/VideoReceiver/GstVideoReceiver.h
  16. 2
      src/VideoReceiver/README.md
  17. 65
      src/VideoReceiver/VideoReceiver.h
  18. 9
      src/api/QGCCorePlugin.cc
  19. 2
      src/api/QGCCorePlugin.h

2
.gitmodules vendored
Unescape View File

@ -7,6 +7,6 @@ @@ -7,6 +7,6 @@
[submodule "libs/OpenSSL/android_openssl"]
path = libs/OpenSSL/android_openssl
url = https://github.com/Auterion/android_openssl
[submodule "libs/gst-plugins-good"]
[submodule "libs/qmlglsink/gst-plugins-good"]
path = libs/qmlglsink/gst-plugins-good
url = https://github.com/mavlink/gst-plugins-good.git

52
VideoReceiverApp/README.md
Unescape View File

@ -0,0 +1,52 @@ @@ -0,0 +1,52 @@
# VideoReceiverApp
## Application
This is a simple test application developed to make VideoReceiver library development and testing easier. It can also be used as part of CI for system tests.
## Use cases and options
Application's behaviour depends on the executable name. There are two modes - QML and console. QML mode is enabled by renaming application executable to something that starts with **Q** (for example QVideoReceiverApp). In this case **video-sink** option is not available and application always tries to use **qmlglsink** for video rendering. In regular case (executable name does not start with **Q**) **autovideosink** or **fakesink** are used, depending on options.
### Available options and required arguments
```VideoReceiverApp [options] url```
for example:
```VideoReceiverApp -d --stop-decoding 30 rtsp://127.0.0.1:8554/test```
#### Options
```-h, --help``` - displays help
```-t, --timeout <seconds>``` - specifies source timeout
```-c, --connect <attempts>``` - specifies number of connection attempts
```-d, --decode``` - enables or disables video decoding and rendering
```--no-decode``` - disables video decoding and rendering if it was enabled by default
```--stop-decoding <seconds>``` - specifies amount of seconds after which decoding should be stopped
```-r, --record <file>``` - enables record video into file
```-f, --format <format>``` - specifies recording file format, where format 0 - MKV, 1 - MOV, 2 - MP4
```--stop-recording <seconds>``` - specifies amount of seconds after which recording should be stopped
```--video-sink <sink>``` - specifies which video sink to use : 0 - autovideosink, 1 - fakesink
#### Arguments
```url``` - required, specifies video URL.
Following URLs are supported:
```rtsp://<host>:<port>/mount/point``` - usual RTSP URL
```udp://<interface>:<port>``` - H.264 over RTP/UDP
```udp265://<interface>:<port>``` - H.265 over RTP/UDP
```tsusb://<interface>:<port>``` - Taisync's forwarded H.264 byte aligned NALU stream over UDP
```tcp://<host>:<port>``` - MPEG-2 TS over TCP
```mpegts://<interface>:<port>``` - MPEG-2 TS over UDP

93
VideoReceiverApp/main.cpp
Unescape View File

@ -155,6 +155,9 @@ private: @@ -155,6 +155,9 @@ private:
unsigned int _fileFormat = VideoReceiver::FILE_FORMAT_MIN;
unsigned _stopRecordingAfter = 15;
bool _useFakeSink = false;
bool _streaming = false;
bool _decoding = false;
bool _recording = false;
};
void
@ -299,52 +302,27 @@ VideoReceiverApp::exec() @@ -299,52 +302,27 @@ VideoReceiverApp::exec()
startStreaming();
QObject::connect(_receiver, &VideoReceiver::timeout, [this](){
QObject::connect(_receiver, &VideoReceiver::timeout, [](){
qCDebug(AppLog) << "Streaming timeout";
_dispatch([this](){
if (_receiver->streaming()) {
_receiver->stop();
} else {
if (--_connect > 0) {
qCDebug(AppLog) << "Restarting streaming";
_dispatch([this](){
startStreaming();
});
} else {
qCDebug(AppLog) << "Closing...";
delete _receiver;
_app.exit();
}
}
});
});
QObject::connect(_receiver, &VideoReceiver::streamingChanged, [this](){
if (_receiver->streaming()) {
QObject::connect(_receiver, &VideoReceiver::streamingChanged, [this](bool active){
_streaming = active;
if (_streaming) {
qCDebug(AppLog) << "Streaming started";
} else {
qCDebug(AppLog) << "Streaming stopped";
_dispatch([this](){
if (--_connect > 0) {
qCDebug(AppLog) << "Restarting streaming";
startStreaming();
} else {
qCDebug(AppLog) << "Closing...";
delete _receiver;
_app.exit();
}
});
}
});
QObject::connect(_receiver, &VideoReceiver::decodingChanged, [this](){
if (_receiver->decoding()) {
QObject::connect(_receiver, &VideoReceiver::decodingChanged, [this](bool active){
_decoding = active;
if (_decoding) {
qCDebug(AppLog) << "Decoding started";
} else {
qCDebug(AppLog) << "Decoding stopped";
if (_receiver->streaming()) {
if (!_receiver->recording()) {
if (_streaming) {
if (!_recording) {
_dispatch([this](){
_receiver->stop();
});
@ -353,13 +331,14 @@ VideoReceiverApp::exec() @@ -353,13 +331,14 @@ VideoReceiverApp::exec()
}
});
QObject::connect(_receiver, &VideoReceiver::recordingChanged, [this](){
if (_receiver->recording()) {
QObject::connect(_receiver, &VideoReceiver::recordingChanged, [this](bool active){
_recording = active;
if (_recording) {
qCDebug(AppLog) << "Recording started";
} else {
qCDebug(AppLog) << "Recording stopped";
if (_receiver->streaming()) {
if (!_receiver->decoding()) {
if (_streaming) {
if (!_decoding) {
_dispatch([this](){
_receiver->stop();
});
@ -368,6 +347,44 @@ VideoReceiverApp::exec() @@ -368,6 +347,44 @@ VideoReceiverApp::exec()
}
});
QObject::connect(_receiver, &VideoReceiver::onStartComplete, [this](VideoReceiver::STATUS status){
if (status != VideoReceiver::STATUS_OK) {
qCDebug(AppLog) << "Video receiver start failed";
_dispatch([this](){
if (--_connect > 0) {
qCDebug(AppLog) << "Restarting ...";
_dispatch([this](){
startStreaming();
});
} else {
qCDebug(AppLog) << "Closing...";
delete _receiver;
_app.exit();
}
});
} else {
qCDebug(AppLog) << "Video receiver started";
}
});
QObject::connect(_receiver, &VideoReceiver::onStopComplete, [this](VideoReceiver::STATUS ){
qCDebug(AppLog) << "Video receiver stopped";
_dispatch([this](){
if (--_connect > 0) {
qCDebug(AppLog) << "Restarting ...";
_dispatch([this](){
startStreaming();
});
} else {
qCDebug(AppLog) << "Closing...";
delete _receiver;
_app.exit();
}
});
});
return _app.exec();
}

10
custom-example/res/CustomCameraControl.qml
Unescape View File

@ -44,8 +44,8 @@ Item { @@ -44,8 +44,8 @@ Item {
property real _labelFieldWidth: ScreenTools.defaultFontPixelWidth * 28
property real _editFieldWidth: ScreenTools.defaultFontPixelWidth * 30
property real _editFieldHeight: ScreenTools.defaultFontPixelHeight * 2
property var _videoReceiver: QGroundControl.videoManager.videoReceiver
property bool _recordingLocalVideo: _videoReceiver && _videoReceiver.recording
property var _videoManager: QGroundControl.videoManager
property bool _recordingLocalVideo: QGroundControl.videoManager.recording
property var _dynamicCameras: activeVehicle ? activeVehicle.dynamicCameras : null
property bool _isCamera: _dynamicCameras ? _dynamicCameras.cameras.count > 0 : false
@ -305,15 +305,15 @@ Item { @@ -305,15 +305,15 @@ Item {
_camera.stopVideo()
//-- Local video as well
if (_recordingVideo) {
_videoReceiver.stopRecording()
_videoManager.stopRecording()
}
} else {
if(!_fullSD) {
_camera.startVideo()
}
//-- Local video as well
if(_videoReceiver) {
_videoReceiver.startRecording()
if(_videoManager) {
_videoManager.startRecording()
}
}
} else {

6
custom-example/src/CustomPlugin.cc
Unescape View File

@ -29,12 +29,8 @@ @@ -29,12 +29,8 @@
QGC_LOGGING_CATEGORY(CustomLog, "CustomLog")
CustomVideoReceiver::CustomVideoReceiver(QObject* parent)
: VideoReceiver(parent)
: GstVideoReceiver(parent)
{
#if defined(QGC_GST_STREAMING)
//-- Shorter RTSP test interval
_restart_time_ms = 1000;
#endif
}
CustomVideoReceiver::~CustomVideoReceiver()

4
custom-example/src/CustomPlugin.h
Unescape View File

@ -14,7 +14,7 @@ @@ -14,7 +14,7 @@
#include "QGCCorePlugin.h"
#include "QGCOptions.h"
#include "QGCLoggingCategory.h"
#include "VideoReceiver.h"
#include "GstVideoReceiver.h"
#include "SettingsManager.h"
#include <QTranslator>
@ -25,7 +25,7 @@ class CustomSettings; @@ -25,7 +25,7 @@ class CustomSettings;
Q_DECLARE_LOGGING_CATEGORY(CustomLog)
//-- Our own, custom video receiver
class CustomVideoReceiver : public VideoReceiver
class CustomVideoReceiver : public GstVideoReceiver
{
Q_OBJECT
public:

4
custom-example/src/CustomVideoManager.cc
Unescape View File

@ -20,10 +20,10 @@ CustomVideoManager::CustomVideoManager(QGCApplication* app, QGCToolbox* toolbox) @@ -20,10 +20,10 @@ CustomVideoManager::CustomVideoManager(QGCApplication* app, QGCToolbox* toolbox)
//-----------------------------------------------------------------------------
void
CustomVideoManager::_updateSettings()
CustomVideoManager::_updateSettings(unsigned id)
{
if(!_videoSettings || !_videoReceiver)
return;
VideoManager::_updateSettings();
VideoManager::_updateSettings(id);
}

2
custom-example/src/CustomVideoManager.h
Unescape View File

@ -23,6 +23,6 @@ public: @@ -23,6 +23,6 @@ public:
CustomVideoManager (QGCApplication* app, QGCToolbox* toolbox);
protected:
void _updateSettings ();
void _updateSettings (unsigned id);
};

10
src/FlightDisplay/FlightDisplayViewVideo.qml
Unescape View File

@ -25,7 +25,6 @@ Item { @@ -25,7 +25,6 @@ Item {
clip: true
property double _ar: QGroundControl.videoManager.aspectRatio
property bool _showGrid: QGroundControl.settingsManager.videoSettings.gridLines.rawValue > 0
property var _videoReceiver: QGroundControl.videoManager.videoReceiver
property var _dynamicCameras: activeVehicle ? activeVehicle.dynamicCameras : null
property bool _connected: activeVehicle ? !activeVehicle.connectionLost : false
property int _curCameraIndex: _dynamicCameras ? _dynamicCameras.currentCamera : 0
@ -40,7 +39,7 @@ Item { @@ -40,7 +39,7 @@ Item {
id: noVideo
anchors.fill: parent
color: Qt.rgba(0,0,0,0.75)
visible: !(_videoReceiver && _videoReceiver.decoding)
visible: !(QGroundControl.videoManager.decoding)
QGCLabel {
text: QGroundControl.settingsManager.videoSettings.streamEnabled.rawValue ? qsTr("WAITING FOR VIDEO") : qsTr("VIDEO DISABLED")
font.family: ScreenTools.demiboldFontFamily
@ -58,7 +57,7 @@ Item { @@ -58,7 +57,7 @@ Item {
Rectangle {
anchors.fill: parent
color: "black"
visible: _videoReceiver && _videoReceiver.decoding
visible: QGroundControl.videoManager.decoding
function getWidth() {
//-- Fit Width or Stretch
if(_fitMode === 0 || _fitMode === 2) {
@ -80,10 +79,9 @@ Item { @@ -80,10 +79,9 @@ Item {
QGCVideoBackground {
id: videoContent
objectName: "videoContent"
receiver: _videoReceiver
Connections {
target: _videoReceiver
target: QGroundControl.videoManager
onImageFileChanged: {
videoContent.grabToImage(function(result) {
if (!result.saveToFile(QGroundControl.videoManager.imageFile)) {
@ -130,7 +128,7 @@ Item { @@ -130,7 +128,7 @@ Item {
height: parent.getHeight()
width: parent.getWidth()
anchors.centerIn: parent
visible: _videoReceiver && _videoReceiver.decoding
visible: QGroundControl.videoManager.decoding
sourceComponent: videoBackgroundComponent
property bool videoDisabled: QGroundControl.settingsManager.videoSettings.videoSource.rawValue === QGroundControl.settingsManager.videoSettings.disabledVideoSource

9
src/FlightMap/Widgets/VideoPageWidget.qml
Unescape View File

@ -31,9 +31,8 @@ Item { @@ -31,9 +31,8 @@ Item {
anchors.centerIn: parent
property bool _communicationLost: activeVehicle ? activeVehicle.connectionLost : false
property var _videoReceiver: QGroundControl.videoManager.videoReceiver
property bool _recordingVideo: _videoReceiver && _videoReceiver.recording
property bool _decodingVideo: _videoReceiver && _videoReceiver.decoding
property bool _recordingVideo: QGroundControl.videoManager.recording
property bool _decodingVideo: QGroundControl.videoManager.decoding
property bool _streamingEnabled: QGroundControl.settingsManager.videoSettings.streamConfigured
property var _dynamicCameras: activeVehicle ? activeVehicle.dynamicCameras : null
property int _curCameraIndex: _dynamicCameras ? _dynamicCameras.currentCamera : 0
@ -70,10 +69,10 @@ Item { @@ -70,10 +69,10 @@ Item {
onClicked: {
if(checked) {
QGroundControl.settingsManager.videoSettings.streamEnabled.rawValue = 1
_videoReceiver.start()
QGroundControl.videoManager.startVideo()
} else {
QGroundControl.settingsManager.videoSettings.streamEnabled.rawValue = 0
_videoReceiver.stop()
QGroundControl.videoManager.stopVideo()
}
}
}

408
src/VideoManager/VideoManager.cc
Unescape View File

@ -34,6 +34,10 @@ @@ -34,6 +34,10 @@
#include "GLVideoItemStub.h"
#endif
#ifdef QGC_GST_TAISYNC_ENABLED
#include "TaisyncHandler.h"
#endif
QGC_LOGGING_CATEGORY(VideoManagerLog, "VideoManagerLog")
#if defined(QGC_GST_STREAMING)
@ -60,16 +64,22 @@ VideoManager::VideoManager(QGCApplication* app, QGCToolbox* toolbox) @@ -60,16 +64,22 @@ VideoManager::VideoManager(QGCApplication* app, QGCToolbox* toolbox)
//-----------------------------------------------------------------------------
VideoManager::~VideoManager()
{
delete _videoReceiver;
_videoReceiver = nullptr;
delete _thermalVideoReceiver;
_thermalVideoReceiver = nullptr;
for (int i = 0; i < 2; i++) {
if (_videoReceiver[i] != nullptr) {
delete _videoReceiver[i];
_videoReceiver[i] = nullptr;
}
#if defined(QGC_GST_STREAMING)
GStreamer::releaseVideoSink(_thermalVideoSink);
_thermalVideoSink = nullptr;
GStreamer::releaseVideoSink(_videoSink);
_videoSink = nullptr;
if (_videoSink[i] != nullptr) {
// FIXME: AV: we need some interaface for video sink with .release() call
// Currently VideoManager is destroyed after corePlugin() and we are crashing on app exit
// calling qgcApp()->toolbox()->corePlugin()->releaseVideoSink(_videoSink[i]);
// As for now let's call GStreamer::releaseVideoSink() directly
GStreamer::releaseVideoSink(_videoSink[i]);
_videoSink[i] = nullptr;
}
#endif
}
}
//-----------------------------------------------------------------------------
@ -102,21 +112,90 @@ VideoManager::setToolbox(QGCToolbox *toolbox) @@ -102,21 +112,90 @@ VideoManager::setToolbox(QGCToolbox *toolbox)
emit isGStreamerChanged();
qCDebug(VideoManagerLog) << "New Video Source:" << videoSource;
#if defined(QGC_GST_STREAMING)
_videoReceiver = toolbox->corePlugin()->createVideoReceiver(this);
_thermalVideoReceiver = toolbox->corePlugin()->createVideoReceiver(this);
_videoReceiver[0] = toolbox->corePlugin()->createVideoReceiver(this);
_videoReceiver[1] = toolbox->corePlugin()->createVideoReceiver(this);
connect(_videoReceiver[0], &VideoReceiver::streamingChanged, this, [this](bool active){
_streaming = active;
emit streamingChanged();
});
connect(_videoReceiver[0], &VideoReceiver::onStartComplete, this, [this](VideoReceiver::STATUS status) {
if (status == VideoReceiver::STATUS_OK) {
_videoStarted[0] = true;
if (_videoSink[0] != nullptr) {
// It is absolytely ok to have video receiver active (streaming) and decoding not active
// It should be handy for cases when you have many streams and want to show only some of them
// NOTE that even if decoder did not start it is still possible to record video
_videoReceiver[0]->startDecoding(_videoSink[0]);
}
} else if (status == VideoReceiver::STATUS_INVALID_URL) {
// Invalid URL - don't restart
} else if (status == VideoReceiver::STATUS_INVALID_STATE) {
// Already running
} else {
_restartVideo(0);
}
});
connect(_videoReceiver[0], &VideoReceiver::onStopComplete, this, [this](VideoReceiver::STATUS) {
_videoStarted[0] = false;
_startReceiver(0);
});
connect(_videoReceiver[0], &VideoReceiver::decodingChanged, this, [this](bool active){
_decoding = active;
emit decodingChanged();
});
connect(_videoReceiver[0], &VideoReceiver::recordingChanged, this, [this](bool active){
_recording = active;
if (!active) {
_subtitleWriter.stopCapturingTelemetry();
}
emit recordingChanged();
});
connect(_videoReceiver[0], &VideoReceiver::recordingStarted, this, [this](){
_subtitleWriter.startCapturingTelemetry(_videoFile);
});
connect(_videoReceiver[0], &VideoReceiver::videoSizeChanged, this, [this](QSize size){
_videoSize = ((quint32)size.width() << 16) | (quint32)size.height();
emit videoSizeChanged();
});
connect(_videoReceiver, &VideoReceiver::timeout, this, &VideoManager::_restartVideo);
connect(_videoReceiver, &VideoReceiver::streamingChanged, this, &VideoManager::_streamingChanged);
connect(_videoReceiver, &VideoReceiver::recordingStarted, this, &VideoManager::_recordingStarted);
connect(_videoReceiver, &VideoReceiver::recordingChanged, this, &VideoManager::_recordingChanged);
connect(_videoReceiver, &VideoReceiver::screenshotComplete, this, &VideoManager::_screenshotComplete);
//connect(_videoReceiver, &VideoReceiver::onTakeScreenshotComplete, this, [this](VideoReceiver::STATUS status){
// if (status == VideoReceiver::STATUS_OK) {
// }
//});
// FIXME: AV: I believe _thermalVideoReceiver should be handled just like _videoReceiver in terms of event
// and I expect that it will be changed during multiple video stream activity
connect(_thermalVideoReceiver, &VideoReceiver::timeout, this, &VideoManager::_restartVideo);
connect(_thermalVideoReceiver, &VideoReceiver::streamingChanged, this, &VideoManager::_streamingChanged);
if (_videoReceiver[1] != nullptr) {
connect(_videoReceiver[1], &VideoReceiver::onStartComplete, this, [this](VideoReceiver::STATUS status) {
if (status == VideoReceiver::STATUS_OK) {
_videoStarted[1] = true;
if (_videoSink[1] != nullptr) {
_videoReceiver[1]->startDecoding(_videoSink[1]);
}
} else if (status == VideoReceiver::STATUS_INVALID_URL) {
// Invalid URL - don't restart
} else if (status == VideoReceiver::STATUS_INVALID_STATE) {
// Already running
} else {
_restartVideo(1);
}
});
connect(_videoReceiver[1], &VideoReceiver::onStopComplete, this, [this](VideoReceiver::STATUS) {
_videoStarted[1] = false;
_startReceiver(1);
});
}
#endif
_updateSettings();
_updateSettings(0);
_updateSettings(1);
if(isGStreamer()) {
startVideo();
} else {
@ -180,23 +259,8 @@ VideoManager::startVideo() @@ -180,23 +259,8 @@ VideoManager::startVideo()
return;
}
#if defined(QGC_GST_STREAMING)
const unsigned timeout = _videoSettings->rtspTimeout()->rawValue().toUInt();
if(_videoReceiver != nullptr) {
_videoReceiver->start(_videoUri, timeout);
if (_videoSink != nullptr) {
_videoReceiver->startDecoding(_videoSink);
}
}
if(_thermalVideoReceiver != nullptr) {
_thermalVideoReceiver->start(_thermalVideoUri, timeout);
if (_thermalVideoSink != nullptr) {
_thermalVideoReceiver->startDecoding(_thermalVideoSink);
}
}
#endif
_startReceiver(0);
_startReceiver(1);
}
//-----------------------------------------------------------------------------
@ -206,10 +270,9 @@ VideoManager::stopVideo() @@ -206,10 +270,9 @@ VideoManager::stopVideo()
if (qgcApp()->runningUnitTests()) {
return;
}
#if defined(QGC_GST_STREAMING)
if(_videoReceiver) _videoReceiver->stop();
if(_thermalVideoReceiver) _thermalVideoReceiver->stop();
#endif
_stopReceiver(1);
_stopReceiver(0);
}
void
@ -219,7 +282,7 @@ VideoManager::startRecording(const QString& videoFile) @@ -219,7 +282,7 @@ VideoManager::startRecording(const QString& videoFile)
return;
}
#if defined(QGC_GST_STREAMING)
if (!_videoReceiver) {
if (!_videoReceiver[0]) {
qgcApp()->showAppMessage(tr("Video receiver is not ready."));
return;
}
@ -230,22 +293,31 @@ VideoManager::startRecording(const QString& videoFile) @@ -230,22 +293,31 @@ VideoManager::startRecording(const QString& videoFile)
qgcApp()->showAppMessage(tr("Invalid video format defined."));
return;
}
QString ext = kFileExtension[fileFormat - VideoReceiver::FILE_FORMAT_MIN];
//-- Disk usage maintenance
_cleanupOldVideos();
QString savePath = qgcApp()->toolbox()->settingsManager()->appSettings()->videoSavePath();
if(savePath.isEmpty()) {
if (savePath.isEmpty()) {
qgcApp()->showAppMessage(tr("Unabled to record video. Video save path must be specified in Settings."));
return;
}
_videoFile = savePath + "/"
+ (videoFile.isEmpty() ? QDateTime::currentDateTime().toString("yyyy-MM-dd_hh.mm.ss") : videoFile)
+ "." + kFileExtension[fileFormat - VideoReceiver::FILE_FORMAT_MIN];
+ ".";
QString videoFile2 = _videoFile + "2." + ext;
_videoFile += ext;
if (_videoReceiver[0] && _videoStarted[0]) {
_videoReceiver[0]->startRecording(_videoFile, fileFormat);
}
if (_videoReceiver[1] && _videoStarted[1]) {
_videoReceiver[1]->startRecording(videoFile2, fileFormat);
}
_videoReceiver->startRecording(_videoFile, fileFormat);
#else
Q_UNUSED(videoFile)
#endif
@ -258,11 +330,12 @@ VideoManager::stopRecording() @@ -258,11 +330,12 @@ VideoManager::stopRecording()
return;
}
#if defined(QGC_GST_STREAMING)
if (!_videoReceiver) {
return;
}
_videoReceiver->stopRecording();
for (int i = 0; i < 2; i++) {
if (_videoReceiver[i]) {
_videoReceiver[i]->stopRecording();
}
}
#endif
}
@ -273,7 +346,7 @@ VideoManager::grabImage(const QString& imageFile) @@ -273,7 +346,7 @@ VideoManager::grabImage(const QString& imageFile)
return;
}
#if defined(QGC_GST_STREAMING)
if (!_videoReceiver) {
if (!_videoReceiver[0]) {
return;
}
@ -281,7 +354,7 @@ VideoManager::grabImage(const QString& imageFile) @@ -281,7 +354,7 @@ VideoManager::grabImage(const QString& imageFile)
emit imageFileChanged();
_videoReceiver->takeScreenshot(_imageFile);
_videoReceiver[0]->takeScreenshot(_imageFile);
#else
Q_UNUSED(imageFile)
#endif
@ -399,35 +472,35 @@ VideoManager::_videoSourceChanged() @@ -399,35 +472,35 @@ VideoManager::_videoSourceChanged()
emit hasVideoChanged();
emit isGStreamerChanged();
emit isAutoStreamChanged();
_restartVideo();
_restartVideo(0);
}
//-----------------------------------------------------------------------------
void
VideoManager::_udpPortChanged()
{
_restartVideo();
_restartVideo(0);
}
//-----------------------------------------------------------------------------
void
VideoManager::_rtspUrlChanged()
{
_restartVideo();
_restartVideo(0);
}
//-----------------------------------------------------------------------------
void
VideoManager::_tcpUrlChanged()
{
_restartVideo();
_restartVideo(0);
}
//-----------------------------------------------------------------------------
void
VideoManager::_lowLatencyModeChanged()
{
//restartVideo();
_restartAllVideos();
}
//-----------------------------------------------------------------------------
@ -482,6 +555,7 @@ VideoManager::setfullScreen(bool f) @@ -482,6 +555,7 @@ VideoManager::setfullScreen(bool f)
emit fullScreenChanged();
}
//-----------------------------------------------------------------------------
void
VideoManager::_initVideo()
{
@ -495,9 +569,12 @@ VideoManager::_initVideo() @@ -495,9 +569,12 @@ VideoManager::_initVideo()
QQuickItem* widget = root->findChild<QQuickItem*>("videoContent");
if (widget != nullptr && _videoReceiver != nullptr) {
if ((_videoSink = qgcApp()->toolbox()->corePlugin()->createVideoSink(this, widget)) != nullptr) {
_videoReceiver->startDecoding(_videoSink);
if (widget != nullptr && _videoReceiver[0] != nullptr) {
_videoSink[0] = qgcApp()->toolbox()->corePlugin()->createVideoSink(this, widget);
if (_videoSink[0] != nullptr) {
if (_videoStarted[0]) {
_videoReceiver[0]->startDecoding(_videoSink[0]);
}
} else {
qCDebug(VideoManagerLog) << "createVideoSink() failed";
}
@ -507,9 +584,12 @@ VideoManager::_initVideo() @@ -507,9 +584,12 @@ VideoManager::_initVideo()
widget = root->findChild<QQuickItem*>("thermalVideo");
if (widget != nullptr && _thermalVideoReceiver != nullptr) {
if ((_thermalVideoSink = qgcApp()->toolbox()->corePlugin()->createVideoSink(this, widget)) != nullptr) {
_thermalVideoReceiver->startDecoding(_thermalVideoSink);
if (widget != nullptr && _videoReceiver[1] != nullptr) {
_videoSink[1] = qgcApp()->toolbox()->corePlugin()->createVideoSink(this, widget);
if (_videoSink[1] != nullptr) {
if (_videoStarted[1]) {
_videoReceiver[1]->startDecoding(_videoSink[1]);
}
} else {
qCDebug(VideoManagerLog) << "createVideoSink() failed";
}
@ -520,150 +600,186 @@ VideoManager::_initVideo() @@ -520,150 +600,186 @@ VideoManager::_initVideo()
}
//-----------------------------------------------------------------------------
void
VideoManager::_updateSettings()
bool
VideoManager::_updateSettings(unsigned id)
{
if(!_videoSettings)
return;
return false;
const bool lowLatencyStreaming =_videoSettings->lowLatencyMode()->rawValue().toBool();
bool settingsChanged = _lowLatencyStreaming[id] != lowLatencyStreaming;
_lowLatencyStreaming[id] = lowLatencyStreaming;
//-- Auto discovery
if(_activeVehicle && _activeVehicle->dynamicCameras()) {
QGCVideoStreamInfo* pInfo = _activeVehicle->dynamicCameras()->currentStreamInstance();
if(pInfo) {
qCDebug(VideoManagerLog) << "Configure primary stream: " << pInfo->uri();
switch(pInfo->type()) {
case VIDEO_STREAM_TYPE_RTSP:
_setVideoUri(pInfo->uri());
_toolbox->settingsManager()->videoSettings()->videoSource()->setRawValue(VideoSettings::videoSourceRTSP);
break;
case VIDEO_STREAM_TYPE_TCP_MPEG:
_setVideoUri(pInfo->uri());
_toolbox->settingsManager()->videoSettings()->videoSource()->setRawValue(VideoSettings::videoSourceTCP);
break;
case VIDEO_STREAM_TYPE_RTPUDP:
_setVideoUri(QStringLiteral("udp://0.0.0.0:%1").arg(pInfo->uri()));
_toolbox->settingsManager()->videoSettings()->videoSource()->setRawValue(VideoSettings::videoSourceUDPH264);
break;
case VIDEO_STREAM_TYPE_MPEG_TS_H264:
_setVideoUri(QStringLiteral("mpegts://0.0.0.0:%1").arg(pInfo->uri()));
_toolbox->settingsManager()->videoSettings()->videoSource()->setRawValue(VideoSettings::videoSourceMPEGTS);
break;
default:
_setVideoUri(pInfo->uri());
break;
}
//-- Thermal stream (if any)
QGCVideoStreamInfo* pTinfo = _activeVehicle->dynamicCameras()->thermalStreamInstance();
if(pTinfo) {
qCDebug(VideoManagerLog) << "Configure secondary stream: " << pTinfo->uri();
switch(pTinfo->type()) {
if (id == 0) {
qCDebug(VideoManagerLog) << "Configure primary stream:" << pInfo->uri();
switch(pInfo->type()) {
case VIDEO_STREAM_TYPE_RTSP:
if ((settingsChanged |= _updateVideoUri(id, pInfo->uri()))) {
_toolbox->settingsManager()->videoSettings()->videoSource()->setRawValue(VideoSettings::videoSourceRTSP);
}
break;
case VIDEO_STREAM_TYPE_TCP_MPEG:
_setThermalVideoUri(pTinfo->uri());
if ((settingsChanged |= _updateVideoUri(id, pInfo->uri()))) {
_toolbox->settingsManager()->videoSettings()->videoSource()->setRawValue(VideoSettings::videoSourceTCP);
}
break;
case VIDEO_STREAM_TYPE_RTPUDP:
_setThermalVideoUri(QStringLiteral("udp://0.0.0.0:%1").arg(pTinfo->uri()));
if ((settingsChanged |= _updateVideoUri(id, QStringLiteral("udp://0.0.0.0:%1").arg(pInfo->uri())))) {
_toolbox->settingsManager()->videoSettings()->videoSource()->setRawValue(VideoSettings::videoSourceUDPH264);
}
break;
case VIDEO_STREAM_TYPE_MPEG_TS_H264:
_setThermalVideoUri(QStringLiteral("mpegts://0.0.0.0:%1").arg(pTinfo->uri()));
if ((settingsChanged |= _updateVideoUri(id, QStringLiteral("mpegts://0.0.0.0:%1").arg(pInfo->uri())))) {
_toolbox->settingsManager()->videoSettings()->videoSource()->setRawValue(VideoSettings::videoSourceMPEGTS);
}
break;
default:
_setThermalVideoUri(pTinfo->uri());
settingsChanged |= _updateVideoUri(id, pInfo->uri());
break;
}
}
return;
else if (id == 1) { //-- Thermal stream (if any)
QGCVideoStreamInfo* pTinfo = _activeVehicle->dynamicCameras()->thermalStreamInstance();
if (pTinfo) {
qCDebug(VideoManagerLog) << "Configure secondary stream:" << pTinfo->uri();
switch(pTinfo->type()) {
case VIDEO_STREAM_TYPE_RTSP:
case VIDEO_STREAM_TYPE_TCP_MPEG:
settingsChanged |= _updateVideoUri(id, pTinfo->uri());
break;
case VIDEO_STREAM_TYPE_RTPUDP:
settingsChanged |= _updateVideoUri(id, QStringLiteral("udp://0.0.0.0:%1").arg(pTinfo->uri()));
break;
case VIDEO_STREAM_TYPE_MPEG_TS_H264:
settingsChanged |= _updateVideoUri(id, QStringLiteral("mpegts://0.0.0.0:%1").arg(pTinfo->uri()));
break;
default:
settingsChanged |= _updateVideoUri(id, pTinfo->uri());
break;
}
}
}
return settingsChanged;
}
}
QString source = _videoSettings->videoSource()->rawValue().toString();
if (source == VideoSettings::videoSourceUDPH264)
_setVideoUri(QStringLiteral("udp://0.0.0.0:%1").arg(_videoSettings->udpPort()->rawValue().toInt()));
settingsChanged |= _updateVideoUri(0, QStringLiteral("udp://0.0.0.0:%1").arg(_videoSettings->udpPort()->rawValue().toInt()));
else if (source == VideoSettings::videoSourceUDPH265)
_setVideoUri(QStringLiteral("udp265://0.0.0.0:%1").arg(_videoSettings->udpPort()->rawValue().toInt()));
settingsChanged |= _updateVideoUri(0, QStringLiteral("udp265://0.0.0.0:%1").arg(_videoSettings->udpPort()->rawValue().toInt()));
else if (source == VideoSettings::videoSourceMPEGTS)
_setVideoUri(QStringLiteral("mpegts://0.0.0.0:%1").arg(_videoSettings->udpPort()->rawValue().toInt()));
settingsChanged |= _updateVideoUri(0, QStringLiteral("mpegts://0.0.0.0:%1").arg(_videoSettings->udpPort()->rawValue().toInt()));
else if (source == VideoSettings::videoSourceRTSP)
_setVideoUri(_videoSettings->rtspUrl()->rawValue().toString());
settingsChanged |= _updateVideoUri(0, _videoSettings->rtspUrl()->rawValue().toString());
else if (source == VideoSettings::videoSourceTCP)
_setVideoUri(QStringLiteral("tcp://%1").arg(_videoSettings->tcpUrl()->rawValue().toString()));
settingsChanged |= _updateVideoUri(0, QStringLiteral("tcp://%1").arg(_videoSettings->tcpUrl()->rawValue().toString()));
return settingsChanged;
}
void
VideoManager::_setVideoUri(const QString& uri)
//-----------------------------------------------------------------------------
bool
VideoManager::_updateVideoUri(unsigned id, const QString& uri)
{
#if defined(QGC_GST_TAISYNC_ENABLED) && (defined(__android__) || defined(__ios__))
//-- Taisync on iOS or Android sends a raw h.264 stream
if (isTaisync()) {
_videoUri = QString("tsusb://0.0.0.0:%1").arg(TAISYNC_VIDEO_UDP_PORT);
return;
if (id == 0) {
return _updateVideoUri(0, QString("tsusb://0.0.0.0:%1").arg(TAISYNC_VIDEO_UDP_PORT));
} if (id == 1) {
// FIXME: AV: TAISYNC_VIDEO_UDP_PORT is used by video stream, thermal stream should go via its own proxy
if (!_videoUri[1].isEmpty()) {
_videoUri[1].clear();
return true;
} else {
return false;
}
}
}
#endif
_videoUri = uri;
if (uri == _videoUri[id]) {
return false;
}
_videoUri[id] = uri;
return true;
}
//-----------------------------------------------------------------------------
void
VideoManager::_setThermalVideoUri(const QString& uri)
VideoManager::_restartVideo(unsigned id)
{
#if defined(QGC_GST_TAISYNC_ENABLED) && (defined(__android__) || defined(__ios__))
//-- Taisync on iOS or Android sends a raw h.264 stream
if (isTaisync()) {
// FIXME: AV: TAISYNC_VIDEO_UDP_PORT is used by video stream, thermal stream should go via its own proxy
_thermalVideoUri = QString("tsusb://0.0.0.0:%1").arg(TAISYNC_VIDEO_UDP_PORT);
if (qgcApp()->runningUnitTests()) {
return;
}
#endif
_thermalVideoUri = uri;
}
//-----------------------------------------------------------------------------
void
VideoManager::_streamingChanged()
{
#if defined(QGC_GST_STREAMING)
// FIXME: AV: we need VideoReceiver::running() to avoid restarting if one of streams is not active
// but since VideoManager is going to be relpaced by Video Model during multiple video streaming development activity
// I'll leave it as is for week or two
if ((_videoReceiver && !_videoReceiver->streaming())
/*|| (_thermalVideoReceiver && !_thermalVideoReceiver->streaming())*/) {
_restartVideo();
bool oldLowLatencyStreaming = _lowLatencyStreaming[id];
QString oldUri = _videoUri[id];
_updateSettings(id);
bool newLowLatencyStreaming = _lowLatencyStreaming[id];
QString newUri = _videoUri[id];
// FIXME: AV: use _updateSettings() result to check if settings were changed
if (oldUri == newUri && oldLowLatencyStreaming == newLowLatencyStreaming && _videoStarted[id]) {
qCDebug(VideoManagerLog) << "No sense to restart video streaming, skipped" << id;
return;
}
#endif
}
//-----------------------------------------------------------------------------
void
VideoManager::_restartVideo()
{
#if defined(QGC_GST_STREAMING)
qCDebug(VideoManagerLog) << "Restart video streaming";
stopVideo();
_updateSettings();
startVideo();
emit aspectRatioChanged();
qCDebug(VideoManagerLog) << "Restart video streaming" << id;
if (_videoStarted[id]) {
_stopReceiver(id);
} else {
_startReceiver(id);
}
#endif
}
//-----------------------------------------------------------------------------
void
VideoManager::_recordingStarted()
VideoManager::_restartAllVideos()
{
_subtitleWriter.startCapturingTelemetry(_videoFile);
_restartVideo(0);
_restartVideo(1);
}
//-----------------------------------------------------------------------------
//----------------------------------------------------------------------------------------
void
VideoManager::_recordingChanged()
VideoManager::_startReceiver(unsigned id)
{
#if defined(QGC_GST_STREAMING)
if (_videoReceiver && !_videoReceiver->recording()) {
_subtitleWriter.stopCapturingTelemetry();
const unsigned timeout = _videoSettings->rtspTimeout()->rawValue().toUInt();
if (id > 1) {
qCDebug(VideoManagerLog) << "Unsupported receiver id" << id;
} else if (_videoReceiver[id] != nullptr/* && _videoSink[id] != nullptr*/) {
if (!_videoUri[id].isEmpty()) {
_videoReceiver[id]->start(_videoUri[id], timeout, _lowLatencyStreaming[id] ? -1 : 0);
}
}
#endif
}
//----------------------------------------------------------------------------------------
void
VideoManager::_screenshotComplete()
VideoManager::_stopReceiver(unsigned id)
{
#if defined(QGC_GST_STREAMING)
if (id > 1) {
qCDebug(VideoManagerLog) << "Unsupported receiver id" << id;
} else if (_videoReceiver[id] != nullptr) {
_videoReceiver[id]->stop();
}
#endif
}
//----------------------------------------------------------------------------------------
@ -677,14 +793,14 @@ VideoManager::_setActiveVehicle(Vehicle* vehicle) @@ -677,14 +793,14 @@ VideoManager::_setActiveVehicle(Vehicle* vehicle)
if(pCamera) {
pCamera->stopStream();
}
disconnect(_activeVehicle->dynamicCameras(), &QGCCameraManager::streamChanged, this, &VideoManager::_restartVideo);
disconnect(_activeVehicle->dynamicCameras(), &QGCCameraManager::streamChanged, this, &VideoManager::_restartAllVideos);
}
}
_activeVehicle = vehicle;
if(_activeVehicle) {
connect(_activeVehicle, &Vehicle::connectionLostChanged, this, &VideoManager::_connectionLostChanged);
if(_activeVehicle->dynamicCameras()) {
connect(_activeVehicle->dynamicCameras(), &QGCCameraManager::streamChanged, this, &VideoManager::_restartVideo);
connect(_activeVehicle->dynamicCameras(), &QGCCameraManager::streamChanged, this, &VideoManager::_restartAllVideos);
QGCCameraControl* pCamera = _activeVehicle->dynamicCameras()->currentCameraInstance();
if(pCamera) {
pCamera->resumeStream();
@ -695,7 +811,7 @@ VideoManager::_setActiveVehicle(Vehicle* vehicle) @@ -695,7 +811,7 @@ VideoManager::_setActiveVehicle(Vehicle* vehicle)
setfullScreen(false);
}
emit autoStreamConfiguredChanged();
_restartVideo();
_restartAllVideos();
}
//----------------------------------------------------------------------------------------

83
src/VideoManager/VideoManager.h
Unescape View File

@ -49,8 +49,12 @@ public: @@ -49,8 +49,12 @@ public:
Q_PROPERTY(double hfov READ hfov NOTIFY aspectRatioChanged)
Q_PROPERTY(double thermalHfov READ thermalHfov NOTIFY aspectRatioChanged)
Q_PROPERTY(bool autoStreamConfigured READ autoStreamConfigured NOTIFY autoStreamConfiguredChanged)
Q_PROPERTY(bool hasThermal READ hasThermal NOTIFY aspectRatioChanged)
Q_PROPERTY(bool hasThermal READ hasThermal NOTIFY decodingChanged)
Q_PROPERTY(QString imageFile READ imageFile NOTIFY imageFileChanged)
Q_PROPERTY(bool streaming READ streaming NOTIFY streamingChanged)
Q_PROPERTY(bool decoding READ decoding NOTIFY decodingChanged)
Q_PROPERTY(bool recording READ recording NOTIFY recordingChanged)
Q_PROPERTY(QSize videoSize READ videoSize NOTIFY videoSizeChanged)
virtual bool hasVideo ();
virtual bool isGStreamer ();
@ -65,9 +69,27 @@ public: @@ -65,9 +69,27 @@ public:
virtual bool hasThermal ();
virtual QString imageFile ();
bool streaming(void) {
return _streaming;
}
virtual VideoReceiver* videoReceiver () { return _videoReceiver; }
virtual VideoReceiver* thermalVideoReceiver () { return _thermalVideoReceiver; }
bool decoding(void) {
return _decoding;
}
bool recording(void) {
return _recording;
}
QSize videoSize(void) {
const quint32 size = _videoSize;
return QSize((size >> 16) & 0xFFFF, size & 0xFFFF);
}
// FIXME: AV: they should be removed after finishing multiple video stream support
// new arcitecture does not assume direct access to video receiver from QML side, even if it works for now
virtual VideoReceiver* videoReceiver () { return _videoReceiver[0]; }
virtual VideoReceiver* thermalVideoReceiver () { return _videoReceiver[1]; }
#if defined(QGC_DISABLE_UVC)
virtual bool uvcEnabled () { return false; }
@ -99,6 +121,11 @@ signals: @@ -99,6 +121,11 @@ signals:
void aspectRatioChanged ();
void autoStreamConfiguredChanged();
void imageFileChanged ();
void streamingChanged ();
void decodingChanged ();
void recordingChanged ();
void recordingStarted ();
void videoSizeChanged ();
protected slots:
void _videoSourceChanged ();
@ -115,31 +142,37 @@ protected: @@ -115,31 +142,37 @@ protected:
friend class FinishVideoInitialization;
void _initVideo ();
void _updateSettings ();
void _setVideoUri (const QString& uri);
void _setThermalVideoUri (const QString& uri);
bool _updateSettings (unsigned id);
bool _updateVideoUri (unsigned id, const QString& uri);
void _cleanupOldVideos ();
void _restartVideo ();
void _streamingChanged ();
void _recordingStarted ();
void _recordingChanged ();
void _screenshotComplete ();
void _restartAllVideos ();
void _restartVideo (unsigned id);
void _startReceiver (unsigned id);
void _stopReceiver (unsigned id);
protected:
QString _videoFile;
QString _imageFile;
SubtitleWriter _subtitleWriter;
bool _isTaisync = false;
VideoReceiver* _videoReceiver = nullptr;
VideoReceiver* _thermalVideoReceiver = nullptr;
void* _videoSink = nullptr;
void* _thermalVideoSink = nullptr;
VideoSettings* _videoSettings = nullptr;
QString _videoUri;
QString _thermalVideoUri;
QString _videoSourceID;
bool _fullScreen = false;
Vehicle* _activeVehicle = nullptr;
QString _videoFile;
QString _imageFile;
SubtitleWriter _subtitleWriter;
bool _isTaisync = false;
VideoReceiver* _videoReceiver[2] = { nullptr, nullptr };
void* _videoSink[2] = { nullptr, nullptr };
QString _videoUri[2];
// FIXME: AV: _videoStarted seems to be access from 3 different threads, from time to time
// 1) Video Receiver thread
// 2) Video Manager/main app thread
// 3) Qt rendering thread (during video sink creation process which should happen in this thread)
// It works for now but...
bool _videoStarted[2] = { false, false };
bool _lowLatencyStreaming[2] = { false, false };
QAtomicInteger<bool> _streaming = false;
QAtomicInteger<bool> _decoding = false;
QAtomicInteger<bool> _recording = false;
QAtomicInteger<quint32> _videoSize = 0;
VideoSettings* _videoSettings = nullptr;
QString _videoSourceID;
bool _fullScreen = false;
Vehicle* _activeVehicle = nullptr;
};
#endif

21
src/VideoReceiver/GStreamer.cc
Unescape View File

@ -105,9 +105,28 @@ static void qgcputenv(const QString& key, const QString& root, const QString& pa @@ -105,9 +105,28 @@ static void qgcputenv(const QString& key, const QString& root, const QString& pa
}
#endif
static void
blacklist()
{
GstRegistry* reg;
if ((reg = gst_registry_get()) == nullptr) {
return;
}
GstPluginFeature* plugin;
if ((plugin = gst_registry_lookup_feature(reg, "bcmdec")) != nullptr) {
qCCritical(GStreamerLog) << "Disable bcmdec";
gst_plugin_feature_set_rank(plugin, GST_RANK_NONE);
}
}
void
GStreamer::initialize(int argc, char* argv[], int debuglevel)
{
qRegisterMetaType<VideoReceiver::STATUS>("STATUS");
#ifdef Q_OS_MAC
#ifdef QGC_INSTALL_RELEASE
QString currentDir = QCoreApplication::applicationDirPath();
@ -171,6 +190,8 @@ GStreamer::initialize(int argc, char* argv[], int debuglevel) @@ -171,6 +190,8 @@ GStreamer::initialize(int argc, char* argv[], int debuglevel)
gst_ios_post_init();
#endif
blacklist();
/* the plugin must be loaded before loading the qml file to register the
* GstGLVideoItem qml item
* FIXME Add a QQmlExtensionPlugin into qmlglsink to register GstGLVideoItem

493
src/VideoReceiver/GstVideoReceiver.cc
Unescape View File

@ -35,6 +35,9 @@ QGC_LOGGING_CATEGORY(VideoReceiverLog, "VideoReceiverLog") @@ -35,6 +35,9 @@ QGC_LOGGING_CATEGORY(VideoReceiverLog, "VideoReceiverLog")
GstVideoReceiver::GstVideoReceiver(QObject* parent)
: VideoReceiver(parent)
, _streaming(false)
, _decoding(false)
, _recording(false)
, _removingDecoder(false)
, _removingRecorder(false)
, _source(nullptr)
@ -50,47 +53,54 @@ GstVideoReceiver::GstVideoReceiver(QObject* parent) @@ -50,47 +53,54 @@ GstVideoReceiver::GstVideoReceiver(QObject* parent)
, _resetVideoSink(true)
, _videoSinkProbeId(0)
, _udpReconnect_us(5000000)
, _signalDepth(0)
, _endOfStream(false)
{
_apiHandler.start();
_notificationHandler.start();
_slotHandler.start();
connect(&_watchdogTimer, &QTimer::timeout, this, &GstVideoReceiver::_watchdog);
_watchdogTimer.start(1000);
}
GstVideoReceiver::~GstVideoReceiver(void)
{
stop();
_notificationHandler.shutdown();
_apiHandler.shutdown();
_slotHandler.shutdown();
}
void
GstVideoReceiver::start(const QString& uri, unsigned timeout)
GstVideoReceiver::start(const QString& uri, unsigned timeout, int buffer)
{
if (_apiHandler.needDispatch()) {
_apiHandler.dispatch([this, uri, timeout]() {
start(uri, timeout);
if (_needDispatch()) {
QString cachedUri = uri;
_slotHandler.dispatch([this, cachedUri, timeout, buffer]() {
start(cachedUri, timeout, buffer);
});
return;
}
if(_pipeline) {
qCDebug(VideoReceiverLog) << "Already running!";
qCDebug(VideoReceiverLog) << "Already running!" << _uri;
_dispatchSignal([this](){
emit onStartComplete(STATUS_INVALID_STATE);
});
return;
}
if (uri.isEmpty()) {
qCDebug(VideoReceiverLog) << "Failed because URI is not specified";
_dispatchSignal([this](){
emit onStartComplete(STATUS_INVALID_URL);
});
return;
}
qCDebug(VideoReceiverLog) << "Starting";
_uri = uri;
_timeout = timeout;
_buffer = buffer;
qCDebug(VideoReceiverLog) << "Starting" << _uri << ", buffer" << _buffer;
_endOfStream = false;
_timeout = timeout;
bool running = false;
bool pipelineUp = false;
@ -152,12 +162,35 @@ GstVideoReceiver::start(const QString& uri, unsigned timeout) @@ -152,12 +162,35 @@ GstVideoReceiver::start(const QString& uri, unsigned timeout)
break;
}
g_signal_connect(_source, "pad-added", G_CALLBACK(_onNewPad), this);
gst_bin_add_many(GST_BIN(_pipeline), _source, _tee, decoderQueue, _decoderValve, recorderQueue, _recorderValve, nullptr);
pipelineUp = true;
GstPad* srcPad = nullptr;
GstIterator* it;
if ((it = gst_element_iterate_src_pads(_source)) != nullptr) {
GValue vpad = G_VALUE_INIT;
if (gst_iterator_next(it, &vpad) == GST_ITERATOR_OK) {
srcPad = GST_PAD(g_value_get_object(&vpad));
gst_object_ref(srcPad);
g_value_reset(&vpad);
}
gst_iterator_free(it);
it = nullptr;
}
if (srcPad != nullptr) {
_onNewSourcePad(srcPad);
gst_object_unref(srcPad);
srcPad = nullptr;
} else {
g_signal_connect(_source, "pad-added", G_CALLBACK(_onNewPad), this);
}
if(!gst_element_link_many(_tee, decoderQueue, _decoderValve, nullptr)) {
qCCritical(VideoReceiverLog) << "Unable to link decoder queue";
break;
@ -223,23 +256,36 @@ GstVideoReceiver::start(const QString& uri, unsigned timeout) @@ -223,23 +256,36 @@ GstVideoReceiver::start(const QString& uri, unsigned timeout)
_source = nullptr;
}
}
_dispatchSignal([this](){
emit onStartComplete(STATUS_FAIL);
});
} else {
GST_DEBUG_BIN_TO_DOT_FILE(GST_BIN(_pipeline), GST_DEBUG_GRAPH_SHOW_ALL, "pipeline-started");
qCDebug(VideoReceiverLog) << "Started";
qCDebug(VideoReceiverLog) << "Started" << _uri;
_dispatchSignal([this](){
emit onStartComplete(STATUS_OK);
});
}
}
void
GstVideoReceiver::stop(void)
{
if (_apiHandler.needDispatch()) {
_apiHandler.dispatch([this]() {
if (_needDispatch()) {
_slotHandler.dispatch([this]() {
stop();
});
return;
}
qCDebug(VideoReceiverLog) << "Stopping";
if (_uri.isEmpty()) {
qCWarning(VideoReceiverLog) << "Stop called on empty URI";
return;
}
qCDebug(VideoReceiverLog) << "Stopping" << _uri;
if (_pipeline != nullptr) {
GstBus* bus;
@ -247,21 +293,29 @@ GstVideoReceiver::stop(void) @@ -247,21 +293,29 @@ GstVideoReceiver::stop(void)
if ((bus = gst_pipeline_get_bus(GST_PIPELINE(_pipeline))) != nullptr) {
gst_bus_disable_sync_message_emission(bus);
gst_element_send_event(_pipeline, gst_event_new_eos());
g_signal_handlers_disconnect_by_data(bus, this);
GstMessage* msg;
gboolean recordingValveClosed = TRUE;
if((msg = gst_bus_timed_pop_filtered(bus, GST_CLOCK_TIME_NONE, (GstMessageType)(GST_MESSAGE_EOS|GST_MESSAGE_ERROR))) != nullptr) {
if(GST_MESSAGE_TYPE(msg) == GST_MESSAGE_ERROR) {
qCCritical(VideoReceiverLog) << "Error stopping pipeline!";
} else if(GST_MESSAGE_TYPE(msg) == GST_MESSAGE_EOS) {
qCDebug(VideoReceiverLog) << "End of stream received!";
}
g_object_get(_recorderValve, "drop", &recordingValveClosed, nullptr);
gst_message_unref(msg);
msg = nullptr;
} else {
qCCritical(VideoReceiverLog) << "gst_bus_timed_pop_filtered() failed";
if (recordingValveClosed == FALSE) {
gst_element_send_event(_pipeline, gst_event_new_eos());
GstMessage* msg;
if((msg = gst_bus_timed_pop_filtered(bus, GST_CLOCK_TIME_NONE, (GstMessageType)(GST_MESSAGE_EOS|GST_MESSAGE_ERROR))) != nullptr) {
if(GST_MESSAGE_TYPE(msg) == GST_MESSAGE_ERROR) {
qCCritical(VideoReceiverLog) << "Error stopping pipeline!";
} else if(GST_MESSAGE_TYPE(msg) == GST_MESSAGE_EOS) {
qCDebug(VideoReceiverLog) << "End of stream received!";
}
gst_message_unref(msg);
msg = nullptr;
} else {
qCCritical(VideoReceiverLog) << "gst_bus_timed_pop_filtered() failed";
}
}
gst_object_unref(bus);
@ -295,35 +349,41 @@ GstVideoReceiver::stop(void) @@ -295,35 +349,41 @@ GstVideoReceiver::stop(void)
if (_streaming) {
_streaming = false;
qCDebug(VideoReceiverLog) << "Streaming stopped";
_notificationHandler.dispatch([this](){
emit streamingChanged();
qCDebug(VideoReceiverLog) << "Streaming stopped" << _uri;
_dispatchSignal([this](){
emit streamingChanged(_streaming);
});
} else {
qCDebug(VideoReceiverLog) << "Streaming did not start" << _uri;
}
}
qCDebug(VideoReceiverLog) << "Stopped";
qCDebug(VideoReceiverLog) << "Stopped" << _uri;
_dispatchSignal([this](){
emit onStopComplete(STATUS_OK);
});
}
void
GstVideoReceiver::startDecoding(void* sink)
{
if (sink == nullptr) {
qCCritical(VideoReceiverLog) << "VideoSink is NULL";
qCCritical(VideoReceiverLog) << "VideoSink is NULL" << _uri;
return;
}
if (_apiHandler.needDispatch()) {
if (_needDispatch()) {
GstElement* videoSink = GST_ELEMENT(sink);
gst_object_ref(videoSink);
_apiHandler.dispatch([this, videoSink]() mutable {
_slotHandler.dispatch([this, videoSink]() mutable {
startDecoding(videoSink);
gst_object_unref(videoSink);
});
return;
}
qCDebug(VideoReceiverLog) << "Starting decoding";
qCDebug(VideoReceiverLog) << "Starting decoding" << _uri;
if (_pipeline == nullptr) {
if (_videoSink != nullptr) {
@ -335,14 +395,20 @@ GstVideoReceiver::startDecoding(void* sink) @@ -335,14 +395,20 @@ GstVideoReceiver::startDecoding(void* sink)
GstElement* videoSink = GST_ELEMENT(sink);
if(_videoSink != nullptr || _decoding) {
qCDebug(VideoReceiverLog) << "Already decoding!";
qCDebug(VideoReceiverLog) << "Already decoding!" << _uri;
_dispatchSignal([this](){
emit onStartDecodingComplete(STATUS_INVALID_STATE);
});
return;
}
GstPad* pad;
if ((pad = gst_element_get_static_pad(videoSink, "sink")) == nullptr) {
qCCritical(VideoReceiverLog) << "Unable to find sink pad of video sink";
qCCritical(VideoReceiverLog) << "Unable to find sink pad of video sink" << _uri;
_dispatchSignal([this](){
emit onStartDecodingComplete(STATUS_FAIL);
});
return;
}
@ -359,34 +425,47 @@ GstVideoReceiver::startDecoding(void* sink) @@ -359,34 +425,47 @@ GstVideoReceiver::startDecoding(void* sink)
_removingDecoder = false;
if (!_streaming) {
_dispatchSignal([this](){
emit onStartDecodingComplete(STATUS_OK);
});
return;
}
if (!_addDecoder(_decoderValve)) {
qCCritical(VideoReceiverLog) << "_addDecoder() failed";
qCCritical(VideoReceiverLog) << "_addDecoder() failed" << _uri;
_dispatchSignal([this](){
emit onStartDecodingComplete(STATUS_FAIL);
});
return;
}
g_object_set(_decoderValve, "drop", FALSE, nullptr);
qCDebug(VideoReceiverLog) << "Decoding started";
qCDebug(VideoReceiverLog) << "Decoding started" << _uri;
_dispatchSignal([this](){
emit onStartDecodingComplete(STATUS_OK);
});
}
void
GstVideoReceiver::stopDecoding(void)
{
if (_apiHandler.needDispatch()) {
_apiHandler.dispatch([this]() {
if (_needDispatch()) {
_slotHandler.dispatch([this]() {
stopDecoding();
});
return;
}
qCDebug(VideoReceiverLog) << "Stopping decoding";
qCDebug(VideoReceiverLog) << "Stopping decoding" << _uri;
// exit immediately if we are not decoding
if (_pipeline == nullptr || !_decoding) {
qCDebug(VideoReceiverLog) << "Not decoding!";
qCDebug(VideoReceiverLog) << "Not decoding!" << _uri;
_dispatchSignal([this](){
emit onStopDecodingComplete(STATUS_INVALID_STATE);
});
return;
}
@ -394,35 +473,51 @@ GstVideoReceiver::stopDecoding(void) @@ -394,35 +473,51 @@ GstVideoReceiver::stopDecoding(void)
_removingDecoder = true;
_unlinkBranch(_decoderValve);
bool ret = _unlinkBranch(_decoderValve);
// FIXME: AV: it is much better to emit onStopDecodingComplete() after decoding is really stopped
// (which happens later due to async design) but as for now it is also not so bad...
_dispatchSignal([this, ret](){
emit onStopDecodingComplete(ret ? STATUS_OK : STATUS_FAIL);
});
}
void
GstVideoReceiver::startRecording(const QString& videoFile, FILE_FORMAT format)
{
if (_apiHandler.needDispatch()) {
_apiHandler.dispatch([this, videoFile, format]() {
startRecording(videoFile, format);
if (_needDispatch()) {
QString cachedVideoFile = videoFile;
_slotHandler.dispatch([this, cachedVideoFile, format]() {
startRecording(cachedVideoFile, format);
});
return;
}
qCDebug(VideoReceiverLog) << "Starting recording";
qCDebug(VideoReceiverLog) << "Starting recording" << _uri;
if (_pipeline == nullptr) {
qCDebug(VideoReceiverLog) << "Streaming is not active!";
qCDebug(VideoReceiverLog) << "Streaming is not active!" << _uri;
_dispatchSignal([this](){
emit onStartRecordingComplete(STATUS_INVALID_STATE);
});
return;
}
if (_recording) {
qCDebug(VideoReceiverLog) << "Already recording!";
qCDebug(VideoReceiverLog) << "Already recording!" << _uri;
_dispatchSignal([this](){
emit onStartRecordingComplete(STATUS_INVALID_STATE);
});
return;
}
qCDebug(VideoReceiverLog) << "New video file:" << videoFile;
qCDebug(VideoReceiverLog) << "New video file:" << videoFile << "" << _uri;
if ((_fileSink = _makeFileSink(videoFile, format)) == nullptr) {
qCCritical(VideoReceiverLog) << "_makeFileSink() failed";
qCCritical(VideoReceiverLog) << "_makeFileSink() failed" << _uri;
_dispatchSignal([this](){
emit onStartRecordingComplete(STATUS_FAIL);
});
return;
}
@ -433,7 +528,10 @@ GstVideoReceiver::startRecording(const QString& videoFile, FILE_FORMAT format) @@ -433,7 +528,10 @@ GstVideoReceiver::startRecording(const QString& videoFile, FILE_FORMAT format)
gst_bin_add(GST_BIN(_pipeline), _fileSink);
if (!gst_element_link(_recorderValve, _fileSink)) {
qCCritical(VideoReceiverLog) << "Failed to link valve and file sink";
qCCritical(VideoReceiverLog) << "Failed to link valve and file sink" << _uri;
_dispatchSignal([this](){
emit onStartRecordingComplete(STATUS_FAIL);
});
return;
}
@ -447,7 +545,10 @@ GstVideoReceiver::startRecording(const QString& videoFile, FILE_FORMAT format) @@ -447,7 +545,10 @@ GstVideoReceiver::startRecording(const QString& videoFile, FILE_FORMAT format)
GstPad* probepad;
if ((probepad = gst_element_get_static_pad(_recorderValve, "src")) == nullptr) {
qCCritical(VideoReceiverLog) << "gst_element_get_static_pad() failed";
qCCritical(VideoReceiverLog) << "gst_element_get_static_pad() failed" << _uri;
_dispatchSignal([this](){
emit onStartRecordingComplete(STATUS_FAIL);
});
return;
}
@ -458,9 +559,10 @@ GstVideoReceiver::startRecording(const QString& videoFile, FILE_FORMAT format) @@ -458,9 +559,10 @@ GstVideoReceiver::startRecording(const QString& videoFile, FILE_FORMAT format)
g_object_set(_recorderValve, "drop", FALSE, nullptr);
_recording = true;
qCDebug(VideoReceiverLog) << "Recording started";
_notificationHandler.dispatch([this](){
emit recordingChanged();
qCDebug(VideoReceiverLog) << "Recording started" << _uri;
_dispatchSignal([this](){
emit onStartRecordingComplete(STATUS_OK);
emit recordingChanged(_recording);
});
}
@ -468,18 +570,21 @@ GstVideoReceiver::startRecording(const QString& videoFile, FILE_FORMAT format) @@ -468,18 +570,21 @@ GstVideoReceiver::startRecording(const QString& videoFile, FILE_FORMAT format)
void
GstVideoReceiver::stopRecording(void)
{
if (_apiHandler.needDispatch()) {
_apiHandler.dispatch([this]() {
if (_needDispatch()) {
_slotHandler.dispatch([this]() {
stopRecording();
});
return;
}
qCDebug(VideoReceiverLog) << "Stopping recording";
qCDebug(VideoReceiverLog) << "Stopping recording" << _uri;
// exit immediately if we are not recording
if (_pipeline == nullptr || !_recording) {
qCDebug(VideoReceiverLog) << "Not recording!";
qCDebug(VideoReceiverLog) << "Not recording!" << _uri;
_dispatchSignal([this](){
emit onStopRecordingComplete(STATUS_INVALID_STATE);
});
return;
}
@ -487,22 +592,29 @@ GstVideoReceiver::stopRecording(void) @@ -487,22 +592,29 @@ GstVideoReceiver::stopRecording(void)
_removingRecorder = true;
_unlinkBranch(_recorderValve);
bool ret = _unlinkBranch(_recorderValve);
// FIXME: AV: it is much better to emit onStopRecordingComplete() after recording is really stopped
// (which happens later due to async design) but as for now it is also not so bad...
_dispatchSignal([this, ret](){
emit onStopRecordingComplete(ret ? STATUS_OK : STATUS_FAIL);
});
}
void
GstVideoReceiver::takeScreenshot(const QString& imageFile)
{
if (_apiHandler.needDispatch()) {
_apiHandler.dispatch([this, imageFile]() {
takeScreenshot(imageFile);
if (_needDispatch()) {
QString cachedImageFile = imageFile;
_slotHandler.dispatch([this, cachedImageFile]() {
takeScreenshot(cachedImageFile);
});
return;
}
// FIXME: AV: record screenshot here
_notificationHandler.dispatch([this](){
emit screenshotComplete();
_dispatchSignal([this](){
emit onTakeScreenshotComplete(STATUS_NOT_IMPLEMENTED);
});
}
@ -515,7 +627,7 @@ const char* GstVideoReceiver::_kFileMux[FILE_FORMAT_MAX - FILE_FORMAT_MIN] = { @@ -515,7 +627,7 @@ const char* GstVideoReceiver::_kFileMux[FILE_FORMAT_MAX - FILE_FORMAT_MIN] = {
void
GstVideoReceiver::_watchdog(void)
{
_apiHandler.dispatch([this](){
_slotHandler.dispatch([this](){
if(_pipeline == nullptr) {
return;
}
@ -527,10 +639,11 @@ GstVideoReceiver::_watchdog(void) @@ -527,10 +639,11 @@ GstVideoReceiver::_watchdog(void)
}
if (now - _lastSourceFrameTime > _timeout) {
qCDebug(VideoReceiverLog) << "Stream timeout, no frames for " << now - _lastSourceFrameTime;
_notificationHandler.dispatch([this](){
qCDebug(VideoReceiverLog) << "Stream timeout, no frames for " << now - _lastSourceFrameTime << "" << _uri;
_dispatchSignal([this](){
emit timeout();
});
stop();
}
if (_decoding && !_removingDecoder) {
@ -539,10 +652,11 @@ GstVideoReceiver::_watchdog(void) @@ -539,10 +652,11 @@ GstVideoReceiver::_watchdog(void)
}
if (now - _lastVideoFrameTime > _timeout * 2) {
qCDebug(VideoReceiverLog) << "Video decoder timeout, no frames for " << now - _lastVideoFrameTime;
_notificationHandler.dispatch([this](){
qCDebug(VideoReceiverLog) << "Video decoder timeout, no frames for " << now - _lastVideoFrameTime << " " << _uri;
_dispatchSignal([this](){
emit timeout();
});
stop();
}
}
});
@ -552,7 +666,6 @@ void @@ -552,7 +666,6 @@ void
GstVideoReceiver::_handleEOS(void)
{
if(_pipeline == nullptr) {
qCWarning(VideoReceiverLog) << "We should not be here";
return;
}
@ -646,6 +759,8 @@ GstVideoReceiver::_makeSource(const QString& uri) @@ -646,6 +759,8 @@ GstVideoReceiver::_makeSource(const QString& uri)
break;
}
g_signal_connect(parser, "autoplug-query", G_CALLBACK(_filterParserCaps), nullptr);
gst_bin_add_many(GST_BIN(bin), source, parser, nullptr);
// FIXME: AV: Android does not determine MPEG2-TS via parsebin - have to explicitly state which demux to use
@ -672,7 +787,7 @@ GstVideoReceiver::_makeSource(const QString& uri) @@ -672,7 +787,7 @@ GstVideoReceiver::_makeSource(const QString& uri)
gst_element_foreach_src_pad(source, _padProbe, &probeRes);
if (probeRes & 1) {
if (probeRes & 2) {
if (probeRes & 2 && _buffer >= 0) {
if ((buffer = gst_element_factory_make("rtpjitterbuffer", nullptr)) == nullptr) {
qCCritical(VideoReceiverLog) << "gst_element_factory_make('rtpjitterbuffer') failed";
break;
@ -691,7 +806,7 @@ GstVideoReceiver::_makeSource(const QString& uri) @@ -691,7 +806,7 @@ GstVideoReceiver::_makeSource(const QString& uri)
}
}
} else {
g_signal_connect(source, "pad-added", G_CALLBACK(_linkPadWithOptionalBuffer), parser);
g_signal_connect(source, "pad-added", G_CALLBACK(_linkPad), parser);
}
g_signal_connect(parser, "pad-added", G_CALLBACK(_wrapWithGhostPad), nullptr);
@ -847,9 +962,9 @@ GstVideoReceiver::_onNewSourcePad(GstPad* pad) @@ -847,9 +962,9 @@ GstVideoReceiver::_onNewSourcePad(GstPad* pad)
if (!_streaming) {
_streaming = true;
qCDebug(VideoReceiverLog) << "Streaming started";
_notificationHandler.dispatch([this](){
emit streamingChanged();
qCDebug(VideoReceiverLog) << "Streaming started" << _uri;
_dispatchSignal([this](){
emit streamingChanged(_streaming);
});
}
@ -868,7 +983,7 @@ GstVideoReceiver::_onNewSourcePad(GstPad* pad) @@ -868,7 +983,7 @@ GstVideoReceiver::_onNewSourcePad(GstPad* pad)
g_object_set(_decoderValve, "drop", FALSE, nullptr);
qCDebug(VideoReceiverLog) << "Decoding started";
qCDebug(VideoReceiverLog) << "Decoding started" << _uri;
}
void
@ -876,6 +991,8 @@ GstVideoReceiver::_onNewDecoderPad(GstPad* pad) @@ -876,6 +991,8 @@ GstVideoReceiver::_onNewDecoderPad(GstPad* pad)
{
GST_DEBUG_BIN_TO_DOT_FILE(GST_BIN(_pipeline), GST_DEBUG_GRAPH_SHOW_ALL, "pipeline-with-new-decoder-pad");
qCDebug(VideoReceiverLog) << "_onNewDecoderPad" << _uri;
if (!_addVideoSink(pad)) {
qCCritical(VideoReceiverLog) << "_addVideoSink() failed";
}
@ -915,11 +1032,6 @@ GstVideoReceiver::_addDecoder(GstElement* src) @@ -915,11 +1032,6 @@ GstVideoReceiver::_addDecoder(GstElement* src)
gst_caps_unref(caps);
caps = nullptr;
// FIXME: AV: check if srcpad exists - if it does then no need to wait for new pad
// int probeRes = 0;
// gst_element_foreach_src_pad(source, _padProbe, &probeRes);
g_signal_connect(_decoder, "pad-added", G_CALLBACK(_onNewPad), this);
gst_bin_add(GST_BIN(_pipeline), _decoder);
gst_element_sync_state_with_parent(_decoder);
@ -931,6 +1043,31 @@ GstVideoReceiver::_addDecoder(GstElement* src) @@ -931,6 +1043,31 @@ GstVideoReceiver::_addDecoder(GstElement* src)
return false;
}
GstPad* srcPad = nullptr;
GstIterator* it;
if ((it = gst_element_iterate_src_pads(_decoder)) != nullptr) {
GValue vpad = G_VALUE_INIT;
if (gst_iterator_next(it, &vpad) == GST_ITERATOR_OK) {
srcPad = GST_PAD(g_value_get_object(&vpad));
gst_object_ref(srcPad);
g_value_reset(&vpad);
}
gst_iterator_free(it);
it = nullptr;
}
if (srcPad != nullptr) {
_onNewDecoderPad(srcPad);
gst_object_unref(srcPad);
srcPad = nullptr;
} else {
g_signal_connect(_decoder, "pad-added", G_CALLBACK(_onNewPad), this);
}
return true;
}
@ -955,6 +1092,8 @@ GstVideoReceiver::_addVideoSink(GstPad* pad) @@ -955,6 +1092,8 @@ GstVideoReceiver::_addVideoSink(GstPad* pad)
gst_element_sync_state_with_parent(_videoSink);
g_object_set(_videoSink, "sync", _buffer >= 0, NULL);
GST_DEBUG_BIN_TO_DOT_FILE(GST_BIN(_pipeline), GST_DEBUG_GRAPH_SHOW_ALL, "pipeline-with-videosink");
if (caps != nullptr) {
@ -964,21 +1103,19 @@ GstVideoReceiver::_addVideoSink(GstPad* pad) @@ -964,21 +1103,19 @@ GstVideoReceiver::_addVideoSink(GstPad* pad)
gint width, height;
gst_structure_get_int(s, "width", &width);
gst_structure_get_int(s, "height", &height);
_setVideoSize(QSize(width, height));
_dispatchSignal([this, width, height](){
emit videoSizeChanged(QSize(width, height));
});
}
gst_caps_unref(caps);
caps = nullptr;
} else {
_setVideoSize(QSize(0, 0));
_dispatchSignal([this](){
emit videoSizeChanged(QSize(0, 0));
});
}
_decoding = true;
qCDebug(VideoReceiverLog) << "Decoding started";
_notificationHandler.dispatch([this](){
emit decodingChanged();
});
return true;
}
@ -992,6 +1129,13 @@ void @@ -992,6 +1129,13 @@ void
GstVideoReceiver::_noteVideoSinkFrame(void)
{
_lastVideoFrameTime = QDateTime::currentSecsSinceEpoch();
if (!_decoding) {
_decoding = true;
qCDebug(VideoReceiverLog) << "Decoding started";
_dispatchSignal([this](){
emit decodingChanged(_decoding);
});
}
}
void
@ -1002,14 +1146,14 @@ GstVideoReceiver::_noteEndOfStream(void) @@ -1002,14 +1146,14 @@ GstVideoReceiver::_noteEndOfStream(void)
// -Unlink the branch from the src pad
// -Send an EOS event at the beginning of that branch
void
bool
GstVideoReceiver::_unlinkBranch(GstElement* from)
{
GstPad* src;
if ((src = gst_element_get_static_pad(from, "src")) == nullptr) {
qCCritical(VideoReceiverLog) << "gst_element_get_static_pad() failed";
return;
return false;
}
GstPad* sink;
@ -1018,7 +1162,7 @@ GstVideoReceiver::_unlinkBranch(GstElement* from) @@ -1018,7 +1162,7 @@ GstVideoReceiver::_unlinkBranch(GstElement* from)
gst_object_unref(src);
src = nullptr;
qCCritical(VideoReceiverLog) << "gst_pad_get_peer() failed";
return;
return false;
}
if (!gst_pad_unlink(src, sink)) {
@ -1027,7 +1171,7 @@ GstVideoReceiver::_unlinkBranch(GstElement* from) @@ -1027,7 +1171,7 @@ GstVideoReceiver::_unlinkBranch(GstElement* from)
gst_object_unref(sink);
sink = nullptr;
qCCritical(VideoReceiverLog) << "gst_pad_unlink() failed";
return;
return false;
}
gst_object_unref(src);
@ -1039,11 +1183,14 @@ GstVideoReceiver::_unlinkBranch(GstElement* from) @@ -1039,11 +1183,14 @@ GstVideoReceiver::_unlinkBranch(GstElement* from)
gst_object_unref(sink);
sink = nullptr;
if (ret) {
qCDebug(VideoReceiverLog) << "Branch EOS was sent";
} else {
if (!ret) {
qCCritical(VideoReceiverLog) << "Branch EOS was NOT sent";
return false;
}
qCDebug(VideoReceiverLog) << "Branch EOS was sent";
return true;
}
void
@ -1092,8 +1239,8 @@ GstVideoReceiver::_shutdownDecodingBranch(void) @@ -1092,8 +1239,8 @@ GstVideoReceiver::_shutdownDecodingBranch(void)
if (_decoding) {
_decoding = false;
qCDebug(VideoReceiverLog) << "Decoding stopped";
_notificationHandler.dispatch([this](){
emit decodingChanged();
_dispatchSignal([this](){
emit decodingChanged(_decoding);
});
}
@ -1113,14 +1260,28 @@ GstVideoReceiver::_shutdownRecordingBranch(void) @@ -1113,14 +1260,28 @@ GstVideoReceiver::_shutdownRecordingBranch(void)
if (_recording) {
_recording = false;
qCDebug(VideoReceiverLog) << "Recording stopped";
_notificationHandler.dispatch([this](){
emit recordingChanged();
_dispatchSignal([this](){
emit recordingChanged(_recording);
});
}
GST_DEBUG_BIN_TO_DOT_FILE(GST_BIN(_pipeline), GST_DEBUG_GRAPH_SHOW_ALL, "pipeline-recording-stopped");
}
bool
GstVideoReceiver::_needDispatch(void)
{
return _slotHandler.needDispatch();
}
void
GstVideoReceiver::_dispatchSignal(std::function<void()> emitter)
{
_signalDepth += 1;
emitter();
_signalDepth -= 1;
}
gboolean
GstVideoReceiver::_onBusMessage(GstBus* bus, GstMessage* msg, gpointer data)
{
@ -1142,18 +1303,20 @@ GstVideoReceiver::_onBusMessage(GstBus* bus, GstMessage* msg, gpointer data) @@ -1142,18 +1303,20 @@ GstVideoReceiver::_onBusMessage(GstBus* bus, GstMessage* msg, gpointer data)
}
if (error != nullptr) {
qCCritical(VideoReceiverLog) << error->message;
qCCritical(VideoReceiverLog) << "GStreamer error:" << error->message;
g_error_free(error);
error = nullptr;
}
pThis->_apiHandler.dispatch([pThis](){
pThis->_slotHandler.dispatch([pThis](){
qCDebug(VideoReceiverLog) << "Stopping because of error";
pThis->stop();
});
} while(0);
break;
case GST_MESSAGE_EOS:
pThis->_apiHandler.dispatch([pThis](){
pThis->_slotHandler.dispatch([pThis](){
qCDebug(VideoReceiverLog) << "Received EOS";
pThis->_handleEOS();
});
break;
@ -1174,7 +1337,8 @@ GstVideoReceiver::_onBusMessage(GstBus* bus, GstMessage* msg, gpointer data) @@ -1174,7 +1337,8 @@ GstVideoReceiver::_onBusMessage(GstBus* bus, GstMessage* msg, gpointer data)
}
if (GST_MESSAGE_TYPE(forward_msg) == GST_MESSAGE_EOS) {
pThis->_apiHandler.dispatch([pThis](){
pThis->_slotHandler.dispatch([pThis](){
qCDebug(VideoReceiverLog) << "Received branch EOS";
pThis->_handleEOS();
});
}
@ -1236,57 +1400,8 @@ GstVideoReceiver::_wrapWithGhostPad(GstElement* element, GstPad* pad, gpointer d @@ -1236,57 +1400,8 @@ GstVideoReceiver::_wrapWithGhostPad(GstElement* element, GstPad* pad, gpointer d
}
void
GstVideoReceiver::_linkPadWithOptionalBuffer(GstElement* element, GstPad* pad, gpointer data)
GstVideoReceiver::_linkPad(GstElement* element, GstPad* pad, gpointer data)
{
bool isRtpPad = false;
GstCaps* filter;
if ((filter = gst_caps_from_string("application/x-rtp")) != nullptr) {
GstCaps* caps = gst_pad_query_caps(pad, nullptr);
if (caps != nullptr) {
if (!gst_caps_is_any(caps) && gst_caps_can_intersect(caps, filter)) {
isRtpPad = true;
}
gst_caps_unref(caps);
caps = nullptr;
}
gst_caps_unref(filter);
filter = nullptr;
}
if (isRtpPad) {
GstElement* buffer;
if ((buffer = gst_element_factory_make("rtpjitterbuffer", nullptr)) != nullptr) {
gst_bin_add(GST_BIN(GST_ELEMENT_PARENT(element)), buffer);
gst_element_sync_state_with_parent(buffer);
GstPad* sinkpad = gst_element_get_static_pad(buffer, "sink");
if (sinkpad != nullptr) {
const GstPadLinkReturn ret = gst_pad_link(pad, sinkpad);
gst_object_unref(sinkpad);
sinkpad = nullptr;
if (ret == GST_PAD_LINK_OK) {
pad = gst_element_get_static_pad(buffer, "src");
element = buffer;
} else {
qCDebug(VideoReceiverLog) << "Partially failed - gst_pad_link()";
}
} else {
qCDebug(VideoReceiverLog) << "Partially failed - gst_element_get_static_pad()";
}
} else {
qCDebug(VideoReceiverLog) << "Partially failed - gst_element_factory_make('rtpjitterbuffer')";
}
}
gchar* name;
if ((name = gst_pad_get_name(pad)) != nullptr) {
@ -1332,6 +1447,58 @@ GstVideoReceiver::_padProbe(GstElement* element, GstPad* pad, gpointer user_data @@ -1332,6 +1447,58 @@ GstVideoReceiver::_padProbe(GstElement* element, GstPad* pad, gpointer user_data
}
gboolean
GstVideoReceiver::_filterParserCaps(GstElement* bin, GstPad* pad, GstElement* element, GstQuery* query, gpointer data)
{
Q_UNUSED(bin)
Q_UNUSED(pad)
Q_UNUSED(element)
Q_UNUSED(data)
if (GST_QUERY_TYPE(query) != GST_QUERY_CAPS) {
return FALSE;
}
GstCaps* srcCaps;
gst_query_parse_caps(query, &srcCaps);
if (srcCaps == nullptr || gst_caps_is_any(srcCaps)) {
return FALSE;
}
GstCaps* sinkCaps = nullptr;
GstCaps* filter;
if (sinkCaps == nullptr && (filter = gst_caps_from_string("video/x-h264")) != nullptr) {
if (gst_caps_can_intersect(srcCaps, filter)) {
sinkCaps = gst_caps_from_string("video/x-h264,stream-format=avc");
}
gst_caps_unref(filter);
filter = nullptr;
} else if (sinkCaps == nullptr && (filter = gst_caps_from_string("video/x-h265")) != nullptr) {
if (gst_caps_can_intersect(srcCaps, filter)) {
sinkCaps = gst_caps_from_string("video/x-h265,stream-format=hvc1");
}
gst_caps_unref(filter);
filter = nullptr;
}
if (sinkCaps == nullptr) {
return FALSE;
}
gst_query_set_caps_result(query, sinkCaps);
gst_caps_unref(sinkCaps);
sinkCaps = nullptr;
return TRUE;
}
gboolean
GstVideoReceiver::_autoplugQueryCaps(GstElement* bin, GstPad* pad, GstElement* element, GstQuery* query, gpointer data)
{
Q_UNUSED(bin)
@ -1505,7 +1672,9 @@ GstVideoReceiver::_keyframeWatch(GstPad* pad, GstPadProbeInfo* info, gpointer us @@ -1505,7 +1672,9 @@ GstVideoReceiver::_keyframeWatch(GstPad* pad, GstPadProbeInfo* info, gpointer us
qCDebug(VideoReceiverLog) << "Got keyframe, stop dropping buffers";
pThis->recordingStarted();
pThis->_dispatchSignal([pThis]() {
pThis->recordingStarted();
});
return GST_PAD_PROBE_REMOVE;
}

25
src/VideoReceiver/GstVideoReceiver.h
Unescape View File

@ -89,7 +89,7 @@ public: @@ -89,7 +89,7 @@ public:
~GstVideoReceiver(void);
public slots:
virtual void start(const QString& uri, unsigned timeout);
virtual void start(const QString& uri, unsigned timeout, int buffer = 0);
virtual void stop(void);
virtual void startDecoding(void* sink);
virtual void stopDecoding(void);
@ -102,11 +102,6 @@ protected slots: @@ -102,11 +102,6 @@ protected slots:
virtual void _handleEOS(void);
protected:
void _setVideoSize(const QSize& size) {
_videoSize = ((quint32)size.width() << 16) | (quint32)size.height();
emit videoSizeChanged();
}
virtual GstElement* _makeSource(const QString& uri);
virtual GstElement* _makeDecoder(GstCaps* caps, GstElement* videoSink);
virtual GstElement* _makeFileSink(const QString& videoFile, FILE_FORMAT format);
@ -118,16 +113,19 @@ protected: @@ -118,16 +113,19 @@ protected:
virtual void _noteTeeFrame(void);
virtual void _noteVideoSinkFrame(void);
virtual void _noteEndOfStream(void);
virtual void _unlinkBranch(GstElement* from);
virtual bool _unlinkBranch(GstElement* from);
virtual void _shutdownDecodingBranch (void);
virtual void _shutdownRecordingBranch(void);
private:
bool _needDispatch(void);
void _dispatchSignal(std::function<void()> emitter);
static gboolean _onBusMessage(GstBus* bus, GstMessage* message, gpointer user_data);
static void _onNewPad(GstElement* element, GstPad* pad, gpointer data);
static void _wrapWithGhostPad(GstElement* element, GstPad* pad, gpointer data);
static void _linkPadWithOptionalBuffer(GstElement* element, GstPad* pad, gpointer data);
static void _linkPad(GstElement* element, GstPad* pad, gpointer data);
static gboolean _padProbe(GstElement* element, GstPad* pad, gpointer user_data);
static gboolean _filterParserCaps(GstElement* bin, GstPad* pad, GstElement* element, GstQuery* query, gpointer data);
static gboolean _autoplugQueryCaps(GstElement* bin, GstPad* pad, GstElement* element, GstQuery* query, gpointer data);
static gboolean _autoplugQueryContext(GstElement* bin, GstPad* pad, GstElement* element, GstQuery* query, gpointer data);
static gboolean _autoplugQuery(GstElement* bin, GstPad* pad, GstElement* element, GstQuery* query, gpointer data);
@ -136,6 +134,9 @@ private: @@ -136,6 +134,9 @@ private:
static GstPadProbeReturn _eosProbe(GstPad* pad, GstPadProbeInfo* info, gpointer user_data);
static GstPadProbeReturn _keyframeWatch(GstPad* pad, GstPadProbeInfo* info, gpointer user_data);
bool _streaming;
bool _decoding;
bool _recording;
bool _removingDecoder;
bool _removingRecorder;
GstElement* _source;
@ -157,10 +158,12 @@ private: @@ -157,10 +158,12 @@ private:
//-- RTSP UDP reconnect timeout
uint64_t _udpReconnect_us;
QString _uri;
unsigned _timeout;
int _buffer;
Worker _apiHandler;
Worker _notificationHandler;
Worker _slotHandler;
uint32_t _signalDepth;
bool _endOfStream;

2
src/VideoReceiver/README.md
Unescape View File

@ -39,7 +39,7 @@ gst-launch-1.0 videotestsrc ! video/x-raw,width=640,height=480 ! videoconvert ! @@ -39,7 +39,7 @@ gst-launch-1.0 videotestsrc ! video/x-raw,width=640,height=480 ! videoconvert !
On the receiving end, if you want to test it from the command line, you can use something like:
```
gst-launch-1.0 udpsrc port=5600 caps='application/x-rtp, media=(string)video, clock-rate=(int)90000, encoding-name=(string)H264' ! rtph264depay ! h264parse ! avdec_h264 ! autovideosink fps-update-interval=1000 sync=false
gst-launch-1.0 udpsrc port=5600 caps='application/x-rtp, media=(string)video, clock-rate=(int)90000, encoding-name=(string)H264' ! rtpjitterbuffer ! rtph264depay ! h264parse ! avdec_h264 ! autovideosink fps-update-interval=1000 sync=false
```
### Additional Protocols

65
src/VideoReceiver/VideoReceiver.h
Unescape View File

@ -17,9 +17,6 @@ @@ -17,9 +17,6 @@
#include <QObject>
#include <QSize>
#include <QQuickItem>
#include <atomic>
class VideoReceiver : public QObject
{
@ -28,10 +25,6 @@ class VideoReceiver : public QObject @@ -28,10 +25,6 @@ class VideoReceiver : public QObject
public:
explicit VideoReceiver(QObject* parent = nullptr)
: QObject(parent)
, _streaming(false)
, _decoding(false)
, _recording(false)
, _videoSize(0)
{}
virtual ~VideoReceiver(void) {}
@ -44,50 +37,42 @@ public: @@ -44,50 +37,42 @@ public:
FILE_FORMAT_MAX
} FILE_FORMAT;
Q_PROPERTY(bool streaming READ streaming NOTIFY streamingChanged)
Q_PROPERTY(bool decoding READ decoding NOTIFY decodingChanged)
Q_PROPERTY(bool recording READ recording NOTIFY recordingChanged)
Q_PROPERTY(QSize videoSize READ videoSize NOTIFY videoSizeChanged)
bool streaming(void) {
return _streaming;
}
bool decoding(void) {
return _decoding;
}
bool recording(void) {
return _recording;
}
typedef enum {
STATUS_OK = 0,
STATUS_FAIL,
STATUS_INVALID_STATE,
STATUS_INVALID_URL,
STATUS_NOT_IMPLEMENTED
} STATUS;
QSize videoSize(void) {
const quint32 size = _videoSize;
return QSize((size >> 16) & 0xFFFF, size & 0xFFFF);
}
Q_ENUM(STATUS)
signals:
void timeout(void);
void streamingChanged(void);
void decodingChanged(void);
void recordingChanged(void);
void streamingChanged(bool active);
void decodingChanged(bool active);
void recordingChanged(bool active);
void recordingStarted(void);
void videoSizeChanged(void);
void screenshotComplete(void);
void videoSizeChanged(QSize size);
void onStartComplete(STATUS status);
void onStopComplete(STATUS status);
void onStartDecodingComplete(STATUS status);
void onStopDecodingComplete(STATUS status);
void onStartRecordingComplete(STATUS status);
void onStopRecordingComplete(STATUS status);
void onTakeScreenshotComplete(STATUS status);
public slots:
virtual void start(const QString& uri, unsigned timeout) = 0;
// buffer:
// -1 - disable buffer and video sync
// 0 - default buffer length
// N - buffer length, ms
virtual void start(const QString& uri, unsigned timeout, int buffer = 0) = 0;
virtual void stop(void) = 0;
virtual void startDecoding(void* sink) = 0;
virtual void stopDecoding(void) = 0;
virtual void startRecording(const QString& videoFile, FILE_FORMAT format) = 0;
virtual void stopRecording(void) = 0;
virtual void takeScreenshot(const QString& imageFile) = 0;
protected:
std::atomic<bool> _streaming;
std::atomic<bool> _decoding;
std::atomic<bool> _recording;
std::atomic<quint32>_videoSize;
};

9
src/api/QGCCorePlugin.cc
Unescape View File

@ -485,6 +485,15 @@ void* QGCCorePlugin::createVideoSink(QObject* parent, QQuickItem* widget) @@ -485,6 +485,15 @@ void* QGCCorePlugin::createVideoSink(QObject* parent, QQuickItem* widget)
#endif
}
void QGCCorePlugin::releaseVideoSink(void* sink)
{
#if defined(QGC_GST_STREAMING)
GStreamer::releaseVideoSink(sink);
#else
Q_UNUSED(sink)
#endif
}
bool QGCCorePlugin::guidedActionsControllerLogging() const
{
return GuidedActionsControllerLog().isDebugEnabled();

2
src/api/QGCCorePlugin.h
Unescape View File

@ -119,6 +119,8 @@ public: @@ -119,6 +119,8 @@ public:
virtual VideoReceiver* createVideoReceiver(QObject* parent);
/// Allows the plugin to override the creation of VideoSink.
virtual void* createVideoSink(QObject* parent, QQuickItem* widget);
/// Allows the plugin to override the release of VideoSink.
virtual void releaseVideoSink(void* sink);
/// Allows the plugin to see all mavlink traffic to a vehicle
/// @return true: Allow vehicle to continue processing, false: Vehicle should not process message

Write Preview
Loading…
Cancel
Save