We are developing a camera streaming application using Qt 5.6.3 on Processor Renesas RZG2N and PACKAGE VLP3.0.5. We are able to successfully integrate camera and capture live streaming using the below command directly using gstreamer 1.0
gst-launch-1.0 rtspsrc location="rtsp://admin:[email protected]:554/Streaming/Channels/101" latency=300 ! rtph264depay ! h264parse ! decodebin ! Autovideosink
We need to use QML to disply the video in the Qt application. Since we need to customize the gstreamer pipeline, we are calling gstreamer API's from backend in C++.
As per the example provided in the gst-plugins-good module, we are using qmlglsink as the video sink for QML.
https://github.com/GStreamer/gst-plugins-good/blob/1.18/tests/examples/qt/qmlsink/main.cpp
The sample application we developed based on the above link is also attached here.
CameraStream.zip
However, when we execute the application, the qmlglsink element creation fails.
On further analysis, it looks like the plugin is missing as per below screenshot.
As per the below stack overflow link, we observed that we need to add the below plugins in the gstreamer package.
https://stackoverflow.com/questions/56122402/display-video-in-qml-via-gstreamer-qmlglsink-plugin
Please correct our understanding on the same. Based on this, we tried to build the image with below components
Core-image-qt
gstreamer1.0
gstreamer1.0-plugins-base
gstreamer1.0-plugins-good
gstreamer1.0-plugins-bad
gstreamer1.0-plugins-ugly
Even though build is successful, we get below warnings while building
WARNING: gstreamer1.0-omx-1.16.3-r0 do_patch: Fuzz detected: Applying patch 0001-omxvideodec-don-t-drop-frame-if-it-contains-header-d.patch patching file omx/gstomxvideodec.c Hunk #1 succeeded at 3083 with fuzz 1 (offset 560 lines). The context lines in the patches can be updated with devtool: devtool modify gstreamer1.0-omx devtool finish --force-patch-refresh gstreamer1.0-omx <layer_path> Don't forget to review changes done by devtool! WARNING: gstreamer1.0-omx-1.16.3-r0 do_patch: QA Issue: Patch log indicates that patches do not apply cleanly. [patch-fuzz] WARNING: lttng-modules-2.11.9-r0 do_package: lttng-modules: no modules were created; this may be due to CONFIG_TRACEPOINTS not being enabled in your kernel. WARNING: qtmultimedia-5.6.3+gitAUTOINC+4dd780a81e-r0 do_package_qa: QA Issue: qtmultimedia-qmlplugins-dbg: found library in wrong location: /usr/share/qt5/examples/multimedia/spectrum/.debug/libfftreal.so.1.0.0 [libdir]
However, when we try to include the plugin gstreamer1.0-qt5, we get the following error,
vishnu@vishnu-Latitude-3520:~/rzg_vlp_v3.0.5/build$ MACHINE=hihope-rzg2n bitbake core-image-qt WARNING: Layer qt5-layer should set LAYERSERIES_COMPAT_qt5-layer in its conf/layer.conf file to list the core layer names it is compatible with. WARNING: Layer qt5-layer should set LAYERSERIES_COMPAT_qt5-layer in its conf/layer.conf file to list the core layer names it is compatible with. Parsing recipes: 100% |#####################################################################################################################################################################| Time: 0:01:32 Parsing of 2172 .bb files complete (0 cached, 2172 parsed). 5480 targets, 877 skipped, 7 masked, 0 errors. NOTE: Resolving any missing task queue dependencies ERROR: Nothing RPROVIDES 'gstreamer1.0-qt5' (but /home/vishnu/rzg_vlp_v3.0.5/build/../meta-renesas/meta-rz-common/dynamic-layers/qt5-layer/images/core-image-qt.bb RDEPENDS on or otherwise requires it) NOTE: Runtime target 'gstreamer1.0-qt5' is unbuildable, removing... Missing or unbuildable dependency chain was: ['gstreamer1.0-qt5'] ERROR: Required build target 'core-image-qt' has no buildable providers. Missing or unbuildable dependency chain was: ['core-image-qt', 'gstreamer1.0-qt5'] Summary: There were 2 WARNING messages shown. Summary: There were 2 ERROR messages shown, returning a non-zero exit code.
Please guide us with the above errors and how to properly display video from QML using Qt5.6.3 and gstreamer 1.0
Please let me know if you need any further information.
Thanks & Regards,
Rohini
Hi Rohini,
"gstreamer1.0-qt5" does not exist in Yocto. It is a name used by another Linux distro.
For qtqmlsink, it belongs to gstreamer1.0-plugins-good.However, qt option in gstreamer1.0-plugins-good is off by default, so qtqmlsink is not available.You can try to enable it with below line in local.conf (then rebuid gstreamer1.0-plugins-good from configure):
PACKAGECONFIG_append_pn-gstreamer1.0-plugins-good = " qt5"
Ok HungT. Thanks for your reply. We will try it and get back to you.
Maybe you could also leave GStreamer as a separate application (not part of QML) since you are already controlling it using C code.
In Weston, you can then then force the GStreamer application to always be on top of the Qt application. Here is the instructions to do that: renesas.info/.../RZG_weston
While building with the above mentioned configuration enabled, we are getting other errors. The error log is attached in a file. Could you please have a look at it and tell if we are missing anything.
vishnu@vishnu-Latitude-3520:~/rzg_vlp_v3.0.5/build$ MACHINE=hihope-rzg2n bitbake core-image-qt WARNING: Layer qt5-layer should set LAYERSERIES_COMPAT_qt5-layer in its conf/layer.conf file to list the core layer names it is compatible with. WARNING: Layer qt5-layer should set LAYERSERIES_COMPAT_qt5-layer in its conf/layer.conf file to list the core layer names it is compatible with. Parsing recipes: 100% |#####################################################################################################################################################################| Time: 0:01:27 Parsing of 2172 .bb files complete (0 cached, 2172 parsed). 5480 targets, 877 skipped, 7 masked, 0 errors. NOTE: Resolving any missing task queue dependencies NOTE: Multiple providers are available for virtual/crypt (glibc, libxcrypt) Consider defining a PREFERRED_PROVIDER entry to match virtual/crypt NOTE: Multiple providers are available for virtual/nativesdk-crypt (nativesdk-glibc, nativesdk-libxcrypt) Consider defining a PREFERRED_PROVIDER entry to match virtual/nativesdk-crypt WARNING: preferred version 2.28% of glibc-mtrace not available (for item libc-mtrace) WARNING: versions of glibc-mtrace available: 2.31 NOTE: Multiple providers are available for virtual/lib32-crypt (lib32-glibc, lib32-libxcrypt) Consider defining a PREFERRED_PROVIDER entry to match virtual/lib32-crypt WARNING: preferred version 2.28% of glibc-mtrace not available (for item glibc-mtrace-dev) WARNING: versions of glibc-mtrace available: 2.31 WARNING: preferred version 2.28% of glibc-mtrace not available (for item glibc-mtrace) WARNING: versions of glibc-mtrace available: 2.31 Build Configuration: BB_VERSION = "1.46.0" BUILD_SYS = "x86_64-linux" NATIVELSBSTRING = "universal" TARGET_SYS = "aarch64-poky-linux" MACHINE = "hihope-rzg2n" DISTRO = "poky" DISTRO_VERSION = "3.1.26" TUNE_FEATURES = "aarch64 cortexa57 crc" TARGET_FPU = "" SOC_FAMILY = "rzg2:r8a774b1" SOC_FAMILY = "rzg2:r8a774b1" meta-gplv2 meta meta-poky meta-yocto-bsp meta-rz-common meta-rzg2h meta-oe meta-python meta-multimedia meta-rz-graphics meta-rz-codecs meta-qt5 meta-swupdate-dunfell = "<unknown>:<unknown>" WARNING: /home/vishnu/rzg_vlp_v3.0.5/build/../meta-renesas/meta-rz-common/recipes-kernel/linux/linux-renesas_5.10.bb:do_compile is tainted from a forced run | ETA: 0:00:00 Initialising tasks: 100% |##################################################################################################################################################################| Time: 0:00:06 Sstate summary: Wanted 32 Found 18 Missed 14 Current 2898 (56% match, 99% complete) NOTE: Executing Tasks ERROR: gstreamer1.0-plugins-good-1.16.3-r0 do_configure: meson failed ERROR: gstreamer1.0-plugins-good-1.16.3-r0 do_configure: Execution of '/home/vishnu/rzg_vlp_v3.0.5/build/tmp/work/aarch64-poky-linux/gstreamer1.0-plugins-good/1.16.3-r0/temp/run.do_configure.1258481' failed with exit code 1 ERROR: Logfile of failure stored in: /home/vishnu/rzg_vlp_v3.0.5/build/tmp/work/aarch64-poky-linux/gstreamer1.0-plugins-good/1.16.3-r0/temp/log.do_configure.1258481 Log data follows: | DEBUG: Executing shell function do_configure | NOTE: Executing meson -Dexamples=disabled -Dnls=enabled -Daalib=disabled -Ddirectsound=disabled -Ddv=disabled -Dlibcaca=disabled -Doss=enabled -Doss4=disabled -Dosxaudio=disabled -Dosxvideo=disabled -Dshout2=disabled -Dtwolame=disabled -Dwaveform=disabled -Dbz2=enabled -Dcairo=enabled -Ddv1394=disabled -Dflac=enabled -Dgdk-pixbuf=enabled -Dgtk3=disabled -Dv4l2-gudev=enabled -Djack=disabled -Djpeg=enabled -Dlame=enabled -Dpng=enabled -Dv4l2-libv4l2=disabled -Dmpg123=enabled -Dorc=enabled -Dpulse=enabled -Dqt5=enabled -Dsoup=enabled -Dspeex=enabled -Dtaglib=enabled -Dv4l2=enabled -Dv4l2-probe=true -Dvpx=disabled -Dwavpack=disabled -Dximagesrc=disabled -Dximagesrc-xshm=disabled -Dximagesrc-xfixes=disabled -Dximagesrc-xdamage=disabled -Dignore-fps-of-video-standard=true -Dcont-frame-capture=true ... | The Meson build system | Version: 0.53.2 | Source dir: /home/vishnu/rzg_vlp_v3.0.5/build/tmp/work/aarch64-poky-linux/gstreamer1.0-plugins-good/1.16.3-r0/git | Build dir: /home/vishnu/rzg_vlp_v3.0.5/build/tmp/work/aarch64-poky-linux/gstreamer1.0-plugins-good/1.16.3-r0/build | Build type: cross build | Project name: gst-plugins-good | Project version: 1.16.3 | C compiler for the build machine: gcc (gcc 9.4.0 "gcc (Ubuntu 9.4.0-1ubuntu1~20.04.2) 9.4.0") | C linker for the build machine: gcc ld.bfd 2.34 | C compiler for the host machine: aarch64-poky-linux-gcc -mcpu=cortex-a57+crc -fstack-protector-strong -D_FORTIFY_SOURCE=2 -Wformat -Wformat-security -Werror=format-security --sysroot=/home/vishnu/rzg_vlp_v3.0.5/build/tmp/work/aarch64-poky-linux/gstreamer1.0-plugins-good/1.16.3-r0/recipe-sysroot (gcc 8.3.0 "aarch64-poky-linux-gcc (GCC) 8.3.0") | C linker for the host machine: aarch64-poky-linux-gcc -mcpu=cortex-a57+crc -fstack-protector-strong -D_FORTIFY_SOURCE=2 -Wformat -Wformat-security -Werror=format-security --sysroot=/home/vishnu/rzg_vlp_v3.0.5/build/tmp/work/aarch64-poky-linux/gstreamer1.0-plugins-good/1.16.3-r0/recipe-sysroot ld.bfd 2.31.1 | Build machine cpu family: x86_64 | Build machine cpu: x86_64 | Host machine cpu family: aarch64 | Host machine cpu: aarch64 | Target machine cpu family: aarch64 | Target machine cpu: aarch64 | C++ compiler for the build machine: g++ (gcc 9.4.0 "g++ (Ubuntu 9.4.0-1ubuntu1~20.04.2) 9.4.0") | C++ linker for the build machine: g++ ld.bfd 2.34 | C++ compiler for the host machine: aarch64-poky-linux-g++ -mcpu=cortex-a57+crc -fstack-protector-strong -D_FORTIFY_SOURCE=2 -Wformat -Wformat-security -Werror=format-security --sysroot=/home/vishnu/rzg_vlp_v3.0.5/build/tmp/work/aarch64-poky-linux/gstreamer1.0-plugins-good/1.16.3-r0/recipe-sysroot (gcc 8.3.0 "aarch64-poky-linux-g++ (GCC) 8.3.0") | C++ linker for the host machine: aarch64-poky-linux-g++ -mcpu=cortex-a57+crc -fstack-protector-strong -D_FORTIFY_SOURCE=2 -Wformat -Wformat-security -Werror=format-security --sysroot=/home/vishnu/rzg_vlp_v3.0.5/build/tmp/work/aarch64-poky-linux/gstreamer1.0-plugins-good/1.16.3-r0/recipe-sysroot ld.bfd 2.31.1 | Compiler for C supports link arguments -Wl,-Bsymbolic-functions: YES | Compiler for C supports arguments -fvisibility=hidden: YES | Compiler for C supports arguments -fno-strict-aliasing: YES | Message: Disabling GLib cast checks | Has header "dlfcn.h" : YES | Has header "fcntl.h" : YES | Has header "inttypes.h" : YES | Has header "memory.h" : YES | Has header "process.h" : NO | Has header "stdint.h" : YES | Has header "stdlib.h" : YES | Has header "strings.h" : YES | Has header "string.h" : YES | Has header "sys/ioctl.h" : YES | Has header "sys/param.h" : YES | Has header "sys/socket.h" : YES | Has header "sys/stat.h" : YES | Has header "sys/time.h" : YES | Has header "sys/types.h" : YES | Has header "unistd.h" : YES | Library m found: YES | Checking for function "asinh" with dependency -lm: YES | Checking for function "clock_gettime" with dependency -lm: YES | Checking for function "cosh" with dependency -lm: YES | Checking for function "dcgettext" with dependency -lm: YES | Checking for function "getpagesize" with dependency -lm: YES | Checking for function "isinf" with dependency -lm: YES | Checking for function "mmap" with dependency -lm: YES | Checking for function "mmap64" with dependency -lm: NO | Checking for function "sinh" with dependency -lm: YES | Checking for size of "char" : 1 | Checking for size of "int" : 4 | Checking for size of "long" : 8 | Checking for size of "short" : 2 | Checking for size of "void*" : 8 | Checking for size of "off_t" : 8 | Compiler for C supports arguments -Wmissing-declarations: YES | Compiler for C++ supports arguments -Wmissing-declarations: YES | Compiler for C supports arguments -Wredundant-decls: YES | Compiler for C++ supports arguments -Wredundant-decls: YES | Compiler for C supports arguments -Wwrite-strings: YES | Compiler for C++ supports arguments -Wwrite-strings: YES | Compiler for C supports arguments -Winit-self: YES | Compiler for C++ supports arguments -Winit-self: YES | Compiler for C supports arguments -Wmissing-include-dirs: YES | Compiler for C++ supports arguments -Wmissing-include-dirs: YES | Compiler for C supports arguments -Wno-multichar -Wmultichar: YES | Compiler for C++ supports arguments -Wno-multichar -Wmultichar: YES | Compiler for C supports arguments -Wvla: YES | Compiler for C++ supports arguments -Wvla: YES | Compiler for C supports arguments -Wpointer-arith: YES | Compiler for C++ supports arguments -Wpointer-arith: YES | Compiler for C supports arguments -Wmissing-prototypes: YES | Compiler for C supports arguments -Wdeclaration-after-statement: YES | Compiler for C supports arguments -Wold-style-definition: YES | Compiler for C supports arguments -Waggregate-return: YES | Found pkg-config: /home/vishnu/rzg_vlp_v3.0.5/build/tmp/work/aarch64-poky-linux/gstreamer1.0-plugins-good/1.16.3-r0/recipe-sysroot-native/usr/bin/pkg-config (0.29) | Run-time dependency gstreamer-1.0 found: YES 1.16.3 | Run-time dependency gstreamer-base-1.0 found: YES 1.16.3 | Run-time dependency gstreamer-net-1.0 found: YES 1.16.3 | Run-time dependency gstreamer-controller-1.0 found: YES 1.16.3 | Run-time dependency gstreamer-check-1.0 found: YES 1.16.3 | Run-time dependency gstreamer-pbutils-1.0 found: YES 1.16.3 | Run-time dependency gstreamer-allocators-1.0 found: YES 1.16.3 | Run-time dependency gstreamer-app-1.0 found: YES 1.16.3 | Run-time dependency gstreamer-audio-1.0 found: YES 1.16.3 | Run-time dependency gstreamer-fft-1.0 found: YES 1.16.3 | Run-time dependency gstreamer-riff-1.0 found: YES 1.16.3 | Run-time dependency gstreamer-rtp-1.0 found: YES 1.16.3 | Run-time dependency gstreamer-rtsp-1.0 found: YES 1.16.3 | Run-time dependency gstreamer-sdp-1.0 found: YES 1.16.3 | Run-time dependency gstreamer-tag-1.0 found: YES 1.16.3 | Run-time dependency gstreamer-video-1.0 found: YES 1.16.3 | Run-time dependency gstreamer-gl-1.0 found: YES 1.16.3 | Message: GStreamer OpenGL window systems: wayland | Message: GStreamer OpenGL platforms: egl | Message: GStreamer OpenGL apis: gles2 | Run-time dependency zlib found: YES 1.2.11 | Run-time dependency glib-2.0 found: YES 2.58.3 | Run-time dependency gobject-2.0 found: YES 2.58.3 | Run-time dependency gio-2.0 found: YES 2.58.3 | Run-time dependency orc-0.4 found: YES 0.4.31 | Program orcc found: YES (/home/vishnu/rzg_vlp_v3.0.5/build/tmp/work/aarch64-poky-linux/gstreamer1.0-plugins-good/1.16.3-r0/recipe-sysroot-native/usr/bin/orcc) | Header <gst/gstconfig.h> has symbol "GST_DISABLE_GST_DEBUG" with dependency gstreamer-1.0: NO | Message: GStreamer debug system is enabled | Library mmngrbuf found: YES | Checking for function "mmngr_export_start_in_user" with dependency -lmmngrbuf: YES | Library bz2 found: YES | Has header "sys/soundcard.h" : YES | Message: Not building Open Sound System 4 plugin, required headers not found. | Has header "linux/videodev2.h" : YES | Message: building v4l2 plugin | Run-time dependency gudev-1.0 found: YES 233 | Dependency libv4l2 skipped: feature v4l2-libv4l2 disabled | Dependency x11 skipped: feature ximagesrc disabled | Run-time dependency cairo-gobject found: YES 1.16.0 | Run-time dependency flac found: YES 1.3.3 | Run-time dependency gdk-pixbuf-2.0 found: YES 2.40.0 | Dependency gtk+-3.0 skipped: feature gtk3 disabled | Dependency jack skipped: feature jack disabled | Run-time dependency libjpeg found: YES 2.0.4 | Library mp3lame found: YES | Header <lame/lame.h> has symbol "lame_init" : YES | Header <lame/lame.h> has symbol "lame_set_VBR_quality" : YES | Header <lame/lame.h> has symbol "MEDIUM" : YES | Dependency caca skipped: feature libcaca disabled | Dependency libdv skipped: feature dv disabled | Run-time dependency libpng found: YES 1.6.37 | Run-time dependency libmpg123 found: YES 1.25.13 | Dependency libraw1394 skipped: feature dv1394 disabled | Dependency libavc1394 skipped: feature dv1394 disabled | Dependency libiec61883 skipped: feature dv1394 disabled | WARNING: rcc dependencies will not work reliably until this upstream issue is fixed: https://bugreports.qt.io/browse/QTBUG-45460 | Run-time dependency qt5 (modules: Core, Gui, Qml, Quick) found: YES 5.6.3 (pkg-config) | Program moc-qt5 found: NO | Program moc found: NO | | ext/qt/meson.build:29:4: ERROR: Program(s) ['moc-qt5', 'moc'] not found or not executable | | A full log can be found at /home/vishnu/rzg_vlp_v3.0.5/build/tmp/work/aarch64-poky-linux/gstreamer1.0-plugins-good/1.16.3-r0/build/meson-logs/meson-log.txt | ERROR: meson failed | WARNING: exit code 1 from a shell command. | ERROR: Execution of '/home/vishnu/rzg_vlp_v3.0.5/build/tmp/work/aarch64-poky-linux/gstreamer1.0-plugins-good/1.16.3-r0/temp/run.do_configure.1258481' failed with exit code 1 ERROR: Task (/home/vishnu/rzg_vlp_v3.0.5/build/../poky/meta/recipes-multimedia/gstreamer/gstreamer1.0-plugins-good_1.16.3.bb:do_configure) failed with exit code '1' NOTE: Tasks Summary: Attempted 7001 tasks of which 6999 didn't need to be rerun and 1 failed. Summary: 1 task failed: /home/vishnu/rzg_vlp_v3.0.5/build/../poky/meta/recipes-multimedia/gstreamer/gstreamer1.0-plugins-good_1.16.3.bb:do_configure Summary: There were 9 WARNING messages shown. Summary: There were 2 ERROR messages shown, returning a non-zero exit code.
Thank you Chris for your suggestion. We built the core-image-weston image and sdk as per your suggestion. However, when compiling Qt applications, it is throwing errors as Qt libraries are not present in this image.
Is there anything we need to do to enable Qt in Weston build.
Do you have that library in your system? Did you search for it?
Yes Chris. When we search for QT libraries in Core_Image_Weston, they are missing.
I am attaching screenshots of Qt libraries searched in Core_Image_Weston and Core_Image_Qt.
Qt Libraries in Core_Image_Qt
Qt libraries not present in Core_Image_Weston
Please help us with adding QT libraries in Core_Image_Weston