Multiple instances of Gstreamer pipeline results in video corruption?

I am trying to stream 4 videos using gstreamer. I can stream 2 videos without a problem, but when I launch the 3rd gstreamer pipeline, all 3 videos become corrupted. Could anyone tell me why?

Here is the my script/pipeline for the viewer.

gst-launch-1.0 udpsrc port=100 ! 
application/x-rtp, media=video, encoding-name=H264,playload=96 ! 
queue ! 
rtph264depay ! 
h264parse ! 
video/x-h264,alignment=au ! 
nvv4l2decoder disable-dpb=true enable-max-performance=1 ! 
queue max-size-bytes=0 ! 
nvvidconv ! 
nveglglessink window-width=720 window-height=480 sync=false"

Here is my pipeline for the streamer. I update the udpsrc port for each instance and I have a unique udpsink port for each instance.

udpsrc port=5555 buffer-size=60000000 
! application/x-rtp, media=video, clock-rate=90000, encoding-name=H265,playload=96 
! queue ! rtph265depay ! h265parse 
! video/x-h265,alignment=au 
! nvv4l2decoder disable-dpb=true enable-max-performance=1 
! queue max-size-bytes=0 ! queue 
! nvvidconv ! video/x-raw,width=1920,height=1080,framerate=30/1 
! omxh264enc 
! video/x-h264,stream-format=byte-stream 
! h264parse 
! rtph264pay pt=96 ! tee name=ovl1 ovl1. 
!  udpsink host=127.0.0.1 port=2222 sync=false async=false

I noticed that if I don't use udpsink and simply use a shmsink, the video corruption issue goes away. So I think there's something wrong with the decoding elements?

Solution 1:

Edit your question

Please clean up your question a bit so we can help troubleshoot.

1. Simplify your pipeline:
   • Remove complex elements, such as any branches from the main pipeline made with tee
   • Use pseudo-code to help describe each pipeline:

udpsrc -> depay -> parse -> GPUdecode -> tee=dec

dec -> GPUconvert -> shmsink

dec -> GPUconvert -> qtoverlay -> GPUencode -> parse -> payload -> tee=ovl1

ovl1 -> udpsink

2. Include more information about the errors you encountered:
   • What was the error output?
   • Does the corruption occur on the streamer end or the viewer end?
   • What exactly do you mean when you say corrupted?
   • Did the program crash, or keep playing?

My best guess at the problem without a better information

There are several elements using GPU in both the "streamer" and "viewer" pipelines, and right up until display, they are in 1080p resolution. If you are running multiple instances of either pipeline (or both) on the same computer, they are going to be taking a significant portion of the GPU's memory. If the video itself is garbled, it may be caused by multiple instances of pipelines competing for memory space on the GPU. As far as I can tell from documentation, all "nv" elements and "omx" elements require GPU memory.

Possible solutions, if memory competition is the problem

1. Look into the nvcompositor element. It can support multiple sink pads, and composites those into a single source pad (turns multiple frames into one frame).

2. Write an actual application rather than a gst-launch script. gst-launch is great for quick prototyping, but becomes unwieldy very quickly with complexity.