FFMPEG - Failed locking bitstream buffer: not enough buffer (14): .35x Video encoding failed

I'm trying to use FFmpeg for converting several video files from AVC to HEVC using hevc_nvenc and after it converts the file at the last moment, it gives me this error:

[hevc_nvenc @ 00000293155d5880] Failed locking bitstream buffer: not enough buffer (14): .35x
Video encoding failed
[aac @ 0000021d74665040] Qavg: 585.557
[aac @ 0000021d74665040] 2 frames left in the queue on closing
Conversion Failed

and this warning before starting the conversion:

[aac @ 000001387b19d040] Estimating duration from bitrate, this may be inaccurate

my FFmpeg Command:

ffmpeg.exe -hwaccel nvdec -i '.\videoFileInput.mkv' -map 0 -c:v hevc_nvenc -rc:v vbr -tune hq -preset p5 -multipass 1 -bf 4 -b_ref_mode 1 -spatial-aq 1 -aq-strength 8 -cq 32 -c:a aac -b:a 250K -c:s mov_text '.\videoFileOutput.mp4'

my GPU: RTX2060 | Turing Architecture

for a lot of video files, this problem doesn't occur and sometimes they have similar codec and properties just it doesn't work for one of them

MediaInfo information of one of the files

Complete name                            : drive/videofile.mkv
Format                                   : Matroska
Format version                           : Version 4
File size                                : 1.14 GiB
Duration                                 : 26 min 32 s
Overall bit rate mode                    : Variable
Overall bit rate                         : 6 142 kb/s
Encoded date                             : UTC 2018-07-18 05:01:14 / UTC 2018-05-27 01:24:09
Writing application                      : mkvmerge v21.0.0 ('Tardigrades Will Inherit The Earth') 64-bit
Writing library                          : libebml v1.3.5 + libmatroska v1.4.8 / Lavf58.0.0

Video
ID                                       : 1
Format                                   : AVC
Format/Info                              : Advanced Video Codec
Format profile                           : Main@L4
Format settings                          : CABAC / 3 Ref Frames
Format settings, CABAC                   : Yes
Format settings, Reference frames        : 3 frames
Codec ID                                 : V_MPEG4/ISO/AVC
Duration                                 : 26 min 32 s
Bit rate mode                            : Variable
Bit rate                                 : 5 620 kb/s
Maximum bit rate                         : 8 430 kb/s
Width                                    : 1 920 pixels
Height                                   : 1 080 pixels
Display aspect ratio                     : 16:9
Frame rate mode                          : Constant
Frame rate                               : 23.976 (24000/1001) FPS
Color space                              : YUV
Chroma subsampling                       : 4:2:0
Bit depth                                : 8 bits
Scan type                                : Progressive
Bits/(Pixel*Frame)                       : 0.113
Stream size                              : 1 021 MiB (88%)
Writing library                          : x264 core 118
Encoding settings                        : cabac=1 / ref=3 / deblock=1:0:0 / analyse=0x1:0x111 / me=umh / subme=10 / psy=1 / psy_rd=1.00:0.00 / mixed_ref=1 / me_range=24 / chroma_me=1 / trellis=2 / 8x8dct=0 / cqm=0 / deadzone=21,11 / fast_pskip=1 / chroma_qp_offset=-2 / threads=6 / sliced_threads=0 / nr=0 / decimate=1 / interlaced=0 / bluray_compat=0 / constrained_intra=0 / bframes=2 / b_pyramid=0 / b_adapt=2 / b_bias=0 / direct=3 / weightb=1 / open_gop=0 / weightp=2 / keyint=48 / keyint_min=25 / scenecut=0 / intra_refresh=0 / rc_lookahead=0 / rc=2pass / mbtree=1 / bitrate=5620 / ratetol=1.0 / qcomp=0.50 / qpmin=6 / qpmax=51 / qpstep=4 / cplxblur=20.0 / qblur=0.5 / vbv_maxrate=8430 / vbv_bufsize=11240 / nal_hrd=vbr / ip_ratio=1.40 / aq=1:1.00
Default                                  : Yes
Forced                                   : No

Audio
ID                                       : 2
Format                                   : E-AC-3
Format/Info                              : Enhanced AC-3
Commercial name                          : Dolby Digital Plus
Codec ID                                 : A_EAC3
Duration                                 : 26 min 32 s
Bit rate mode                            : Constant
Bit rate                                 : 640 kb/s
Channel(s)                               : 6 channels
Channel layout                           : L R C LFE Ls Rs
Sampling rate                            : 48.0 kHz
Frame rate                               : 31.250 FPS (1536 SPF)
Compression mode                         : Lossy
Stream size                              : 121 MiB (10%)
Language                                 : English
Service kind                             : Complete Main
Default                                  : Yes
Forced                                   : No
Encoded date                             : UTC 2018-05-27 01:24:09

ffprobe info

Input #0, matroska,webm, from '.\videoFile.mkv':
  Metadata:
    CREATION_TIME   : 2018-05-27 01:24:09
    COMPATIBLE_BRANDS: isomiso2dashiso6
    MAJOR_BRAND     : mp42
    MINOR_VERSION   : 0
    ENCODER         : Lavf58.0.0
  Duration: 00:26:32.10, start: 0.000000, bitrate: 6141 kb/s
  Stream #0:0: Video: h264 (Main), yuv420p(progressive), 1920x1080 [SAR 1:1 DAR 16:9], 23.98 fps, 23.98 tbr, 1k tbn (default)
  Stream #0:1(eng): Audio: eac3, 48000 Hz, 5.1(side), fltp, 640 kb/s (default)
    Metadata:
      CREATION_TIME   : 2018-05-27 01:24:09
      HANDLER_NAME    : Audio Media Handler

I searched everywhere and couldn't find any solution to this problem

here are things that I've tried

• I suspected that video length might be the problem and I tried to trim the videos with the -ss command and it worked, for a 26:32 minute video I could trim to 26:00 minute and it was perfectly fine but the original file length couldn't get converted

• I also tried mkvextract and extracted the mkv container and entered them into ffmpeg and it didn't work either

my command for merging WebM and aac file

ffmpeg.exe -hwaccel nvdec -i '.\video.webm' -i .\audio.aac -map 0 -map 1 -c:v hevc_nvenc -rc:v vbr -tune hq -preset p5 -multipass 1 -bf 4 -b_ref_mode 1 -spatial-aq 1 -aq-strength 8 -cq 38 -c:a aac -b:a 250K '.\mkv_extract_Merge.mp4'

• then I thought maybe it's because of my file extension and tried to change the mp4 to mkv and it didn't work again

• I tried different AVOptions and even the default AVOptions and it didn't work again

ffmpeg.exe -i ".\video.mkv" -c:v hevc_nvenc -c:a copy ".\outpuFile.mp4"

• I tried the -bufsize:v <int> option and made it higher and higher each time, it didn't work

  • I don't know anything about what is buffer size and how it works, I tried it just to see if it's the source of the problem I was facing

• the only time it worked was the time that it used h264_nvenc and it did it (I tried it with very few AVOptions)

• I suspected the fps of the video and audio because they weren't matching so I checked the videos that got converted just fine and there was a difference in fps of audio, in the first video as you can see in the log above it was 31 fps and in the other videos it was 46, I tried to convert the audio file to 23.976 (the fps of video) using -filter:v fps=23.976 and it became 46 fps after conversion, then I tried to merge them using ffmpeg and it didn't work again

• Right now I suspect there is something wrong with the audio or video Duration, because ffmpeg estimation after it extracted them was 02:10:32 and the video was only 00:26:32 minute (this estimation is for the time I tried to merge the extracted audio and video, but when I just tried to convert the mkv container it estimated the audio to be 00:26:32.10 which is similar the MediaInfo value of duration)

this is the log of FFmpeg during conversion

Input #0, h264, from '.\video.webm':
  Duration: N/A, bitrate: N/A
  Stream #0:0: Video: h264 (Main), yuv420p(progressive), 1920x1080 [SAR 1:1 DAR 16:9], 24 fps, 23.98 tbr, 1200k tbn
[eac3 @ 000001c3e501d040] Estimating duration from bitrate, this may be inaccurate
Input #1, eac3, from '.\audio.aac':
  Duration: 00:26:32.10, start: 0.000000, bitrate: 640 kb/s
  Stream #1:0: Audio: eac3, 48000 Hz, 5.1(side), fltp, 640 kb/s
Stream mapping:
  Stream #0:0 -> #0:0 (h264 (native) -> hevc (hevc_nvenc))
  Stream #1:0 -> #0:1 (eac3 (native) -> aac (native))
Press [q] to stop, [?] for help
[aac @ 000001c3e4d65040] Using a PCE to encode channel layout "5.1(side)"
Output #0, matroska, to '.\mkv_extract_Merge.mkv':
  Metadata:
    encoder         : Lavf59.2.102
  Stream #0:0: Video: hevc (Main), nv12(progressive), 1920x1080 [SAR 1:1 DAR 16:9], q=2-31, 23.98 fps, 1k tbn
    Metadata:
      encoder         : Lavc59.1.101 hevc_nvenc
    Side data:
      cpb: bitrate max/min/avg: 0/0/0 buffer size: 4000000 vbv_delay: N/A
  Stream #0:1: Audio: aac (LC) ([255][0][0][0] / 0x00FF), 48000 Hz, 5.1(side), fltp, 250 kb/s
    Metadata:
      encoder         : Lavc59.1.101 aac
[hevc_nvenc @ 000001c3e4d644c0] Failed locking bitstream buffer: not enough buffer (14): .26x
Video encoding failed
[aac @ 000001c3e4d65040] Qavg: 585.557
[aac @ 000001c3e4d65040] 2 frames left in the queue on closing
Conversion failed!

the h264_nvenc log:

Input #0, matroska,webm, from '.\videoFile.mkv':
  Metadata:
    CREATION_TIME   : 2018-05-27 01:24:09
    COMPATIBLE_BRANDS: isomiso2dashiso6
    MAJOR_BRAND     : mp42
    MINOR_VERSION   : 0
    ENCODER         : Lavf58.0.0
  Duration: 00:26:32.10, start: 0.000000, bitrate: 6141 kb/s
  Stream #0:0: Video: h264 (Main), yuv420p(progressive), 1920x1080 [SAR 1:1 DAR 16:9], 23.98 fps, 23.98 tbr, 1k tbn (default)
  Stream #0:1(eng): Audio: eac3, 48000 Hz, 5.1(side), fltp, 640 kb/s (default)
    Metadata:
      CREATION_TIME   : 2018-05-27 01:24:09
      HANDLER_NAME    : Audio Media Handler
Stream mapping:
  Stream #0:0 -> #0:0 (h264 (native) -> h264 (h264_nvenc))
  Stream #0:1 -> #0:1 (copy)
Press [q] to stop, [?] for help
[mp4 @ 00000284a0af0cc0] track 1: codec frame size is not set
Output #0, mp4, to '.\videofile.mp4':
  Metadata:
    MINOR_VERSION   : 0
    COMPATIBLE_BRANDS: isomiso2dashiso6
    MAJOR_BRAND     : mp42
    encoder         : Lavf59.2.102
  Stream #0:0: Video: h264 (Main) (avc1 / 0x31637661), nv12(progressive), 1920x1080 [SAR 1:1 DAR 16:9], q=2-31, 2000 kb/s, 23.98 fps, 24k tbn (default)
    Metadata:
      encoder         : Lavc59.1.101 h264_nvenc
    Side data:
      cpb: bitrate max/min/avg: 0/0/2000000 buffer size: 4000000 vbv_delay: N/A
  Stream #0:1(eng): Audio: eac3 (ec-3 / 0x332D6365), 48000 Hz, 5.1(side), fltp, 640 kb/s (default)
    Metadata:
      CREATION_TIME   : 2018-05-27 01:24:09
      HANDLER_NAME    : Audio Media Handler
frame=38170 fps=359 q=20.0 Lsize=  506687kB time=00:26:32.06 bitrate=2607.2kbits/s speed=  15x
video:381434kB audio:124382kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.171940%

Edit June 16,21

I found another clue :

I ran the script for a couple of files with the same settings and several of them failed with the massage above, and when I checked them, they had an "encoding settings" sections containing the below text:

wpp / ctu=16 / tu-intra-depth=1 / tu-inter-depth=1 / me=0 / subme=0 / merange=25 / no-rect / no-amp / max-merge=2 / temporal-mvp / early-skip / no-fast-cbf / rdpenalty=0 / no-tskip / no-tskip-fast / strong-intra-smoothing / no-lossless / no-cu-lossless / no-constrained-intra / fast-intra / no-open-gop / interlace=0 / keyint=30 / min-keyint=1 / scenecut=0 / rc-lookahead=1 / bframes=0 / bframe-bias=0 / b-adapt=0 / ref=1 / no-weightp / no-weightb / aq-mode=0 / aq-strength=0.00 / cbqpoffs=0 / crqpoffs=0 / rd=2 / psy-rd=0.00 / psy-rdoq=0.00 / no-signhide / no-lft / no-sao / no-sao-non-deblock / no-b-pyramid / no-cutree / rc=abr / bitrate=2000 / qcomp=0.60 / qpmin=0 / qpmax=51 / qpstep=4 / ipratio=1.40

cabac=1 / ref=4 / deblock=1:0:0 / analyse=0x3:0x111 / me=umh / subme=10 / psy=1 / psy_rd=1.00:0.00 / mixed_ref=1 / me_range=120 / chroma_me=1 / trellis=2 / 8x8dct=1 / cqm=0 / deadzone=21,11 / fast_pskip=1 / chroma_qp_offset=-2 / threads=6 / lookahead_threads=1 / sliced_threads=0 / nr=0 / decimate=1 / interlaced=0 / bluray_compat=0 / stitchable=1 / constrained_intra=0 / bframes=16 / b_pyramid=2 / b_adapt=2 / b_bias=0 / direct=3 / weightb=1 / open_gop=0 / weightp=2 / keyint=240 / keyint_min=121 / scenecut=0 / intra_refresh=0 / rc_lookahead=240 / rc=crf / mbtree=1 / crf=16.0 / qcomp=0.50 / qpmin=6 / qpmax=51 / qpstep=4 / vbv_maxrate=20000 / vbv_bufsize=25000 / crf_max=0.0 / nal_hrd=vbr / filler=0 / ip_ratio=1.40 / aq=1:0.80

I don't know which of these settings might have been the cause of this error but it's probably one of them.

Edit June 24,21

here are some sample files that have problem with encoding.

I tried trimming the videos (i only trimmed several frames) that had the problem and it removed the Encoding Settings part in MediaInfo and now I can convert them without problem, it seems like the Encoding Settings is the problem. any info about what is this option and how to remove it is appreciated.

Solution 1:

The workaround to your issue, assuming that the GPU driver and FFmpeg versions remain untouched is to either:

1. Lower thread counts to 1 (via -threads 1) when NVDEC decoding is in use AND to bump up the -extra_hw_frames value to 3, with such a snippet:

ffmpeg.exe -threads 1 -hwaccel nvdec -extra_hw_frames 3 -i '.\videoFileInput.mkv' -map 0 -c:v hevc_nvenc -b:v 0 -rc:v vbr -tune hq -preset p5 -multipass 1 -bf 4 -b_ref_mode 1 -spatial-aq 1 -aq-strength 8 -cq 32 -c:a aac -b:a 250K -c:s mov_text '.\videoFileOutput.mp4'

NVDEC is quite fragile with multiple threads, and will warn you if thread and surface count per context reaches or exceeds 32, as shown here. The same applies to hardware surface frames that can be easily depleted when dealing with content with multiple B-frames and interlacing.

2. Better still, disable NVDEC completely:

ffmpeg.exe -i '.\videoFileInput.mkv' -map 0 -c:v hevc_nvenc -b:v 0 -rc:v vbr -tune hq -preset p5 -multipass 1 -bf 4 -b_ref_mode 1 -spatial-aq 1 -aq-strength 8 -cq 32 -c:a aac -b:a 250K -c:s mov_text '.\videoFileOutput.mp4'

And your encode should proceed with no further issues. I've added an extra parameter -b:v 0 which unsets the default bitrate NVENC sets internally, due to the rate control method in use (-rc:v vbr).

For more information on rate control options see this and this.

Test and report back.