ffmpeg video compression: Desaturating colors / losing color information

I'm trying to compress some 4K videos (resolution: 3840x2160) that I've recorded with my Nexus 5X (running LineageOS).

My first attempt:

./ffmpeg -i VID_20190908_145514.mp4 -c:a copy -crf 23 -vf "scale=1920:-1" VID_20190908_145514.komprimiert.mp4

Problem: The colors are washed out / desaturated. The effect is quite strong. I used VLC to compare the videos side-by-side and made a screenshot for you:

Here is the ffprobe output for the original video file:

./ffprobe VID_20190908_145514.mp4 
ffprobe version 4.1.4-tessus  https://evermeet.cx/ffmpeg/  Copyright (c) 2007-2019 the FFmpeg developers
  built with Apple LLVM version 10.0.1 (clang-1001.0.46.4)
  configuration: --cc=/usr/bin/clang --prefix=/opt/ffmpeg --extra-version=tessus --enable-avisynth --enable-fontconfig --enable-gpl --enable-libaom --enable-libass --enable-libbluray --enable-libfreetype --enable-libgsm --enable-libmodplug --enable-libmp3lame --enable-libmysofa --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopus --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvo-amrwbenc --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libx264 --enable-libx265 --enable-libxavs --enable-libxvid --enable-libzimg --enable-libzmq --enable-libzvbi --enable-version3 --pkg-config-flags=--static --disable-ffplay
  libavutil      56. 22.100 / 56. 22.100
  libavcodec     58. 35.100 / 58. 35.100
  libavformat    58. 20.100 / 58. 20.100
  libavdevice    58.  5.100 / 58.  5.100
  libavfilter     7. 40.101 /  7. 40.101
  libswscale      5.  3.100 /  5.  3.100
  libswresample   3.  3.100 /  3.  3.100
  libpostproc    55.  3.100 / 55.  3.100
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'VID_20190908_145514.mp4':
  Metadata:
    major_brand     : mp42
    minor_version   : 0
    compatible_brands: isommp42
    creation_time   : 2019-09-08T12:58:35.000000Z
    com.android.version: 8.1.0
    com.android.manufacturer: LGE
    com.android.model: Nexus 5X
  Duration: 00:03:18.27, start: 0.000000, bitrate: 41991 kb/s
    Stream #0:0(eng): Video: h264 (Baseline) (avc1 / 0x31637661), yuvj420p(pc, bt470bg/bt470bg/smpte170m), 3840x2160, 41963 kb/s, SAR 1:1 DAR 16:9, 29.33 fps, 30 tbr, 90k tbn, 180k tbc (default)
    Metadata:
      rotate          : 180
      creation_time   : 2019-09-08T12:58:35.000000Z
      handler_name    : VideoHandle
    Side data:
      displaymatrix: rotation of -180.00 degrees
    Stream #0:1(eng): Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, mono, fltp, 96 kb/s (default)
    Metadata:
      creation_time   : 2019-09-08T12:58:35.000000Z
      handler_name    : SoundHandle

Here is ffprobe output for the compressed video file:

./ffprobe VID_20190908_145514.komprimiert.mp4 
ffprobe version 4.1.4-tessus  https://evermeet.cx/ffmpeg/  Copyright (c) 2007-2019 the FFmpeg developers
  built with Apple LLVM version 10.0.1 (clang-1001.0.46.4)
  configuration: --cc=/usr/bin/clang --prefix=/opt/ffmpeg --extra-version=tessus --enable-avisynth --enable-fontconfig --enable-gpl --enable-libaom --enable-libass --enable-libbluray --enable-libfreetype --enable-libgsm --enable-libmodplug --enable-libmp3lame --enable-libmysofa --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopus --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvo-amrwbenc --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libx264 --enable-libx265 --enable-libxavs --enable-libxvid --enable-libzimg --enable-libzmq --enable-libzvbi --enable-version3 --pkg-config-flags=--static --disable-ffplay
  libavutil      56. 22.100 / 56. 22.100
  libavcodec     58. 35.100 / 58. 35.100
  libavformat    58. 20.100 / 58. 20.100
  libavdevice    58.  5.100 / 58.  5.100
  libavfilter     7. 40.101 /  7. 40.101
  libswscale      5.  3.100 /  5.  3.100
  libswresample   3.  3.100 /  3.  3.100
  libpostproc    55.  3.100 / 55.  3.100
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'VID_20190908_145514.komprimiert.mp4':
  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomiso2avc1mp41
    encoder         : Lavf58.20.100
  Duration: 00:03:18.27, start: 0.000000, bitrate: 6833 kb/s
    Stream #0:0(eng): Video: h264 (High) (avc1 / 0x31637661), yuvj420p(pc), 1920x1080 [SAR 1:1 DAR 16:9], 6742 kb/s, 30 fps, 30 tbr, 15360 tbn, 60 tbc (default)
    Metadata:
      handler_name    : VideoHandle
    Stream #0:1(eng): Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, mono, fltp, 96 kb/s (default)
    Metadata:
      handler_name    : SoundHandle

The problem seems to be the mismatching colorspace/colortransfer/colorprimaries.

Original file is specified as:

yuvj420p(pc, bt470bg/bt470bg/smpte170m)

Compressed file is specified as:

yuvj420p(pc)

I searched for solutions online and came up with an improved command that tries to imitate/reproduce the color settings from the original video file:

./ffmpeg -i VID_20190908_145514.mp4 -vf "scale=1920:-1" -color_primaries 5 -colorspace 5 -color_trc 6 -crf 23 -c:a copy -to 10 VID_20190908_145514.komprimiert3.mp4

Unfortunately, the -color_primaries, -colorspace and -color_trc switches are not well documented by ffmpeg. Anyway, they seem to fix the color problem. At least, the above command now results in a matching ffprobe output regarding the colors:

yuvj420p(pc, bt470bg/bt470bg/smpte170m)

But the strange thing is: I don't see any difference when playing the video. Colors are still washed out / desaturated.

To be fair... during running the above command I got the following warning:

[swscaler @ 0x115075000] deprecated pixel format used, make sure you did set range correctly

I got the same warning on the initial attempt (without specifying -colorspace etc.).

How do I keep the original colors with ffmpeg? I don't want to lose color information. I just want ffmpeg to compress the video (by downscaling and using more CPU than was possible on my smartphone). To be honest, I didn't expect to run into such difficulties...

EDIT 1: The videos are not displayed differently when using ffplay instead of VLC or QuickTime. You can hardly see any difference. See screenshot:

EDIT 2: It could be a playback issue/bug in VLC/Quicktime. Maybe the issue is that they misinterpret the color metadata in the video file. I found out that VLC and ffprobe disagree...

VLC says in the 'Media information' window about VID_20190908_145514.mp4 (I can't upload screenshots currently):

...
Color primaries: ITU-R BT.2020
Color transfer function: ITU-R BT.709
Color space: ITU-R BT.2020 Range
Chroma location: Left
...

In contrast, ffprobe -show_streams VID_20190908_145514.mp4 says:

...
pix_fmt=yuvj420p
level=51
color_range=pc
color_space=bt470bg
color_transfer=smpte170m
color_primaries=bt470bg
chroma_location=left
...

Obviously, assuming the wrong (input) colorspace produces distorted (output) colors on playback. I'm starting to think that the washed out / desaturated colors are actually the correct ones.

Such a problem has been reported on ffmpeg's Trac before: https://trac.ffmpeg.org/ticket/7180

Solution 1:

BT.2020, as detected by VLC, could the correct description for the color encoding of your file. You can easily change at both the container and bitstream level without re-encoding

ffmpeg -i VID_20190908_145514.komprimiert.mp4 -c copy -color_primaries bt2020 -color_trc bt709 -colorspace bt2020_ncl -color_range pc -bsf:v h264_metadata=video_full_range_flag=1:colour_primaries=9:transfer_characteristics=1:matrix_coefficients=9 out.mp4

(there are actually two possibilites for BT.2020 color space / matrix coefficients, if ncl and 9 aren't right, try bt2020_ncl and 10)