MKV to MP4 transcoding script issues

Ok, this one has got me scratching my head - specially as I'm a scripting nOOb, and my *nix expeirence is basic to say the least.

Summary

Need to identify why the shell script for converting filers from one media container to another is generating errors and issues.

Script

I have a shell script provided by a friend. The script is used on a Ubuntu 10.10 machine for converting MKV video files to MP4 (MPEG-4). The script in question is as follows (forgive the lack of commenting):

#!/usr/bin/env bash

if [ -f $1 ] ; then
    filename=$(basename $1)
    extension=${filename##*.}
    name=${filename%.*}
    fname=$1
    video=`mediainfo --Inform=Video\;%ID% "${fname}"`
    audio=`mediainfo --Inform=Audio\;%ID% "${fname}"`
    fps=`mediainfo --Inform=Video\;%FrameRate% "${fname}"`
    `mkvextract tracks ${fname} 1:${name}.h264 2:${name}.ac3`
    `a52dec ${name}.ac3 -o wavdolby > ${name}.wav`
    `faac ${name}.wav -o ${name}.m4a`
    `MP4Box -add ${name}.m4a -add ${name}.h264 -fps $fps ${name}.mp4`
    `rm ${name}.m4a ${name}.ac3 ${name}.h264 ${name}.wav`
fi

Problem

skip errors when running a52dec

The first error happens on only selected MKV files, where when it goes to execute a52dec as part of the audio conversion process.

What happens is a skip error keeps appearing over and over in the console/terminal window. The a52dec process has created the expected file name (filename.wav), but keeps generating these skip messages and proceeds no further (despite having left for half an hour at one point just to see if it would proceed past the problem)

From looking at the Mediainfo output on a couple of different files, it looks like things are coming unstuck where the file includes 5.1 channel audio. I of course have zero idea how I would overcome this issue.

IsoMedia: command not found

This happens all the time when the script has finished running, which to this n00b suggests that it can't find the application in question.

However, I cannot find this package, or if part of a larger package which one it is to install.

More confusing, this file isn't called at any point during the script, and only is called after the rm command to clean up is run.

Extra info

Example of Mediainfo output for a MKV file that transcodes without issues

General
Unique ID                        : 233323168834975742075458986504469215458 (0xAF886862D1B0BB1B9427E04C90A1F8E2)
Complete name                    : \\192.168.2.5\video\sorted\CSI NY\CSI.New.York.S07E10.720p.HDTV.X264-DIMENSION.mkv
Format                           : Matroska
File size                        : 1.09 GiB
Duration                         : 41mn 30s
Overall bit rate                 : 3 768 Kbps
Encoded date                     : UTC 2010-12-03 20:40:51
Writing application              : mkvmerge v3.1.0 ('Happy up here') built on Jan 19 2010 12:09:24
Writing library                  : libebml v0.7.9 + libmatroska v0.8.1

Video
ID                               : 1
Format                           : AVC
Format/Info                      : Advanced Video Codec
Format profile                   : [email protected]
Format settings, CABAC           : Yes
Format settings, ReFrames        : 8 frames
Format settings, GOP             : M=6, N=12
Codec ID                         : V_MPEG4/ISO/AVC
Duration                         : 41mn 30s
Bit rate                         : 3 381 Kbps
Width                            : 1 280 pixels
Height                           : 720 pixels
Display aspect ratio             : 16:9
Frame rate                       : 23.976 fps
Color space                      : YUV
Chroma subsampling               : 4:2:0
Bit depth                        : 8 bits
Scan type                        : Progressive
Bits/(Pixel*Frame)               : 0.153
Stream size                      : 982 MiB (88%)
Writing library                  : x264 core 110 r1804 e89c4cf
Encoding settings                : cabac=1 / ref=8 / deblock=1:0:0 / analyse=0x3:0x113 / me=umh / subme=9 / psy=1 / psy_rd=1.00:0.00 / mixed_ref=1 / me_range=24 / chroma_me=1 / trellis=1 / 8x8dct=1 / cqm=0 / deadzone=21,11 / fast_pskip=0 / chroma_qp_offset=-2 / threads=12 / sliced_threads=0 / nr=0 / decimate=1 / interlaced=0 / constrained_intra=0 / bframes=5 / b_pyramid=2 / b_adapt=1 / b_bias=0 / direct=1 / weightb=1 / open_gop=0 / weightp=2 / keyint=250 / keyint_min=23 / scenecut=40 / intra_refresh=0 / rc_lookahead=40 / rc=2pass / mbtree=1 / bitrate=3381 / ratetol=1.0 / qcomp=0.60 / qpmin=0 / qpmax=51 / qpstep=4 / cplxblur=20.0 / qblur=0.5 / ip_ratio=1.40 / aq=1:1.00
Language                         : English

Audio
ID                               : 2
Format                           : AC-3
Format/Info                      : Audio Coding 3
Mode extension                   : CM (complete main)
Codec ID                         : A_AC3
Duration                         : 41mn 30s
Bit rate mode                    : Constant
Bit rate                         : 384 Kbps
Channel(s)                       : 2 channels
Channel positions                : Front: L R
Sampling rate                    : 48.0 KHz
Bit depth                        : 16 bits
Compression mode                 : Lossy
Stream size                      : 114 MiB (10%)

Example of Mediainfo output for a MKV file that causes a52dec issues

General
Unique ID                        : 173353892635048029459501626055714892286 (0x826ABECAAEC6D2638DD0EC376D6369FE)
Complete name                    : \\192.168.2.5\video\sorted\Conan\conan.2010.11.25.jim.parsons.720p.hdtv.x264-bff.mkv
Format                           : Matroska
File size                        : 1.09 GiB
Duration                         : 42mn 1s
Overall bit rate                 : 3 720 Kbps
Encoded date                     : UTC 2010-11-26 05:45:43
Writing application              : mkvmerge v3.1.0 ('Happy up here') built on Jan 19 2010 12:09:24
Writing library                  : libebml v0.7.9 + libmatroska v0.8.1

Video
ID                               : 2
Format                           : AVC
Format/Info                      : Advanced Video Codec
Format profile                   : [email protected]
Format settings, CABAC           : Yes
Format settings, ReFrames        : 3 frames
Codec ID                         : V_MPEG4/ISO/AVC
Duration                         : 42mn 1s
Bit rate                         : 3 272 Kbps
Width                            : 1 280 pixels
Height                           : 720 pixels
Display aspect ratio             : 16:9
Frame rate                       : 29.970 fps
Color space                      : YUV
Chroma subsampling               : 4:2:0
Bit depth                        : 8 bits
Scan type                        : Progressive
Bits/(Pixel*Frame)               : 0.118
Stream size                      : 961 MiB (86%)
Writing library                  : x264 core 85 r1442tw
Encoding settings                : cabac=1 / ref=3 / deblock=1:0:0 / analyse=0x3:0x113 / me=hex / subme=6 / psy=1 / psy_rd=1.00:0.00 / mixed_ref=0 / me_range=16 / chroma_me=1 / trellis=1 / 8x8dct=1 / cqm=0 / deadzone=21,11 / fast_pskip=1 / chroma_qp_offset=-2 / threads=30 / sliced_threads=0 / nr=0 / decimate=1 / mbaff=0 / constrained_intra=0 / bframes=6 / b_pyramid=1 / b_adapt=1 / b_bias=0 / direct=1 / wpredb=1 / wpredp=0 / keyint=240 / keyint_min=24 / scenecut=40 / intra_refresh=0 / rc_lookahead=40 / rc=2pass / mbtree=1 / bitrate=3272 / ratetol=1.0 / qcomp=0.60 / qpmin=10 / qpmax=51 / qpstep=4 / cplxblur=20.0 / qblur=0.5 / ip_ratio=1.40 / aq=1:1.00
Language                         : English

Audio
ID                               : 1
Format                           : AC-3
Format/Info                      : Audio Coding 3
Mode extension                   : CM (complete main)
Codec ID                         : A_AC3
Duration                         : 42mn 1s
Bit rate mode                    : Constant
Bit rate                         : 448 Kbps
Channel(s)                       : 6 channels
Channel positions                : Front: L C R, Side: L R, LFE
Sampling rate                    : 48.0 KHz
Bit depth                        : 16 bits
Compression mode                 : Lossy
Stream size                      : 135 MiB (12%)

MKV to MP4 Remuxing Script

This is based off thewinchester's script posted in his answer here. I started off with his and it eventually morphed into a rewrite of most of it, but the concept is the same.

The basic function is to take an MKV file containing h.264 video and some audio and convert it to an MP4/M4V file.

My goal is to make the resulting files compatible with the iPad and AppleTV, but it should work with anything that can handle MP4/M4V files. Video is not transcoded, so quality remains the same, audio is converted if necessary, but higher quality tracks are kept if possible.

Changes

A partial list of the changes I've made:

• Handling multiple files or directories passed as arguments
• Uses (and requires) the higher quality NeroAAC to encode AAC audio from AC3 or DTS
• Handling multiple audio tracks in accordance with the AppleTV standard to allow for proper playback and surround sound on AppleTV:
  • 160kB/s stereo AAC
  • (Optional) AC-3 (typically surround sound) as the second track, disabled
• Converting DTS to AC-3
• Handling multiple tracks regardless of order (i.e. AAC, Video, AC3)
• Various coding changes to make it easier to modify later

Requirements

The script requires the aforementioned NeroAAC, as well as Aften (AC3 encode) and libdca/dcadec (DTS decode). As with the original script, ffmpeg is also required.

Known Issues

The script is fairly bulletproof - if it runs into something it can't deal with, it's typically ignored or quits. The original files are left intact, so there's little to no chance of data loss.

I've used it on a lot of my media library and many of the changes have been to fix bugs I've run into.

That said, there are a few things to be aware of, and room for improvement:

• Handling multiple audio tracks of the same type - 1 AC3 and 1 AAC is fine, but 2 AAC tracks won't work. This shouldn't come up often, but keep it in mind.
• No support for non-audio/video tracks. Subtitles, chapters and artwork are all ignored.
• No detection of surround sound AAC. The script copies any AAC track, it won't down mix to stereo. Technically for iPad and AppleTV support, the main audio track should be stereo AAC, but I think in most cases an iPad or AppleTV can handle it gracefully.

Code and use

I'm posting it here hoping that someone will find it useful - please feel free to use and modify as you see fit for non-commercial uses. If you come up with an interesting modification it might be nice if you shared it here or elsewhere.

#!/bin/bash

#Close stdin - avoid accidental keypresses causing problems
exec 0>&-

# Find MKV files
for file in "$@";
do
  find "$file" -type f -not -name ".*" | grep .mkv$ | while read file
  do
    fileProper=$(readlink -f "$file") # full path of file
    pathNoExt=${fileProper%.*} # full path minus extension

    #Check if M4V already exists
    if [ -f "$pathNoExt".m4v ]; then
      echo "M4V already exists, stopping"
    else
      # Get number of tracks
      numberOfTracks=`mkvmerge -i "$fileProper" | grep "Track ID" | wc -l`
      echo "Found $numberOfTracks Tracks"

      # Set base extraction command
      extractCmd+=(mkvextract tracks "$fileProper")

      # Determine type of tracks
      for (( i=1; i<=$numberOfTracks; i++ ))
      do
         trackType=`mkvmerge -i "$fileProper" | grep "Track ID $i" | sed -e 's/^.*: //'`
         if [[ "$trackType" == *video* ]]; then
            echo "Track $i is Video"
            extractCmd+=( $i:"$pathNoExt".264)
            fps=`mkvinfo "$fileProper" | grep duration | sed -e 's/.*(//' -e 's/f.*//' | sed -n ${i}p`
         elif [[ "$trackType" == "audio (A_AAC)" ]]; then
            echo "Track $i is AAC"
            extractCmd+=( $i:"$pathNoExt".aac)
         elif [[ "$trackType" == "audio (A_AC3)" ]]; then
            echo "Track $i is AC3"
            extractCmd+=( $i:"$pathNoExt".ac3)
         elif [[ "$trackType" == "audio (A_DTS)" ]]; then
            echo "Track $i is DTS"
            extractCmd+=( $i:"$pathNoExt".dts)
         fi
         # Insert cases for handling other audio and non-AV tracks here
       done

        "${extractCmd[@]}" # Extract Tracks

        # Check files and encode audio if neccessary
        if [ -f "$pathNoExt".264 ]; then
            # Video file exists
            mp4BoxCmd+=(MP4Box -new "$pathNoExt".m4v -add "$pathNoExt".264 -fps $fps)
            if [ -f "$pathNoExt".aac ]; then
                # AAC exists
                mp4BoxCmd+=( -add "$pathNoExt".aac)
                if [ -f "$pathNoExt".ac3 ]; then
                    mp4BoxCmd+=( -add "$pathNoExt".ac3:disable)
                elif [ -f "$pathNoExt".dts ]; then
                    # Encode DTS to AC3
                    dcadec -o wavall "$pathNoExt".dts | aften -v 0 - "$pathNoExt".ac3
                    mp4BoxCmd+=( -add "$pathNoExt".ac3:disable)
                fi
            else # Encode AAC from AC3 or DTS
                if [ -f "$pathNoExt".ac3 ]; then
                    ffmpeg -i "$pathNoExt".ac3 -acodec pcm_s16le -ac 2 -f wav - | neroAacEnc -lc -br 160000 -ignorelength -if - -of "$pathNoExt".aac
                    mp4BoxCmd+=( -add "$pathNoExt".aac -add "$pathNoExt".ac3:disable)
                elif [ -f "$pathNoExt".dts ]; then
                  ffmpeg -i "$pathNoExt".dts -acodec pcm_s16le -ac 2 -f wav - | neroAacEnc -lc -br 160000 -ignorelength -if - -of "$pathNoExt".aac
                    # Encode DTS to AC3
                    dcadec -o wavall "$pathNoExt".dts | aften -v 0 - "$pathNoExt".ac3
                    mp4BoxCmd+=( -add "$pathNoExt".aac -add "$pathNoExt".ac3:disable)
                else
                    echo "Warning: no audio file found"
                fi
            fi
            # Create m4v
            "${mp4BoxCmd[@]}"
        else
            echo "Error: no video file found"
        fi
  #remove temporary track files
  rm -f "$pathNoExt".aac "$pathNoExt".dts "$pathNoExt".ac3 "$pathNoExt".264 "$pathNoExt".wav
  fi
 done
done

Hello71 prompted a thought with his ffmpeg comment, and I found my answer after some additional searching.

A copy of the successful code to convert the file is as follows. Also fails a little more gracefully when an odd audio format is used.

(Comments added to make it easier to understand)

#!/bin/bash


# Evaluate the file passed to the script for information relevant to the process
find . -type f | grep .mkv$ | while read file
do
directory=`dirname "$file"`
title=`basename "$file" .mkv`

# Check the audio track used in the files
AC3=`mkvinfo "$file" | grep AC3` #check if it's AC3 audio or DTS
AAC=`mkvinfo "$file" | grep AAC`
order=`mkvinfo "$file" | grep "Track type" | sed 's/.*://' | head -n 1 | tr -d " "` #check if the video track is first or the audio track

# Start processing
# If video is the first track
if [ "$order" = "video" ]; then
  fps=`mkvinfo "$file" | grep duration | sed 's/.*(//' | sed 's/f.*//' | head -n 1` #store the fps of the video track

# If audio is encoded in AC3
  if [ -n "$AC3" ]; then
   mkvextract tracks "$file" 1:"${title}".264 2:"${title}".ac3
   ffmpeg -i "${title}".ac3 -acodec libfaac -ab 576k "${title}".aac
#  mplayer -ao pcm:file="${title}".wav:fast "${title}".ac3
#  faac -o "${title}".aac "${title}".wav


# If audio is encoded in AAC
  elif [ -n "$AAC" ]; then
   mkvextract tracks "$file" 1:"${title}".264 2:"${title}".aac

# If encoded in DTS or something else
  else
   mkvextract tracks "$file" 1:"${title}".264 2:"${title}".dts
   ffmpeg -i "${title}".dts -acodec libfaac -ab 576k "${title}".aac
  fi
else

# If video is not the first track
  fps=`mkvinfo "$file" | grep duration | sed 's/.*(//' | sed 's/f.*//' | tail -n 1`
  if [ -n "$AC3" ]; then
   mkvextract tracks "$file" 1:"${title}".ac3 2:"${title}".264
   ffmpeg -i "${title}".ac3 -acodec libfaac -ab 576k "${title}".aac
  # mplayer -ao pcm:file="${title}".wav:fast "${title}".ac3
  # faac -o "${title}".aac "${title}".wav
  elif [ -n "$AAC" ]; then
   mkvextract tracks "$file" 1:"${title}".264 2:"${title}".aac
  else
   mkvextract tracks "$file" 1:"${title}".dts 2:"${title}".264
   ffmpeg -i "${title}".dts -acodec libfaac -ab 576k "${title}".aac
  fi
fi
MP4Box -new "${directory}/${title}".mp4 -add "${title}".264 -add "${title}".aac -fps $fps
rm -f "$title".aac "$title".dts "$title".ac3 "$title".264 "${title}".wav
# if [ -f "${directory}/${title}".mp4 ]; then
# rm -f "$file"
# fi
done

The tool FFmpeg can do most of what is described here.

ffmpeg -i input.mkv -c copy output.mp4

In a for loop (remux every *.mkv in a directory):

for f in *.mkv; do ffmpeg -i "$f" -c copy "${f/%mkv/mp4}"; done

...will work most of the time, since most MKV files will have h.264 video and AAC audio. To convert a file that doesn't use those:

ffmpeg -i input.mkv -c:v libx264 -crf 23 -preset veryfast -c:a libfdk_aac -vbr 3 output.mp4

See the x264 and AAC encoding guides on the FFmpeg wiki for more information.

If you want to transcode only those files that absolutely must be converted, you can use a bash script like so:

#!/bin/bash
for f in *.mkv
do

##  Detect what audio codec is being used:
audio=$(ffprobe "$f" 2>&1 | sed -n '/Audio:/s/.*: \([a-zA-Z0-9]*\).*/\1/p' | sed 1q)
##  Detect video codec:
video=$(ffprobe "$f" 2>&1 | sed -n '/Video:/s/.*: \([a-zA-Z0-9]*\).*/\1/p' | sed 1q)
##  Set default audio settings (you may need to use a different encoder,
##  since libfdk_aac is not re-distributable)
aopts="-c:a libfdk_aac -vbr 3"
##  Set default video settings:
vopts="-c:v libx264 -crf 22 -preset veryfast"

    case "$audio" in
        aac|alac|mp3|mp2|ac3 )
##  If the audio is one of the MP4-supported codecs,
##  copy the stream instead of transcoding
            aopts="-c:a copy"
            ;;
        "" )
##  If there is no audio stream, don't bother with audio
            aopts="-an"
            ;;
        * )
            ;;
    esac

    case "$video" in
##  If the video stream is one of the MP4-compatible formats,
##  copy the stream
        h264|mpeg4|mpeg2video|mpeg1video )
            vopts="-c:v copy"
            ;;
        "" )
## If no video stream is detected, don't bother with video
            vopts="-vn"
            ;;
        * )
            ;;
    esac

##  This will tell you what is going on; that is,
##  it will echo a line that will make the terminal output
##  easier to follow:
echo -e "\n    \E[1;30mffmpeg -i $f -map 0 $vopts $aopts ${f/%mkv/mp4}\E[0m"

##  And now, to the meat of the thing.
##  Normally you should ALWAYS quote your variables
##  so that spaces in filenames etc will be preserved.
##  But in this case, doing so for $vopts and $aopts
##  would break the ffmpeg command
##  because we WANT the spaces to break up the strings:
ffmpeg -y -i "$f" -map 0 $vopts $aopts "${f/%mkv/mp4}"
done

exit 0