lavc/utils: add VP6F hack for setting correct video size in stream

For some obscure reasons avcodec_set_dimensions() resets the width/height
so that the resulting value set in the stream is equal to the
corresponding coded_width/height, which is not the same as the correct
width/height in case of the H.264 and VP6F codecs.

This adds a codec-specific hack for VP6F which disables the call to
avcodec_set_dimensions() in avcodec_open2(), like it is done with H264. A
proper fix needs to be found yet.

Fix trac ticket #1386.

Based on a patch by Michael Niedermayer.

Trolled-By: Daemon404
See thread:
Subject: [FFmpeg-devel] [PATCH] ffprobe: Stash and use width and height before opening the codec
Date: Fri,  1 Mar 2013 10:41:34 -0500

diff --git a/libavcodec/utils.c b/libavcodec/utils.c
index b245914..e46e907 100644 (file)
--- a/libavcodec/utils.c
+++ b/libavcodec/utils.c
@@ -1080,9 +1080,9 @@ int attribute_align_arg avcodec_open2(AVCodecContext *avctx, const AVCodec *code
     if ((ret = av_opt_set_dict(avctx, &tmp)) < 0)
         goto free_and_end;
 
-    //We only call avcodec_set_dimensions() for non h264 codecs so as not to overwrite previously setup dimensions
-    if (!( avctx->coded_width && avctx->coded_height && avctx->width && avctx->height && avctx->codec_id == AV_CODEC_ID_H264)){
-
+    // only call avcodec_set_dimensions() for non H.264/VP6F codecs so as not to overwrite previously setup dimensions
+    if (!(avctx->coded_width && avctx->coded_height && avctx->width && avctx->height &&
+          (avctx->codec_id == AV_CODEC_ID_H264 || avctx->codec_id == AV_CODEC_ID_VP6F))) {
     if (avctx->coded_width && avctx->coded_height)
         avcodec_set_dimensions(avctx, avctx->coded_width, avctx->coded_height);
     else if (avctx->width && avctx->height)