vorbisdec: Fix floor1 decoding

An intermediate value in the floor 1 linear interpolation was
overflowing
resulting in obvious artifacts on some files.

e.g.
http://upload.wikimedia.org/wikipedia/commons/7/79/Big_Buck_Bunny_small.ogv

Prior to this fix 87 out of 128 64kbit/s mono files decoded with ffmpeg
have
lower PEAQ ODG values than the same files decoded with libvorbis. With
this
fix none of that set have significantly worse ODG values than libvorbis.

Fixes issue 2352

Patch by Gregory Maxwell <greg@xiph.org>

Originally committed as revision 25724 to svn://svn.ffmpeg.org/ffmpeg/trunk

diff --git a/libavcodec/vorbis_dec.c b/libavcodec/vorbis_dec.c
index c209912..284b8df 100644 (file)
--- a/libavcodec/vorbis_dec.c
+++ b/libavcodec/vorbis_dec.c
@@ -1146,8 +1146,8 @@ static int vorbis_floor1_decode(vorbis_context *vc,
     int_fast16_t book;
     uint_fast16_t offset;
     uint_fast16_t i,j;
-    /*u*/int_fast16_t adx, ady, off, predicted; // WTF ? dy/adx =  (unsigned)dy/adx ?
-    int_fast16_t dy, err;
+    int_fast16_t adx, ady, dx, off, predicted;
+    int_fast32_t err;
 
 
     if (!get_bits1(gb)) // silence
@@ -1210,7 +1210,7 @@ static int vorbis_floor1_decode(vorbis_context *vc,
         adx = vf->list[high_neigh_offs].x - vf->list[low_neigh_offs].x;
         ady = FFABS(dy);
         err = ady * (vf->list[i].x - vf->list[low_neigh_offs].x);
-        off = (int16_t)err / (int16_t)adx;
+        off = err / adx;
         if (dy < 0) {
             predicted = floor1_Y_final[low_neigh_offs] - off;
         } else {