OSDN Git Service

android-x86/external-ffmpeg.git
7 years agoavcodec/ra144: Fix runtime error: signed integer overflow: 11184810 * 404 cannot...
Michael Niedermayer [Sun, 28 May 2017 19:38:24 +0000 (21:38 +0200)]
avcodec/ra144: Fix runtime error: signed integer overflow: 11184810 * 404 cannot be represented in type 'int'

Fixes: 1884/clusterfuzz-testcase-minimized-4637425835966464

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
7 years agoavcodec/aac_defines: Add missing () to AAC_HALF_SUM() macro
Michael Niedermayer [Sun, 28 May 2017 18:08:49 +0000 (20:08 +0200)]
avcodec/aac_defines: Add missing () to AAC_HALF_SUM() macro

Fixes: runtime error: shift exponent 1073741848 is too large for 32-bit type 'INTFLOAT' (aka 'int')
Fixes: 1880/clusterfuzz-testcase-minimized-4900645322620928

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
7 years agoavcodec/webp: Fixes null pointer dereference
Michael Niedermayer [Wed, 10 May 2017 16:37:50 +0000 (18:37 +0200)]
avcodec/webp: Fixes null pointer dereference

Fixes: 1470/clusterfuzz-testcase-minimized-5404421666111488
Fixes: 1472/clusterfuzz-testcase-minimized-5677426430443520
Fixes: 1875/clusterfuzz-testcase-minimized-5536474562822144

Approved-by: BBB
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
7 years agoavcodec/aacdec_fixed: Fix runtime error: left shift of 1 by 31 places cannot be repre...
Michael Niedermayer [Sun, 28 May 2017 16:09:47 +0000 (18:09 +0200)]
avcodec/aacdec_fixed: Fix runtime error: left shift of 1 by 31 places cannot be represented in type 'int'

Fixes: 1878/clusterfuzz-testcase-minimized-6441918630199296

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
7 years agoavcodec/ylc: Check count in build_vlc()
Michael Niedermayer [Sun, 28 May 2017 15:20:42 +0000 (17:20 +0200)]
avcodec/ylc: Check count in build_vlc()

Fixes: runtime error: signed integer overflow: 211633430 + 2147483647 cannot be represented in type 'int'
Fixes: 1874/clusterfuzz-testcase-minimized-5037763613163520

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
7 years agoavcodec/snow: Fix runtime error: signed integer overflow: 1086573993 + 1086573994...
Michael Niedermayer [Sun, 28 May 2017 15:12:35 +0000 (17:12 +0200)]
avcodec/snow: Fix runtime error: signed integer overflow: 1086573993 + 1086573994 cannot be represented in type 'int'

Fixes: 1871/clusterfuzz-testcase-minimized-5719950331215872

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
7 years agoavcodec/jpeg2000: Fix runtime error: signed integer overflow: 4185 + 2147483394 canno...
Michael Niedermayer [Sun, 28 May 2017 12:00:30 +0000 (14:00 +0200)]
avcodec/jpeg2000: Fix runtime error: signed integer overflow: 4185 + 2147483394 cannot be represented in type 'int'

Fixes: 1870/clusterfuzz-testcase-minimized-4686788029317120

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
7 years agoavcodec/jpeg2000dec: Check tile offsets more completely
Michael Niedermayer [Sun, 28 May 2017 11:52:13 +0000 (13:52 +0200)]
avcodec/jpeg2000dec: Check tile offsets more completely

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
7 years agoavcodec/sheervideo: Check input buffer size before allocating and decoding
Michael Niedermayer [Sun, 28 May 2017 11:30:46 +0000 (13:30 +0200)]
avcodec/sheervideo: Check input buffer size before allocating and decoding

Fixes: Timeout
Fixes: 1858/clusterfuzz-testcase-minimized-6450473802399744

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
7 years agoavcodec/wavpack: Fix runtime error: signed integer overflow: -1386217472 * 4 cannot...
Michael Niedermayer [Sun, 28 May 2017 10:56:25 +0000 (12:56 +0200)]
avcodec/wavpack: Fix runtime error: signed integer overflow: -1386217472 * 4 cannot be represented in type 'int'

Fixes: 1853/clusterfuzz-testcase-minimized-5471155626442752

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
7 years agoavcodec/aacdec_fixed: Fix multiple runtime error: shift exponent 127 is too large...
Michael Niedermayer [Sun, 28 May 2017 01:34:09 +0000 (03:34 +0200)]
avcodec/aacdec_fixed: Fix multiple runtime error: shift exponent 127 is too large for 32-bit type 'int'

Fixes: 1851/clusterfuzz-testcase-minimized-5692607495667712

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
7 years agoavcodec/wnv1: More strict buffer size check
Michael Niedermayer [Sun, 28 May 2017 01:18:02 +0000 (03:18 +0200)]
avcodec/wnv1: More strict buffer size check

This requires at least 25% of a picture to allocate and decode it

Fixes: Timeout
Fixes: 1845/clusterfuzz-testcase-minimized-5075974343360512

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
7 years agoavcodec/libfdk-aacdec: Correct buffer_size parameter
Michael Niedermayer [Thu, 25 May 2017 01:21:50 +0000 (03:21 +0200)]
avcodec/libfdk-aacdec: Correct buffer_size parameter

the timeDataSize argument to aacDecoder_DecodeFrame() seems undocumented and until
2016 04 (203e3f28fbebec7011342017fafc2a0bda0ce530) unused.
after that commit libfdk-aacdec interprets it as size in sample units and memsets that on error.
FFmpeg as well as others (like GStreamer) did interpret it as size in bytes

Fixes: 1442/clusterfuzz-testcase-minimized-4540199973421056 (This requires recent libfdk to reproduce)

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
7 years agoavcodec/sbrdsp_template: Fix: runtime error: signed integer overflow: 849815297 ...
Michael Niedermayer [Tue, 23 May 2017 19:08:48 +0000 (21:08 +0200)]
avcodec/sbrdsp_template: Fix: runtime error: signed integer overflow: 849815297 + 1315389781 cannot be represented in type 'int'

Fixes: 1770/clusterfuzz-testcase-minimized-5285511235108864

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
7 years agoavcodec/aacps: Check border_position to be monotone
Michael Niedermayer [Sun, 21 May 2017 18:46:16 +0000 (20:46 +0200)]
avcodec/aacps: Check border_position to be monotone

Fixes: runtime error: left shift of negative value -67108864
Fixes: 1738/clusterfuzz-testcase-minimized-6734814327603200

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
7 years agomovenc: encryption with time code track fix
erankor [Wed, 17 May 2017 09:32:06 +0000 (12:32 +0300)]
movenc: encryption with time code track fix

instead of deciding whether to encrypt based on the encryption scheme,
decide according to whether cenc was initialized or not.
mov_create_timecode_track calls ff_mov_write_packet with a track that
doesn't have cenc initialized.

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
7 years agoavcodec/ivi_dsp: Fix runtime error: left shift of negative value -2
Michael Niedermayer [Sun, 28 May 2017 01:03:46 +0000 (03:03 +0200)]
avcodec/ivi_dsp: Fix runtime error: left shift of negative value -2

Fixes: 1839/clusterfuzz-testcase-minimized-6238490993885184

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
7 years agodoc/filters: Clarify scale2ref example
Kevin Mark [Sat, 27 May 2017 14:10:46 +0000 (10:10 -0400)]
doc/filters: Clarify scale2ref example

Signed-off-by: Kevin Mark <kmark937@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
7 years agoavformat/mov: add support for reading Content Light Level Box
James Almer [Sat, 22 Apr 2017 03:34:37 +0000 (00:34 -0300)]
avformat/mov: add support for reading Content Light Level Box

As defined in "VP Codec ISO Media File Format Binding v1.0"
https://github.com/webmproject/vp9-dash/blob/master/VPCodecISOMediaFileFormatBinding.md

Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: James Almer <jamrial@gmail.com>
7 years agoavformat/mov: add support for reading Mastering Display Metadata Box
James Almer [Sat, 22 Apr 2017 03:03:21 +0000 (00:03 -0300)]
avformat/mov: add support for reading Mastering Display Metadata Box

As defined in "VP Codec ISO Media File Format Binding v1.0"
https://github.com/webmproject/vp9-dash/blob/master/VPCodecISOMediaFileFormatBinding.md

Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: James Almer <jamrial@gmail.com>
7 years agoavcodec/mlpdec: Do not leave invalid values in matrix_out_ch[] on error
Michael Niedermayer [Sat, 27 May 2017 11:17:34 +0000 (13:17 +0200)]
avcodec/mlpdec: Do not leave invalid values in matrix_out_ch[] on error

Fixes: runtime error: index 12 out of bounds for type 'uint8_t [8]'
Fixes: 1832/clusterfuzz-testcase-minimized-6574546079449088

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
7 years agoavcodec/ra144dec: Fix runtime error: left shift of negative value -17
Michael Niedermayer [Sat, 27 May 2017 11:07:00 +0000 (13:07 +0200)]
avcodec/ra144dec: Fix runtime error: left shift of negative value -17

Fixes: 1830/clusterfuzz-testcase-minimized-5828293733384192

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
7 years agoavcodec/pixlet: Fix runtime error: signed integer overflow: 2147483647 + 32 cannot...
Michael Niedermayer [Sat, 27 May 2017 11:03:36 +0000 (13:03 +0200)]
avcodec/pixlet: Fix runtime error: signed integer overflow: 2147483647 + 32 cannot be represented in type 'int'

Fixes: 1829/clusterfuzz-testcase-minimized-5527165321871360

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
7 years agoavformat/mux: Fix copy an paste typo
Michael Niedermayer [Fri, 26 May 2017 16:01:31 +0000 (18:01 +0200)]
avformat/mux: Fix copy an paste typo

Found-by: Roger Scott <rscott@grammatech.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
7 years agozscale: Add range options aliases to match scale ones
Vittorio Giovara [Thu, 25 May 2017 21:42:22 +0000 (17:42 -0400)]
zscale: Add range options aliases to match scale ones

Signed-off-by: Vittorio Giovara <vittorio.giovara@gmail.com>
7 years agozscale: Add pixdesc-API compatible color names to filter options
Vittorio Giovara [Thu, 25 May 2017 18:43:34 +0000 (14:43 -0400)]
zscale: Add pixdesc-API compatible color names to filter options

Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Vittorio Giovara <vittorio.giovara@gmail.com>
7 years agodoc/libav-merge: remove lines about AVFrame crop fields
James Almer [Fri, 26 May 2017 14:22:23 +0000 (11:22 -0300)]
doc/libav-merge: remove lines about AVFrame crop fields

7 years agoavcodec/theora: export cropping information instead of handling it internally
James Almer [Mon, 8 May 2017 18:46:25 +0000 (15:46 -0300)]
avcodec/theora: export cropping information instead of handling it internally

This merges commit 1202b712690c14f0efb06e4ad8b06c5b3df6822a from libav,
originally written by Anton Khirnov and skipped in
fc63d5ceb357c4b760cb02772de0b50d0557140f.

 libavcodec/vp3.c | 26 +++++++++-----------------
 1 file changed, 9 insertions(+), 17 deletions(-)

7 years agoavcodec/h264dec: export cropping information instead of handling it internally
James Almer [Mon, 8 May 2017 18:46:24 +0000 (15:46 -0300)]
avcodec/h264dec: export cropping information instead of handling it internally

This merges commit c3e84820d67cb1d8cfb4196f9b43971308a81571 from libav,
originally written by Anton Khirnov and skipped in
fc63d5ceb357c4b760cb02772de0b50d0557140f.

 libavcodec/h264_picture.c |  3 ---
 libavcodec/h264_ps.c      |  9 ---------
 libavcodec/h264_slice.c   | 25 +++++++++++++++++++------
 libavcodec/h264dec.c      | 13 +------------
 libavcodec/h264dec.h      |  9 +++++----
 5 files changed, 25 insertions(+), 34 deletions(-)

7 years agoavcodec/h264dec: be more explicit in handling container cropping
James Almer [Mon, 8 May 2017 18:46:23 +0000 (15:46 -0300)]
avcodec/h264dec: be more explicit in handling container cropping

This merges commit 4fded0480f20f4d7ca5e776a85574de34dfead14 from libav,
originally written by Anton Khirnov and skipped in
fc63d5ceb357c4b760cb02772de0b50d0557140f.

 libavcodec/h264_slice.c | 20 +++++++++++++-------
 libavcodec/h264dec.c    |  3 +++
 libavcodec/h264dec.h    |  5 +++++
 3 files changed, 21 insertions(+), 7 deletions(-)

7 years agoavcodec/hevcdec: export cropping information instead of handling it internally
James Almer [Mon, 8 May 2017 18:46:22 +0000 (15:46 -0300)]
avcodec/hevcdec: export cropping information instead of handling it internally

This merges commit a02ae1c6837a54ed9e7735da2b1f789b2f4b6e13 from libav,
originally written by Anton Khirnov and skipped in
fc63d5ceb357c4b760cb02772de0b50d0557140f.

 libavcodec/hevc_parser.c |  6 ++++--
 libavcodec/hevc_ps.c     | 31 ++++++++++++-------------------
 libavcodec/hevc_ps.h     |  2 --
 libavcodec/hevc_refs.c   | 18 +++++-------------
 libavcodec/hevcdec.c     |  7 ++++---
 libavcodec/hevcdec.h     |  2 --
 6 files changed, 25 insertions(+), 41 deletions(-)

Signed-off-by: James Almer <jamrial@gmail.com>
7 years agoavcodec/clearvideo: Check buf_size before decoding frame
Michael Niedermayer [Thu, 25 May 2017 23:37:14 +0000 (01:37 +0200)]
avcodec/clearvideo: Check buf_size before decoding frame

Fixes; Timeout
Fixes: 1826/clusterfuzz-testcase-minimized-5728569256837120

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
7 years agoavcodec/aacdec_fixed: Fix runtime error: signed integer overflow: -2147483648 * ...
Michael Niedermayer [Thu, 25 May 2017 21:01:27 +0000 (23:01 +0200)]
avcodec/aacdec_fixed: Fix runtime error: signed integer overflow: -2147483648 * -1 cannot be represented in type 'int'

Fixes: 1825/clusterfuzz-testcase-minimized-6002833050566656

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
7 years agoavcodec/smc: Check remaining input
Michael Niedermayer [Thu, 25 May 2017 18:07:49 +0000 (20:07 +0200)]
avcodec/smc: Check remaining input

Fixes: Timeout
Fixes: 1818/clusterfuzz-testcase-minimized-5039166473633792

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
7 years agoavcodec/diracdec: Fix off by 1 error in quant check
Michael Niedermayer [Thu, 25 May 2017 14:35:40 +0000 (16:35 +0200)]
avcodec/diracdec: Fix off by 1 error in quant check

Fixes: out of array read
Fixes: 1781/clusterfuzz-testcase-minimized-4617176877105152

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
7 years agoavcodec/diracdec: Factor quant matrix reads
Michael Niedermayer [Thu, 25 May 2017 14:22:49 +0000 (16:22 +0200)]
avcodec/diracdec: Factor quant matrix reads

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
7 years agoframe_thread_encoder: extend critical code covered by finished_task_mutex.
Ronald S. Bultje [Thu, 25 May 2017 12:20:21 +0000 (08:20 -0400)]
frame_thread_encoder: extend critical code covered by finished_task_mutex.

Should fix tsan errors in utvideoenc_rgb_left and related tests.

7 years agohevc: fix race condition in max_ra/seq_decode.
Ronald S. Bultje [Wed, 24 May 2017 15:46:07 +0000 (11:46 -0400)]
hevc: fix race condition in max_ra/seq_decode.

These variables are shared between frame threads, but they are updated
post-setup_finished() if a EOB/EOS slice type occurs. Moving the EOB/EOS
slices to the next frame thread instance (by parsing them leading into
the next picture instead of trailing behind the last picture) effectively
prevents this race condition.

This fixes tsan failures on hevc-conformance-NoOutPrior_A_Qualcomm_1.

7 years agoavcodec/tscc2: Skip duplicate frames
Michael Niedermayer [Sun, 21 May 2017 13:34:21 +0000 (15:34 +0200)]
avcodec/tscc2: Skip duplicate frames

This turns CFR duplicated frames into skiped frames

Fixes: Timeout
Fixes: 1719/clusterfuzz-testcase-minimized-6375090079924224

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
7 years agoavcodec/wavpack: Fix runtime error: left shift of negative value -14778
Michael Niedermayer [Tue, 23 May 2017 22:21:21 +0000 (00:21 +0200)]
avcodec/wavpack: Fix runtime error: left shift of negative value -14778

Fixes: 1778/clusterfuzz-testcase-minimized-5128953268273152

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
7 years agoavcodec/wavpack: Fix: runtime error: signed integer overflow: 3 * -2147483648 cannot...
Michael Niedermayer [Tue, 23 May 2017 22:21:20 +0000 (00:21 +0200)]
avcodec/wavpack: Fix: runtime error: signed integer overflow: 3 * -2147483648 cannot be represented in type 'int'

Fixes: 1776/clusterfuzz-testcase-minimized-6191258231898112

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
7 years agoavcodec/jpeg2000dec: Fix copy and paste error
Michael Niedermayer [Thu, 25 May 2017 09:11:33 +0000 (11:11 +0200)]
avcodec/jpeg2000dec: Fix copy and paste error

Found-by: jamrial
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
7 years agoavcodec/audiotoolboxdec: check packet side data for AAC extradata updates
James Almer [Thu, 25 May 2017 02:37:04 +0000 (23:37 -0300)]
avcodec/audiotoolboxdec: check packet side data for AAC extradata updates

Tested-by: rcombs
Signed-off-by: James Almer <jamrial@gmail.com>
7 years agoavcodec/libopenh264dec: fix return error value when h264_mp4toannexb_bsf is not found
James Almer [Thu, 25 May 2017 02:31:59 +0000 (23:31 -0300)]
avcodec/libopenh264dec: fix return error value when h264_mp4toannexb_bsf is not found

7 years agodoc/libav-merge: remove line about aac_adtstoasc
James Almer [Wed, 24 May 2017 22:46:30 +0000 (19:46 -0300)]
doc/libav-merge: remove line about aac_adtstoasc

Signed-off-by: James Almer <jamrial@gmail.com>
7 years agoffmpeg: remove bsf extradata propagation hack
James Almer [Mon, 10 Apr 2017 01:37:54 +0000 (22:37 -0300)]
ffmpeg: remove bsf extradata propagation hack

The offending bitstream filter was fixed, so this is no longer needed.

Signed-off-by: James Almer <jamrial@gmail.com>
7 years agoavformat/mux: remove autobsf extradata propagation hack
James Almer [Mon, 10 Apr 2017 01:37:49 +0000 (22:37 -0300)]
avformat/mux: remove autobsf extradata propagation hack

The offending bitstream filter was fixed, so this is no longer needed.

Signed-off-by: James Almer <jamrial@gmail.com>
7 years agoavcodec/aac_adtstoasc: propagate new extradata using packet side data
James Almer [Mon, 10 Apr 2017 16:36:19 +0000 (13:36 -0300)]
avcodec/aac_adtstoasc: propagate new extradata using packet side data

This removes the current API violating behavior of overwritting the stream's
extradata during packet filtering, something that should not happen after the
av_bsf_init() call.

The bitstream filter generated extradata is no longer available during
write_header(), and as such not usable with non seekable output. The FATE
tests are updated to reflect this.

Signed-off-by: James Almer <jamrial@gmail.com>
7 years agoavcodec/adtsenc: check packet side data for AAC extradata updates
James Almer [Wed, 24 May 2017 20:02:08 +0000 (17:02 -0300)]
avcodec/adtsenc: check packet side data for AAC extradata updates

This is in preparation for the following patch.

Signed-off-by: James Almer <jamrial@gmail.com>
7 years agoavformat/latmenc: check packet side data for AAC extradata updates
James Almer [Thu, 13 Apr 2017 19:06:02 +0000 (16:06 -0300)]
avformat/latmenc: check packet side data for AAC extradata updates

This is in preparation for a following patch.

Signed-off-by: James Almer <jamrial@gmail.com>
7 years agoavformat/matroskaenc: check packet side data for AAC extradata updates
James Almer [Thu, 13 Apr 2017 15:43:11 +0000 (12:43 -0300)]
avformat/matroskaenc: check packet side data for AAC extradata updates

This adapts and merges commit f4bf236338f6001736a4784b9c23de863057a583
from libav, originally skipped in 13a211e6320d061d9e8c29354c81239324b2db03
as it was not necessary back then.

Is's applied now in preparation for the following patches, where the
aac_adtstoasc bitstream filter will start to correctly propagate the new
extradata through packet side data.

Signed-off-by: James Almer <jamrial@gmail.com>
7 years agoavformat/movenc: always check for new extradata on a packet
James Almer [Mon, 10 Apr 2017 01:05:13 +0000 (22:05 -0300)]
avformat/movenc: always check for new extradata on a packet

Don't just look at zero sized packets, and also check for AAC extradata
updates, in preparation for the following patches.

Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: James Almer <jamrial@gmail.com>
7 years agoavcodec/cllc: Check bitstream end before decoding pixels
Michael Niedermayer [Wed, 24 May 2017 17:52:00 +0000 (19:52 +0200)]
avcodec/cllc: Check bitstream end before decoding pixels

Fixes timeout
Fixes: 1802/clusterfuzz-testcase-minimized-5008293510512640

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
7 years agoavcodec/wavpack: Fix runtime error: left shift of negative value -1
Michael Niedermayer [Wed, 24 May 2017 17:53:57 +0000 (19:53 +0200)]
avcodec/wavpack: Fix runtime error: left shift of negative value -1

Fixes: 1807/clusterfuzz-testcase-minimized-6258676199325696

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
7 years agoavcodec/jpeg2000dec: Check tile offsets
Michael Niedermayer [Wed, 24 May 2017 17:40:42 +0000 (19:40 +0200)]
avcodec/jpeg2000dec: Check tile offsets

Fixes: runtime error: signed integer overflow: 4096 - -2147483648 cannot be represented in type 'int'

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
7 years agoavcodec/sanm: Fix uninitialized reference frames
Max Justicz [Wed, 24 May 2017 13:25:50 +0000 (15:25 +0200)]
avcodec/sanm: Fix uninitialized reference frames

Fixes: poc.snm

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
7 years agoavcodec/jpeglsdec: Check get_bits_left() before decoding a picture
Michael Niedermayer [Tue, 23 May 2017 20:18:52 +0000 (22:18 +0200)]
avcodec/jpeglsdec: Check get_bits_left() before decoding a picture

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
7 years agofate: add aac-al_sbr_ps_06_ur test
James Almer [Tue, 23 May 2017 18:19:14 +0000 (15:19 -0300)]
fate: add aac-al_sbr_ps_06_ur test

Signed-off-by: James Almer <jamrial@gmail.com>
7 years agoavcodec/celp_filters: Fix runtime error: signed integer overflow: 1892453989 + 381702...
Michael Niedermayer [Tue, 23 May 2017 12:50:55 +0000 (14:50 +0200)]
avcodec/celp_filters: Fix runtime error: signed integer overflow: 1892453989 + 381702783 cannot be represented in type 'int'

Fixes: 1766/clusterfuzz-testcase-minimized-6562020075765760

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
7 years agoavcodec/lagarith: Fix runtime error: left shift of 1 by 31 places cannot be represent...
Michael Niedermayer [Tue, 23 May 2017 12:34:00 +0000 (14:34 +0200)]
avcodec/lagarith: Fix runtime error: left shift of 1 by 31 places cannot be represented in type 'int'

Fixes: 1764/clusterfuzz-testcase-minimized-5394243164045312

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
7 years agoavcodec/aacdec_fixed: Fix multiple runtime error: shift exponent 127 is too large...
Michael Niedermayer [Tue, 23 May 2017 10:40:22 +0000 (12:40 +0200)]
avcodec/aacdec_fixed: Fix multiple runtime error: shift exponent 127 is too large for 32-bit type 'int'

Fixes: 1762/clusterfuzz-testcase-minimized-5150981081792512

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
7 years agolavc/ffjni: add missing '\n'
Matthieu Bouron [Sun, 21 May 2017 15:44:12 +0000 (17:44 +0200)]
lavc/ffjni: add missing '\n'

7 years agolavc/mediacodec_wrapper: do not declare JNIAMedia{Codec,CodecList,Format}Fields on...
Matthieu Bouron [Sun, 21 May 2017 15:48:05 +0000 (17:48 +0200)]
lavc/mediacodec_wrapper: do not declare JNIAMedia{Codec,CodecList,Format}Fields on the stack

7 years agolavc/mediacodec_wrapper: fix local reference leaks
Matthieu Bouron [Sun, 21 May 2017 14:48:30 +0000 (16:48 +0200)]
lavc/mediacodec_wrapper: fix local reference leaks

7 years agoavcodec/fmvc: Fix use of uninitialized memory when the first frame is not a keyframe
Max Justicz [Tue, 23 May 2017 09:22:35 +0000 (11:22 +0200)]
avcodec/fmvc: Fix use of uninitialized memory when the first frame is not a keyframe

Fixes: fmvc-poc.avi

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
7 years agoavcodec/nvenc: remove unnecessary alignment
Timo Rothenpieler [Tue, 23 May 2017 09:24:40 +0000 (11:24 +0200)]
avcodec/nvenc: remove unnecessary alignment

Fixes #6260

7 years agoavcodec/g723_1: Fix runtime error: signed integer overflow: -1013481472 + -1139123755...
Michael Niedermayer [Tue, 23 May 2017 01:43:51 +0000 (03:43 +0200)]
avcodec/g723_1: Fix runtime error: signed integer overflow: -1013481472 + -1139123755 cannot be represented in type 'int'

See: LsptoA() and L_add()
Fixes: 1758/clusterfuzz-testcase-minimized-6054857184116736

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
7 years agoavcodec/ffv1dec_template: Fix runtime error: signed integer overflow: 202 + 214748361...
Michael Niedermayer [Tue, 23 May 2017 01:24:49 +0000 (03:24 +0200)]
avcodec/ffv1dec_template: Fix runtime error: signed integer overflow: 202 + 2147483615 cannot be represented in type 'int'

Fixes: 1748/clusterfuzz-testcase-minimized-6690208340770816

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
7 years agoavcodec/xsubdec: Check that RLE coded image and colors fit in the buffer
Michael Niedermayer [Mon, 22 May 2017 22:20:25 +0000 (00:20 +0200)]
avcodec/xsubdec: Check that RLE coded image and colors fit in the buffer

Fixes: Timeout
Fixes: 1747/clusterfuzz-testcase-minimized-6035451213250560

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
7 years agoavcodec/asvdec: Check buf_size
Michael Niedermayer [Mon, 22 May 2017 21:53:53 +0000 (23:53 +0200)]
avcodec/asvdec: Check buf_size

Fixes Timeout
Fixes: 1746/clusterfuzz-testcase-minimized-6687393392361472

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
7 years agoavcodec/takdec: Fix runtime error: signed integer overflow: 2146548196 + 2156738...
Michael Niedermayer [Mon, 22 May 2017 20:19:23 +0000 (22:19 +0200)]
avcodec/takdec: Fix runtime error: signed integer overflow: 2146548196 + 2156738 cannot be represented in type 'int'

Fixes: 1743/clusterfuzz-testcase-minimized-4994834022531072

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
7 years agoavcodec/takdec: Fix runtime error: left shift of negative value -360
Michael Niedermayer [Mon, 22 May 2017 19:43:01 +0000 (21:43 +0200)]
avcodec/takdec: Fix runtime error: left shift of negative value -360

Fixes: 1739/clusterfuzz-testcase-minimized-5399237707694080

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
7 years agoavcodec/takdec: Fix multiple runtime error: signed integer overflow: -512 * 4563386...
Michael Niedermayer [Sat, 20 May 2017 15:52:21 +0000 (17:52 +0200)]
avcodec/takdec: Fix multiple runtime error: signed integer overflow: -512 * 4563386 cannot be represented in type 'int'

Fixes: 1706/clusterfuzz-testcase-minimized-6112772670619648

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
7 years agoavcodec/mlpdec: Fix runtime error: shift exponent -5 is negative
Michael Niedermayer [Sat, 20 May 2017 21:01:03 +0000 (23:01 +0200)]
avcodec/mlpdec: Fix runtime error: shift exponent -5 is negative

Fixes part of 1708/clusterfuzz-testcase-minimized-5035111957397504

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
7 years agoavcodec/escape124: Check depth against num_superblocks
Michael Niedermayer [Sat, 20 May 2017 21:01:02 +0000 (23:01 +0200)]
avcodec/escape124: Check depth against num_superblocks

Fixes: runtime error: left shift of 66184 by 15 places cannot be represented in type 'int'
Fixes: 1707/clusterfuzz-testcase-minimized-6502767008940032

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
7 years agolavf/hls: do not transfer custom IO flag
Clément Bœsch [Mon, 22 May 2017 15:31:32 +0000 (17:31 +0200)]
lavf/hls: do not transfer custom IO flag

See 0dcac9c3f0f8f32009098edb704fac4b08bac951

7 years agolavf/concatdec: do not transfer custom IO flag
Clément Bœsch [Mon, 22 May 2017 14:55:28 +0000 (16:55 +0200)]
lavf/concatdec: do not transfer custom IO flag

If the source is using a custom IO, setting this flag causes heavy leaks
since the segments will not have their avio context closed.

Regression since f5da453b068f55d335ca403d2e2b4dd2ac3d4331.

7 years agoUse AVOnce as a static variable consistently
Hendrik Leppkes [Mon, 22 May 2017 10:00:23 +0000 (12:00 +0200)]
Use AVOnce as a static variable consistently

Using AVOnce as a stack variable makes no sense as the state is lost
when the function exits.

This fixes repeated calls to av(filter/device)_register_all

7 years agoavfilter/avf_showspectrum: properly initialize pts
Paul B Mahol [Mon, 22 May 2017 09:06:48 +0000 (11:06 +0200)]
avfilter/avf_showspectrum: properly initialize pts

Signed-off-by: Paul B Mahol <onemda@gmail.com>
7 years agoavcodec/ivi_dsp: Fix multiple runtime error: left shift of negative value -71
Michael Niedermayer [Sun, 21 May 2017 23:19:50 +0000 (01:19 +0200)]
avcodec/ivi_dsp: Fix multiple runtime error: left shift of negative value -71

Fixes: 1734/clusterfuzz-testcase-minimized-5385630815092736

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
7 years agoavcodec/dct32_template: Fix runtime error: signed integer overflow: -1071326067 ...
Michael Niedermayer [Sun, 21 May 2017 21:35:21 +0000 (23:35 +0200)]
avcodec/dct32_template: Fix runtime error: signed integer overflow: -1071326067 - 1088238847 cannot be represented in type 'int'

Fixes: 1731/clusterfuzz-testcase-minimized-5123972414832640

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
7 years agoavcodec/mjpegdec: Fix runtime error: signed integer overflow: -32767 * 130560 cannot...
Michael Niedermayer [Sun, 21 May 2017 19:49:54 +0000 (21:49 +0200)]
avcodec/mjpegdec: Fix runtime error: signed integer overflow: -32767 * 130560 cannot be represented in type 'int'

Fixes: 1724/clusterfuzz-testcase-minimized-4842395432648704

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
7 years agolavf/img2: Add svg and svgz to allow reading image sequences.
Carl Eugen Hoyos [Sun, 21 May 2017 23:09:21 +0000 (01:09 +0200)]
lavf/img2: Add svg and svgz to allow reading image sequences.

Increase the return value for svg_probe() to make it != AVPROBE_SCORE_EXTENSION.

7 years agolavf/riff: Support more vlc fourcc's for 12 and 16 bit yuv4xx.
Carl Eugen Hoyos [Sun, 21 May 2017 21:39:54 +0000 (23:39 +0200)]
lavf/riff: Support more vlc fourcc's for 12 and 16 bit yuv4xx.

7 years agoavcodec/aacdec_fixed: Fix runtime error: shift exponent 34 is too large for 32-bit...
Michael Niedermayer [Sun, 21 May 2017 14:53:55 +0000 (16:53 +0200)]
avcodec/aacdec_fixed: Fix runtime error: shift exponent 34 is too large for 32-bit type 'int'

Fixes: 1721/clusterfuzz-testcase-minimized-4719352135811072

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
7 years agoavcodec/mpeg4videodec: Check for multiple VOL headers
Michael Niedermayer [Sun, 21 May 2017 14:01:27 +0000 (16:01 +0200)]
avcodec/mpeg4videodec: Check for multiple VOL headers

Fixes multiple: runtime error: signed integer overflow: 2147115008 + 413696 cannot be represented in type 'int'
Fixes: 1723/clusterfuzz-testcase-minimized-5309409372667904
Fixes: 1727/clusterfuzz-testcase-minimized-5900685306494976
Fixes: 1737/clusterfuzz-testcase-minimized-5922321338466304

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
7 years agoavcodec/vp9block: fix runtime error: signed integer overflow: 196675 * 20670 cannot...
Michael Niedermayer [Sun, 21 May 2017 00:12:21 +0000 (02:12 +0200)]
avcodec/vp9block: fix runtime error: signed integer overflow: 196675 * 20670 cannot be represented in type 'int'

Fixes: 1710/clusterfuzz-testcase-minimized-4837032931098624

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Reviewed-by: "Ronald S. Bultje" <rsbultje@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
7 years agoavcodec/vmnc: Check location before use
Michael Niedermayer [Sun, 21 May 2017 11:22:16 +0000 (13:22 +0200)]
avcodec/vmnc: Check location before use

Fixes: runtime error: signed integer overflow: 65535 * 64256 cannot be represented in type 'int'
Fixes: 1717/clusterfuzz-testcase-minimized-5491696676634624

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
7 years agoavcodec/takdec: Fix runtime error: signed integer overflow: 8192 * 524308 cannot...
Michael Niedermayer [Tue, 16 May 2017 22:07:02 +0000 (00:07 +0200)]
avcodec/takdec: Fix runtime error: signed integer overflow: 8192 * 524308 cannot be represented in type 'int'

Fixes: 1630/clusterfuzz-testcase-minimized-6326111917047808

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
7 years agoavcodec/wavpack: Fix multiple runtime error: signed integer overflow: 548 * -21474836...
Michael Niedermayer [Thu, 18 May 2017 00:13:10 +0000 (02:13 +0200)]
avcodec/wavpack: Fix multiple runtime error: signed integer overflow: 548 * -2147483648 cannot be represented in type 'int'

Fixes: 1659/clusterfuzz-testcase-minimized-5396490639900672

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
7 years agoavcodec/aac_defines: Fix: runtime error: left shift of negative value -2
Michael Niedermayer [Sun, 21 May 2017 00:51:04 +0000 (02:51 +0200)]
avcodec/aac_defines: Fix: runtime error: left shift of negative value -2

Fixes: 1716/clusterfuzz-testcase-minimized-4691012196761600

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
7 years agoavcodec/takdec: Fix runtime error: left shift of negative value -63
Michael Niedermayer [Sun, 21 May 2017 00:46:55 +0000 (02:46 +0200)]
avcodec/takdec: Fix runtime error: left shift of negative value -63

Fixes: 1713/clusterfuzz-testcase-minimized-5791887476654080

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
7 years agoavcodec/mlpdsp: Fix runtime error: signed integer overflow: -24419392 * 128 cannot...
Michael Niedermayer [Sun, 21 May 2017 00:42:12 +0000 (02:42 +0200)]
avcodec/mlpdsp: Fix runtime error: signed integer overflow: -24419392 * 128 cannot be represented in type 'int'

Fixes: 1711/clusterfuzz-testcase-minimized-5248503515185152

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
7 years agoavcodec/sbrdsp_fixed: fix runtime error: left shift of 1 by 31 places cannot be repre...
Michael Niedermayer [Sat, 20 May 2017 23:43:04 +0000 (01:43 +0200)]
avcodec/sbrdsp_fixed: fix runtime error: left shift of 1 by 31 places cannot be represented in type 'int'

Fixes: part of 1709/clusterfuzz-testcase-minimized-4513580554649600

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
7 years agoavcodec/aacsbr_fixed: Fix multiple runtime error: shift exponent 170 is too large...
Michael Niedermayer [Sat, 20 May 2017 22:06:10 +0000 (00:06 +0200)]
avcodec/aacsbr_fixed: Fix multiple runtime error: shift exponent 170 is too large for 32-bit type 'int'

Fixes part of 1709/clusterfuzz-testcase-minimized-4513580554649600

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
7 years agoconfigure: fix --tempprefix option
James Almer [Sun, 21 May 2017 00:29:58 +0000 (21:29 -0300)]
configure: fix --tempprefix option

It was broken in 091c9860559e4d33179747c5d651bc9e31bd76eb

7 years agoavfilter: take_samples: do not directly return frame when samples are skipped
Muhammad Faiz [Thu, 18 May 2017 13:10:47 +0000 (20:10 +0700)]
avfilter: take_samples: do not directly return frame when samples are skipped

Modifying data pointer when skipping samples may make it unaligned.
Workaround for Ticket6349.

This should fix the crash of ticket's testcase and a crash/regression
with avxsynth (reported by Michael Niedermayer).

Also change frame->nb_samples < max to frame->nb_samples <= max.
This improves performance. Benchmark:
./ffmpeg -filter_complex "aevalsrc=0:n=1166,firequalizer=fixed=on" -f null null
old:
  25767 decicycles in take_samples,    1023 runs,      1 skips
  25422 decicycles in take_samples,    2047 runs,      1 skips
  25181 decicycles in take_samples,    4095 runs,      1 skips
  24904 decicycles in take_samples,    8191 runs,      1 skips

new:
    550 decicycles in take_samples,    1024 runs,      0 skips
    548 decicycles in take_samples,    2048 runs,      0 skips
    545 decicycles in take_samples,    4096 runs,      0 skips
    544 decicycles in take_samples,    8192 runs,      0 skips

Reviewed-by: Nicolas George <george@nsup.org>
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Muhammad Faiz <mfcc64@gmail.com>
7 years agoavcodec/decode: fix function name
James Almer [Sat, 20 May 2017 16:17:12 +0000 (13:17 -0300)]
avcodec/decode: fix function name

7 years agodecode: fix the code reducing cropping to preserve alignment
Anton Khirnov [Fri, 19 May 2017 09:47:21 +0000 (11:47 +0200)]
decode: fix the code reducing cropping to preserve alignment

Currently it does not work at all.

Libav Bug-Id: 1058
(cherry picked from commit 8652a2c24836ce5546b398f12b7fed45000050e1)

7 years agoavcodec/g723_1dec: Clip after shift in estimate_sid_gain()
Michael Niedermayer [Sat, 20 May 2017 12:16:57 +0000 (14:16 +0200)]
avcodec/g723_1dec: Clip after shift in estimate_sid_gain()

Fixes: runtime error: left shift of 706 by 22 places cannot be represented in type 'int'
See: L_shl() in the reference software
Fixes: 1609/clusterfuzz-testcase-minimized-5102163007111168

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/targets/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>