git.osdn.net Git - android-x86/external-ffmpeg.git/log

nutdec: fix illegal count check in decode_main_header

The existing check has two problems:
1) i + count can overflow, so that the check '< 256' returns true.
2) In the (i == 'N') case occurs a j-- so that the loop runs once more.

This can trigger the assertion 'nut->header_len[0] == 0' or cause
segmentation faults or infinite hangs.

Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 7c24ca1bda2d4df1dc9b2b982941be532d60da21)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

nutdec: check chapter creation in decode_info_header

This fixes a segmentation fault when accessing the metadata.

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 3ff1af2b0db7132d5717be6395227a94c8abab07)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

nutdec: check for negative frame rate in decode_info_header

A negative frame rate triggers an av_assert2 in av_rescale_rnd.

Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 6621105877ce0d65724a8ab60b3a50160adbe65d)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

ffmpeg: remove incorrect network deinit

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit e2877bdf3862325c2982c3237d9bf28f1bbf793f)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

apedec: prevent out of array writes in decode_array_0000

s->decoded_buffer is allocated with a min_size of:
2 * FFALIGN(blockstodecode, 8) * sizeof(*s->decoded_buffer)

Then it is assigned to s->decoded[0] (and s->decoded_buffer + FFALIGN(blockstodecode, 8)
to s->decoded[1]) and passed as out buffer to decode_array_0000.

In this function 64 elements of the out buffer are written
unconditionally and outside the array if blockstodecode is too small.

This causes memory corruption, leading to segmentation faults or other
crashes.

Thus change decode_array_0000 to write at most blockstodecode elements
of the out buffer.

Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 699341d647f7af785fb8ceed67604467b0b9ab12)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

apedec: set s->samples only when init_frame_decoder succeeded

Otherwise range_start_decoding is not necessarily run and thus
ctx->rc.range still 0 in range_dec_normalize leading to an infinite
loop.

Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 464c49155ce7ffc88ed39eb2511e7a75565c24be)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

tests/fate-run: do not attempt to parse tiny_psnrs output if it failed

This avoids confusing syntax errors with awk later

Likely fixes awk errors at:
http://buildd.debian-ports.org/status/fetch.php?pkg=ffmpeg&arch=sparc64&ver=7%3A2.6.2-1&stamp=1428928967

Reviewed-by: Timothy Gu <timothygu99@gmail.com>
Thanks-to: Andreas Cadhalpun <andreas.cadhalpun@googlemail.com> for the link
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit c0d847e457c1ef72843a63853f1135d52b74131e)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

alac: reject rice_limit 0 if compression is used

If rice_limit is 0, k can be 0 in decode_scalar, which calls show_bits(gb, k).

Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 4b657a1b1eedcf38bcf36e89a2f4be6f76b5ce09)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

alsdec: only adapt order for positive max_order

For max_order = 0 the clipping range is invalid. (amin = 2, amax = 1)

Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 58d605ee9b3277289278dc40e022311f8e083833)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

lavf: Reset global flag on deinit

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 32da94fa7f73ac749e0a1e2f20499fad2f6f57fe)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

mpeg4videodec: only allow a positive length

Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit b3408ae4c64cb674b1d5f0f30171759113ce722a)

Conflicts:

libavcodec/mpeg4videodec.c

alsdec: check sample pointer range in revert_channel_correlation

Also change the type of begin, end and smp to ptrdiff_t to make the
comparison well-defined.

Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Reviewed-by: Thilo Borgmann <thilo.borgmann@mail.de>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit afc7748d1f6abc4b3b1cc957b0fa6941837db3d0)

Conflicts:

libavcodec/alsdec.c

avformat/utils: Ensure that AVFMT_FLAG_CUSTOM_IO is set before use

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit ba631b791435c395361e2026fc7419b341e57813)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

alsdec: validate time diff index

If begin is smaller than t, the subtraction 'begin -= t' wraps around,
because begin is unsigned. The same applies for end < t.

This causes segmentation faults.

Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit faf9fe2c224ea81a98afd53e2f0be0a2e13aeca9)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

avcodec/alsdec: Use av_mallocz_array() for chan_data to ensure the arrays never contain random data

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 7e104647a3556fc61a139483cee1cb7dfa2dc5bd)

Conflicts:

libavcodec/alsdec.c

alsdec: ensure channel reordering is reversible

If the same idx is used for more than one i, at least one entry in
sconf->chan_pos remains uninitialized.

This can cause segmentation faults.

Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit ef16501aebed43e34a3721336e8bee732eca2877)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

ac3: validate end in ff_ac3_bit_alloc_calc_mask

This fixes an invalid read if end is 0:
band_end = ff_ac3_bin_to_band_tab[end-1] + 1;

Depending on what is before the array, this can cause stack smashing,
when band_end becomes too large.

Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit bc4fee7f2a51635fa3c0f61d1e5164da1efeded3)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

aacpsy: avoid psy_band->threshold becoming NaN

If band->thr is 0.0f, the division is undefined, making norm_fac not a
number or infinity, which causes psy_band->threshold to become NaN.

This is passed on to other variables until it finally reaches
sce->sf_idx and is converted to an integer (-2147483648).

This causes a segmentation fault when it is used as array index.

Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Reviewed-by: Claudio Freire <klaussfreire@gmail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit e224aa41917454e7b5c23d9f2541425743ce595a)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

aasc: return correct buffer size from aasc_decode_frame

Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 0be54ad280cf114c02306b7063147e8379f8ed1e)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

aacdec: consistently use avctx for logging in decode_eld_specific_config

ac may be NULL and then accessing ac->avctx results in a segmentation fault.

Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 5b75689b987e4c4dd4f34d5c8be389547e9cc701)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

msrledec: use signed pixel_ptr in msrle_decode_pal4

This fixes segmentation faults, when pic->linesize[0] is negative.
In that case 'line * pic->linesize[0] + pixel_ptr' is treated as
unsigned and wraps around.

This reverts commit 7d78a964.
The problem was introduced in commit f7e1367f, which should obsolete
that commit.

Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit ae6fd7300b4e9f81d3b5ba201096ffe7cccf26fb)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

avcodec/aacdec: Fix storing state before PCE decode

Fixes Ticket4460

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit e88b3852aefaa39b2170ef185ad03dda18732821)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

tests: Fix test name for pixfmts tests(cherry picked from commit e1ee0521a698809ed216e9e5c11bd2bbb466ed04)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

tests/fate.sh: report different status for different errors

The order of error codes will be useful in my future fateserver patches.

Signed-off-by: Timothy Gu <timothygu99@gmail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit cc0057a31c7097839f9c4e4da61e2933b5b0e055)
Signed-off-by: Timothy Gu <timothygu99@gmail.com>

avcodec/h264_refs: Do not set reference to things which dont exist

Fixes deadlock
Fixes Ticket4428
Fixes Ticket4429

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 429de043202286a2b5bcc082cc02de860b734db2)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

avutil/pca: Check for av_malloc* failures

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit dadc43eee4d9036aa532665a04720238cc15e922)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

avcodec/msrledec: restructure msrle_decode_pal4() based on the line number instead of the pixel pointer

Fixes out of array access
Fixes: da14e86d8462be6493eab16bc2d40f88/asan_heap-oob_204cfd2_528_cov_340150052_COMPRESS.BMP

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit f7e1367f58263593e6cee3c282f7277d7ee9d553)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

avcodec/hevc_ps: Check cropping parameters more correctly

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 06c70d45373dedc600f28e345685b130b60203c1)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

avcodec/hevc_ps: More complete window reset

Fixes out of array read
Fixes: signal_sigsegv_35bcf26_471_cov_2806540268_CAINIT_A_SHARP_4.bit
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 57e5812198aada016e9ba4149123c541f8c8a7ec)

Conflicts:

libavcodec/hevc_ps.c

hevc: make the crop sizes unsigned
(cherry picked from commit c929659bdd7d2d5848ea52e685a3164c7b901bb0)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

avcodec/dnxhddec: Check that the frame is interlaced before using cur_field

Fixes Ticket4227

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 2c660e34cf3c2b77cd2bef6f292920334dfd9192)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

avformat/mov: Disallow ".." in dref unless use_absolute_path is set

as this kind of allows to circumvent it to some extend.
We also could add a separate parameter or value to choose this

Found-by: ramiro
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 1e4d0498df6621143da1a550006ddc3526ad51cb)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

avformat/mov: Check for string truncation in mov_open_dref()

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 8003816e1619e77d8de051883264aa090e0d78cc)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

avformat/mov: Use sizeof(filename) instead of a literal number

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 21a53dd08dce7cc5b3fdf9c4826b4b74d8300ea0)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

hevc: delay ff_thread_finish_setup for hwaccel

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 31816eae32019ff0e2243533f618efa2a4da9c33)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

ffmdec: make sure the time base is valid

A negative time base can trigger assertions.

Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 4c91d81be23ffacfa3897b2bcfa77445bb0c2f89)

Conflicts:

libavformat/ffmdec.c
(cherry picked from commit 9678ceb6976ca8194848b24535785a298521211f)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

avcodec/012v: redesign main loop

Fixes out of array accesses
Fixes: ffmpeg_012v_crash.ts

Found-by: Thomas Lindroth <thomas.lindroth@gmail.com>
Reviewed-by: Thomas Lindroth <thomas.lindroth@gmail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 48df30d36c3ca360c407d84f96749888d1fbe853)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

avcodec/012v: Check dimensions more completely

Fixes division by 0

Found-by: Thomas Lindroth <thomas.lindroth@gmail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit d3b25383daffac154846daeb4e4fb46569e728db)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

arm: Suppress tags about used cpu arch and extensions

When all the codepaths using manually set .arch/.fpu code is
behind runtime detection, the elf attributes should be suppressed.

This allows tools to know that the final built binary doesn't
strictly require these extensions.

Signed-off-by: Martin Storsjö <martin@martin.st>
(cherry picked from commit dcae2e32f7d8a1ca5fb8c1e4aa81313be854dd73
and b77e335e441040a40fc6156b8e4a134745d10233)
Signed-off-by: Martin Storsjö <martin@martin.st>
(cherry picked from commit 9841654c158c80e9d525ba03754135d3f34e306e)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

Fix read-after-free in matroska_read_seek().

In matroska_read_seek(), |tracks| is assigned at the begining of the function.
However, functions like matroska_parse_cues() could reallocate the tracks so
that |tracks| can get invalidated.

This CL assigns |tracks| only before we use it so that it won't be invalidated.

BUG=427266
TEST=Test case in associated bug passes now.

Change-Id: I9c7065fe8f4311ca846076281df2282d190ed344

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 33301f001747d7a542073c634cc81da5eff051cf)

Conflicts:

libavformat/matroskadec.c

roqvideoenc: set enc->avctx in roq_encode_init

So far it is only set in roq_encode_frame, but it is used in
roq_encode_end to free the coded_frame. This currently segfaults if
roq_encode_frame is not called between roq_encode_init and
roq_encode_end.

Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit cf82c426fadf90105e1fb9d5ecd267cc3aa2b288)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

avcodec/options_table: remove extradata_size from the AVOptions table

allowing access to the size but not the extradata itself is not useful
and could lead to potential problems if writing happens through this field

Reviewed-by: Andreas Cadhalpun <andreas.cadhalpun@googlemail.com>
Reviewed-by: Lukasz Marek <lukasz.m.luki2@gmail.com>
Reviewed-by: Nicolas George <george@nsup.org>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 1f4088b28540080ce1d42345c5614be3e1a6a197)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

ffmdec: limit the backward seek to the last resync position

If resyncing leads to the same position as previously, it will again
lead to a resync attempt, resulting in an infinite loop.

Thus don't seek back beyond the last syncpoint.

Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 6b8263b03ab3d16d70525ae1893cb106be7852f1)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

ffmdec: fix infinite loop at EOF

If EOF is reached, while skipping bytes, avio_tell(pb) won't change
anymore, resulting in an infinite loop.

Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 6fa98822eba501a4898fdec5b75acd3026201005)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

avformat/rm: limit packet size

The chunk size is limited to 0xFFFF (written by avio_wb16), so make
sure that the packet size is not too large.

Such large frames need to be split into slices smaller than 64 kB, but
that is currently supported neither by the rv10/rv20 encoders nor the rm
muxer.

Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
See Ticket244

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 08728f400b8367dc8c983036cb2eff3a2891322b)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

avcodec/webp: validate the distance prefix code

According to the WebP Lossless Bitstream Specification the highest
allowed value for a prefix code is 39.

If prefix_code is too large, the calculated extra_bits has an invalid
value and triggers an assertion in get_bits.

Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 5de2dab12b951b2fe121eb18503accfc91cd1565)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

avcodec/rv10: check size of s->mb_width * s->mb_height

If it doesn't fit into 12 bits it triggers an assertion.

Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 2578a546183da09d49d5bba8ab5e982dece1dede)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

mdec: check for out of bounds read

Bug-Id: CID 1257501
CC: libav-stable@libav.org
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
(cherry picked from commit 2c63081b48d98f3a0d0bed7b0ec3c0347b99144c)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

aic: Fix decoding files with odd dimensions

Normally the aic decoder finds the proper slice combination (multiple of
some number less than 32) but in case of odd width, it resorts to the
default values, which were actually swapped.
The number of slices is modified to account for such odd width cases.

CC: libav-stable@libav.org
(cherry picked from commit e878ec0d47cd6228c367b2f3128b76d7523f7255)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

avcodec/tiff: move bpp check to after "end:"

This ensures that all current and future code-pathes get bpp checked

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit d5e9fc782150d4596c72440a0aa02b7f4f1254b1)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

avcodec/utils: Align YUV411 by as much as the other YUV variants

Fixes out of array accesses
Fixes: ffmpeg_mjpeg_crash2.avi

Found-by: Thomas Lindroth <thomas.lindroth@gmail.com>
Tested-by: Thomas Lindroth <thomas.lindroth@gmail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit e3201c38d53d2b8b24d0bc95d726b2cb1752dc12)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

webp: ensure that each transform is only used once

According to the WebP Lossless Bitstream Specification
"each transform is allowed to be used only once".

If a transform is more than once this can lead to memory
corruption.

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit c089e720c1b753790c746a13053636d7facf6bf0)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

avformat/flvenc: check that the codec_tag fits in the available bits

flags is later written with avio_w8 and if it doesn't fit in one byte it
triggers an av_assert2.

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit e8565d21c276ab9ac5ce785549420321fbd0b093)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

swscale/utils: clear formatConvBuffer on allocation

Fixes use of uninitialized memory
Fixes: asan_heap-oob_35ca682_1474_cov_3230122439_aletrek_tga_16bit.mov

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 007498fc1a639ecee2cda1892cbcff66c7c8c951)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

avcodec/mjpegdec: Skip blocks which are outside the visible area

Fixes out of array accesses
Fixes: ffmpeg_mjpeg_crash.avi

Found-by: Thomas Lindroth <thomas.lindroth@gmail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 08509c8f86626815a3e9e68d600d1aacbb8df4bf)

Conflicts:

libavcodec/mjpegdec.c
(cherry picked from commit 0861bb66d7c281c92a0746167bb141735a660856)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

avformat/bit: only accept the g729 codec and 1 channel

Other codecs/channel numbers are not supported by this muxer.

Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit d0b8640f75ff7569c98d6fdb03d83451104e088c)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

avformat/bit: check that pkt->size is 10 in write_packet

Ohter packet sizes are not supported by this muxer.

This avoids a null pointer dereference of pkt->data.

Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit eeda2c3de8a8484d9e7d1e47ac836bec850b31fc)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

avformat/adxdec: check avctx->channels for invalid values

This avoids a null pointer dereference of pkt->data.

Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 7faa40af982960608b117e20fec999b48011e5e0)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

Fix buffer_size argument to init_put_bits() in multiple encoders.

Several encoders were multiplying the buffer size by 8, in order to get
a bit size. However, the buffer_size argument is for the byte size of
the buffer. We had experienced crashes encoding prores (Anatoliy) at
size 4096x4096.
(cherry picked from commit 50833c9f7b4e1922197a8955669f8ab3589c8cef)

Conflicts:

libavcodec/proresenc_kostya.c

mips/acelp_filters: fix incorrect register constraint

Change register constraint on the v variable from = to +. This was causing GCC
to think that the v variable was never read and therefore not initialize it.

This fixes about 20 fate failures on mips64el.

Signed-off-by: James Cowgill <james410@cowgill.org.uk>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit b9de1303a6414174ab2f3bccefa801bfabcf0f88)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

avcodec/hevc_ps: Sanity checks for some log2_* values

log2 values which imply numeric overflow are not supported

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 205b2ba3d677330e023aac2f4bd3f624039256b9)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

avcodec/zmbv: Check len before reading in decode_frame()

Fixes out of array read
Fixes: asan_heap-oob_4d4eb0_3994_cov_3169972261_zmbv_15bit.avi

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 1f5c7781e63d6519192ada59c1e36bcecc92791d)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

avcodec/h264: Only reinit quant tables if a new PPS is allowed

Fixes null pointer dereference
Fixes: signal_sigsegv_3042097_3007_cov_1741463594_non_monotone_timestamps1.mkv

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit c23a0e77dd492d6c794f89dbff3a438c95745e70)

Conflicts:

libavcodec/h264_slice.c

avcodec/snowdec: Fix ref value check

Fixes integer overflow and out of array read.
Fixes: signal_sigsegv_24169e6_3445_cov_3778346427_snow_chroma_bug.avi

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 8f4cbf940212079a34753c7f4d6c6b5a43586d30)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

swscale/utils: More carefully merge and clear coefficients outside the input

Fixes out of array read
Fixes: asan_heap-oob_35ca682_1474_cov_3230122439_aletrek_tga_16bit.mov

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 1895d414aaacece3b57d7bf19502305e9a064fae)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

avcodec/a64multienc: fix use of uninitialized values in to_meta_with_crop

Averaging over 2 pixels doesn't work correctly for the last pixel, because the
rest of the buffer is not initialized.

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 87513d654546a99f8ddb045ca4fa5d33778a617e)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

avcodec/a64multienc: don't set incorrect packet size

This fixes invalid reads of the packet buffer in av_dup_packet

Based on patch by Andreas Cadhalpun <andreas.cadhalpun@googlemail.com>

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit d96142e9af92ded84f2580620c571ab96c4bb657)

Conflicts:

libavcodec/a64multienc.c

avcodec/a64multienc: use av_frame_ref instead of copying the frame

This fixes freeing the frame buffer twice on cleanup leading to a crash.

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 39e4ed7c1d8d840be47f6d604704d47a59a9ae5d)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

avcodec/x86/mlpdsp_init: Simplify mlp_filter_channel_x86()

Based on patch by Francisco Blas Izquierdo Riera
Commit message partly taken from carl

fixes a compilation
error in mlpdsp_init.c with -fstack-check and some gcc compilers (I
reproduced the issue with gcc 4.7.3) by simplifying the code.

See also https://bugs.gentoo.org/show_bug.cgi?id=471756

$ make libavcodec/x86/mlpdsp_init.o
libavcodec/x86/mlpdsp_init.c: In function ‘mlp_filter_channel_x86’:
libavcodec/x86/mlpdsp_init.c:142:5: error: can’t find a register in
class ‘GENERAL_REGS’ while reloading ‘asm’
libavcodec/x86/mlpdsp_init.c:142:5: error: ‘asm’ operand has impossible
constraints

4551 -> 4509 dezicycles

Reviewed-by: Ramiro Polla <ramiro.polla@gmail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 03f39fbb2a558153a3c464edec1378d637a755fe)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

h264: initialize H264Context.avctx in init_thread_copy

This prevents using a wrong (first thread's) AVCodecContext if decoding
a frame in the first pass over all threads fails.
(cherry picked from commit a06b0b1295c51d100101e0ca0434e199ad6de6b5)

Conflicts:

libavcodec/h264.c

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

avformat/omadec: Use 64bit for ret to avoid overflow

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 0f55bc29d41585d110b126cb4ed4b395fd46d7ac)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

avformat/vqf: Use 64bit for ret to avoid overflow

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit cb08687180683a755d0fe9d425280d0e4d1e6db2)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

avformat/mvdec: Use 64bit for ret to avoid overflow

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 26c0cc154e06cb0064b3a3da49447ac44d82444f)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

avformat/gxf: Use 64bit for res to avoid overflow

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 12987f89007ee82b9d3a6090085dfaef8461ab8b)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

avformat/idcin: Use 64bit for ret to avoid overflow

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit d1923d15a3544cbb94563a59e7169291db76b312)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

avcodec/h264_slice: ignore SAR changes in slices after the first

Fixes race condition and null pointer dereference
Fixes: signal_sigsegv_1472ac3_468_cov_2915641226_CABACI3_Sony_B.jsv

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 38d5241b7f36c1571a88517a0650caade16dd5f4)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Conflicts:

libavcodec/h264_slice.c

avcodec/h264_slice: Check picture structure before setting the related fields

This might fix a hypothetical race condition

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit f111831ed61103f9fa8fdda41473a23da016bdaa)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Conflicts:

libavcodec/h264_slice.c

avcodec/h264_slice: Do not change frame_num after the first slice

Fixes potential race condition
Fixes: signal_sigsegv_1472ac3_468_cov_2915641226_CABACI3_Sony_B.jsv

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit f906982c9411f3062e3ce68013309b37c213c4dd)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Conflicts:

libavcodec/h264_slice.c

avcodec/h264: Be more strict on rejecting pps/sps changes

Fixes race condition
Fixes: signal_sigsegv_1472ac3_468_cov_2915641226_CABACI3_Sony_B.jsv

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 6fafc62b0bd0e206deb77a7aabbf3a370ad80789)

Conflicts:

libavcodec/h264.c

avcodec/h264_ps: More completely check the bit depths

Fixes out of array read
Fixes: asan_static-oob_30328b6_719_cov_3325483287_H264_artifacts_motion.h264

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 69aa79365c1e8e1cb597d33e77bf1062c2ef47d4)

Conflicts:

libavcodec/h264_ps.c

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

avformat/thp: Check av_get_packet() for failure not only for partial output

Fixes null pointer dereference
Fixes: signal_sigsegv_db2c1f_3108_cov_163322880_pikmin2_opening1_partial.thp

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit f2579dbb4b31e6ae731e7f5555680528ef3020ab)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

swscale/utils: Limit filter shifting so as not to read from prior the array

Fixes out of array read
Fixes: asan_heap-oob_1fb2f9b_3780_cov_3984375136_usf.mkv

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 692b22626ec9a9585f667c124a186b1a9796e432)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

avcodec/mjpegdec: Check escape sequence validity

Fixes assertion failure
Fixes: asan_heap-oob_1c1a4ea_1242_cov_2274415971_TESTcmyk.jpg

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

avcodec/mjpegdec: Check number of components for JPEG-LS

Fixes out of array accesses
Fixes: asan_heap-oob_1c1a4ea_1242_cov_2274415971_TESTcmyk.jpg

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit fabbfaa095660982cc0bc63242c459561fa37037)

Conflicts:

libavcodec/mjpegdec.c

avformat/mpc8: Use uint64_t in *_get_v() to avoid undefined behavior

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 05e161952954acf247e0fd1fdef00559675c4d4d)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

avformat/mpc8: fix broken pointer math

This could overflow and crash at least on 32 bit systems.

Reviewed-by: Reimar Döffinger <Reimar.Doeffinger@gmx.de>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit b737a2c52857b214be246ff615c6293730033cfa)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

avformat/mpc8: fix hang with fuzzed file

This can lead to an endless loop by seeking back a few bytes after each
attempted chunk read. Assuming negative sizes are always invalid, this
is easy to fix. Other code in this demuxer treats negative sizes as
invalid as well.

Fixes ticket #4262.

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 56cc024220886927350cfc26ee695062ca7ecaf4)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

avformat/tta: fix crash with corrupted files

av_add_index_entry() can fail, for example because the parameters are
invalid, or because memory allocation fails. Check this; it can actually
happen with corrupted files.

The second hunk is just for robustness. Just in case functions like
ff_reduce_index() remove entries. (Not sure if this can actually
happen.)

Fixes ticket #4294.

Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 6a0cd529a35190d9374b0b26504e71857cd67b83)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

avcodec/mpegvideo_enc: Fix number suffixes in rc_buffer_size calculation

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 4531e2c489d279bfc90d54ca26ed898c5b265a7f)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

avcodec/h264_cabac: use int instead of long for mbb_xy

The mb address fits in int

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 592ba6ec106206f97133c9345313010c76361e12)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

avformat/omadec: fix number suffix

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit f1f7f5903ab49b84789af5341492afbaba808a70)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

avformat/smacker: Fix number suffix

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 465f3705b1ef832fd6904750d018f81f9044f3ab)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

avformat/matroskadec: Fix number suffixes

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit fc3cdb00d084222a107e61e7168903bf3d3d0b47)

Conflicts:

libavformat/matroskadec.c

swresample/dither: Cleanup number suffixes

The <<31 case needs LL

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit c77cc2c1766666cdb5f14daee0f75e397bf7a194)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

avformat/utils: Fix number suffixes in tb_unreliable()

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 4b15bba2aec93776bfdc69a1bca42a4795a7d191)

Conflicts:

libavformat/utils.c

avformat/rmdec: Check for overflow in ff_rm_read_mdpr_codecdata()

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 03abf55f252945c70f4a79eaf4d609cee4d98710)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

swscale: check memory allocations

Bug-Id: CID 1257779
(cherry picked from commit 1dd797e3c9f179f957316a0becbec048b42df8aa)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

avcodec/flac_parser: fix handling EOF if no headers are found

Fixes assertion failure
Fixes Ticket4269

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit c4d85fc23c100f7a27d9bad710eb153214868e27)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>