Fixes: 1504/clusterfuzz-testcase-minimized-
6249212138225664
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/targets/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
if (y <= 0) {
t = seg * 32 + (val + 1 << seg2);
t = t * t - x;
- val = (seg2 - 1 << 4) + val;
+ val = (seg2 - 1) * 16 + val;
if (t >= y)
val++;
} else {
t = seg * 32 + (val - 1 << seg2);
t = t * t - x;
- val = (seg2 - 1 << 4) + val;
+ val = (seg2 - 1) * 16 + val;
if (t >= y)
val--;
}