fix undefined behavior: left shift a negative integer

C/C++ leaves left-shifting a negative integer up to the compiler
implementation.

On most platforms, left-shifting a negative integer is the same as
casting to the unsigned equivalent, then casting back into the signed
type.

This replaces the left-shift with the mathematical equivalent and avoids
the undefined behavior.

Ideally this should be performed for any case of left-shifting a signed
type, but this commit only covers cases picked up by the undefined
behavior sanitizer.
This commit is contained in:
John Regan
2022-07-07 10:35:08 -04:00
parent c2665b890c
commit 5cc33e5af7
6 changed files with 11 additions and 11 deletions

View File

@ -268,7 +268,7 @@ uint8_t SpecGapFiller::getSpecGapFillParams (const SfbQuantizer& sfbQuantizer, c
{
unsigned countOld = 0, countNew = 0;
b = CLIP_PM (((b << 8) + (a >> 1)) / a, SHRT_MAX);
b = CLIP_PM (((b * (1 << 8)) + (a >> 1)) / a, SHRT_MAX);
a = ((ySum << 8) - b * xSum + (size >> 1)) / size;
ySum = grpScFacs[start];