In ac021570, a series of 4 int8_t was cast into an int32_t t, by
casting each int8_t into a uint32_t and shifting by an appropriate
number of bits, then casting back into int32_t.
It seems on GCC, when casting a negative int8_t into a uint32_t,
the sign bit is extended into the rest of the value. So you can wind up
with different values based on whether the integer is constructed using
little-endian ordering or big-endian.
Since the goal of the code is to just check if the values are the same
or not, this is replaced with a memcmp call.
C/C++ leaves left-shifting a negative integer up to the compiler
implementation.
On most platforms, left-shifting a negative integer is the same as
casting to the unsigned equivalent, then casting back into the signed
type.
This replaces the left-shift with the mathematical equivalent and avoids
the undefined behavior.
Ideally this should be performed for any case of left-shifting a signed
type, but this commit only covers cases picked up by the undefined
behavior sanitizer.
Previously, the bestOrder value was being decremented inside
the loop. This caused the following:
In the case where the loop enters with bestOrder == 2, the
value is decremented to 1, and the following operations occurs:
m_tnsPredGains[channelIndex] >> (8 * bestOrder - 16)
bestOrder being 1 - this evaluates to (8 - 16), causing an integer
underflow, and a right shift by 4294967288. m_tnsPredGains is a
32-bit unsigned int, right-shifting by a value > 32 is undefined.
I think the intent of the loop conditional is to avoid running the
calculation with bestOrder == 1 - as far as I can tell, this is
the lowest possible value.
By re-arranging the comparison and using a prefix decrement,
we can exhibit the same behavior but avoid the case where
bestOrder == 1 within the loop body.
