Commit dd830b3ffe30a76fbe8c1f13ebc7483c9ff792e5
2021-04-09T17:36:41
rdbmp.c/rdppm.c: Fix more innocuous UBSan errors - rdbmp.c: Because of 8fb37b81713a0cdc14622dc08892ebd28a3233aa, bfOffBits, biClrUsed, and headerSize were made into unsigned ints. Thus, if bPad would eventually be negative due to a malformed header, UBSan complained about unsigned math being used in the intermediate computations. It was unnecessary to make those variables unsigned, since they are only meant to hold small values, so this commit makes them signed again. The UBSan error was innocuous, because it is effectively (if not officially) the case that (int)((unsigned int)a - (unsigned int)b) == (int)a - (int)b. - rdbmp.c: If (biWidth * source->bits_per_pixel / 8) would overflow an unsigned int, then UBSan complained at the point at which row_width was set in start_input_bmp(), even though the overflow would have been detected later in the function. This commit adds overflow checks prior to setting row_width. - rdppm.c: read_pbm_integer() now bounds-checks the intermediate value computations in order to catch integer overflow caused by a malformed text PPM. It's possible, though extremely unlikely, that the intermediate value computations could have wrapped around to a value smaller than maxval, but the worst case is that this would have generated a bogus pixel in the uncompressed image rather than throwing an error.