[instancer-solver] port solver fix from fonttools See https://github.com/fonttools/fonttools/issues/3139