Fix set implementation to be truly threadsafe even with destroy() callbacks The test/object test is passing again, instead of deadlocking.