Revert "[layout] Limit how far we skip when looking back" This reverts commit 85be877925ddbf34f74a1229f3ca1716bb6170dc.