Existing Structures Shape Your Retreat