back down

move backwards from a certain position; "The bully had to back down"

remove oneself from an obligation; "He bowed out when he heard how much work was involved"