“Don’t throw the baby out with the bathwater”,

An idiom admonition that holds no quarter.

A caution against careless decision...