You Can’t Fool the Optimizer