how do you reject people who wants to carry your baby without sounding rude?

I usually say that my child has anxiety issues and that it would cause him to cry out at night in shock. Yes, it's a lie but it's the kinder way to reject someone without feeling bad about it