I think u have every right to refuse guests esp with a small baby, not just for COVID but cz babies have greater risk of contracting even the slightest illness. Maybe if they inform tt they want to come, can perhaps tell them wait till a bit older. During my confinement I had a few relatives who wanted to come but I told my mom not to allow them cz I wasn't in the best condition to receive guests. Sometimes even resorted to telling them tt we were sick or I'm bck at my own place (cz I'm usually at my mom's when my husband works pm shift). Honestly I don't see why there's a need to visit whenever ppl has given birth cz it just adds on to the mom's stress.