Aiyah, don't worry too much, it is actually quite common for toddlers to go off their milk or food after a fever and vomiting episode, since their tummy and appetite just need time to reset. What I've generally heard is to focus first on keeping him hydrated with enough fluids, it doesn't have to be milk specifically, even water or diluted juice counts while he's still recovering. A small trick that some mums swear by is switching up how you offer it, try a different cup, a straw, or even offer it at a different time of day when he seems more rested and less cranky, since timing and presentation can sometimes make a surprising difference. That said, if he's been refusing milk for more than a week or seems to be losing weight or energy, it really is worth bringing him to the paediatrician just to rule out anything else going on, better to get a proper check than to second-guess yourself.