Not Drinking Formula Milk

My LO is 19 months and he's been drinking lesser formula milk like once or twice a day inclusive of night time. We tried giving him cow's milk but he seems to reject it too. At night when he cries at times, instead of drinking milk, he'll asks for plain water. Is this normal? How should I go about with this?