Baby only drink milk when sleeping

FTM here. Anyone have issue with baby refuse to drink milk when she is awake even though she is hungry? She keeps turning the head and will scream, her stomach is so empty as well. So i know she is hungry. We can only feed her when she is about to sleep, i am afraid this become habit. It has been going on for more than 1.5 months. We have changed 3 formula, any idea what should i do mummies? It gets better with friso. Previously even in her sleep she dont want to drink milk (nan and enfamil) Should i start puree early? She is 4 months.