As long as you are travelling with your child, there will be no problem bringing milk or water on board. Once you're on board, You can also ask the steward or stewardess to help you add some hot water.
In Sg, you can either check in at the counter or use it all the way until the gate. They will collect your stroller just before you enter the plane.
Read more