If your baby is under 6 months old, they only need to drink breastmilk or infant formula. From 6 months of age, you can give your baby small amounts of water, if needed, in addition to their breastmilk or formula feeds.
Source: https://www.pregnancybirthbaby.org.au/amp/article/when-can-babies-drink-water