Baby should drink no more than 2-3 ounces a day or as your pediatrician recommends. You do not want the water to displace the intake of breast milk and/or formula.
This link is super helpful! :) http://wholesomebabyfood.momtastic.com/babyneedwater.htm#r0CdtsijXZqtZvLI.99