Yes it’s advisable. Even baby tooth is not visible or obvious, there are budding teeth and the gums need to clean with wet cloth to prevent the accumulation of sugar from the milk.
Yes, my CL taught me to use sterile gauze and boiled water to clean the baby’s tongue and gums. She will also put a bit of gripe water on the gauze.