Personally I feel you can bring baby along with you.
Although it may be very troublesome but it's the bonding time you have with your baby.
Moreover, your mum is not the main caregiver and if you leave baby with her, i bet you will not enjoy cos you will be more worried than anyone else.
If you are still breastfeeding baby, i think it's fine. Breastfeed on the go at night market. only about 15 mins the most.
Furthermore, Taiwan is a baby friendly country, this will probably ease your concerns more.
Read more