Firstly, I think you shouldn't place your decision whether or not you continue to breastfeed on other people's opinion. It's your body and if you feel that you have the means and you enjoy breastfeeding your child, then by all means, do continue because breastfeeding has lots of benefits. As long as your baby is putting on weight steadily and is developing well, there is no need to worry.
I say, just relax and don't worry too much about what others think. Your breastfeeding journey is yours and baby's to enjoy. Don't let other people cause you distress and affect your baby's best source of vitamins and nutrients.