Scientifically speaking. It's the husband's sperm DNA which will determine the sex of your baby since egg cells have dominant Xgenes while sperm cells are either X and Y.
So if they're belittling you for not having a son you can always give them solid facts that their genes are the one to blame.