Glucose test

Hello, I’m planning to opt out from the glucose test, but I’m also on a dilemma whether to opt out or not as previous checkup gynae said baby was on the bigger side. Any mummies opt out from the test even after gynae saying so? I’ve no records of diabetes in my family tree or even myself.