Do we need to worry about what we eat during pregnancy

Do I need to worry if I ate something that should not be eaten during the entire course of pregnancy. Eg of things I only realized later that I shouldn’t be eating because I didn’t know I was pregnant or I didn’t know is risky include ginseng, sharks fin, peach gum, alcohol. But despite all the advice on Internet that says this or that cannot be eaten, I have not seen any pictures or solid evidence of bad outcomes in any articles or forums (I tried searching all over) . Most are just preemptive advice or inconclusive. But after so many women and years, there is not much real evidence. Do we worry for nothing? My detailed scan is coming up. I just hope that no issues will show up.