Hi I have constant headaches too, partly due to caffeine withdrawal, dehydration (if I go out on hot days) and I also understand it can be related to blood vessels expansion during pregnancy.
However I agree with the mummy below it should be checked in case it is due to hypertension and risk of pre-eclampsia.
As for paracetamol, if taken within the recommended dose and using for the shortest period possible, it is generally safe for mummy and baby (Pregnancy Category A).