Is it really safe to have unprotected sex in your ‘safe days’?

No. Your period regulates and there is no sure way of knowing if your egg has been released, earlier or later. Nothing is "safe" unless you abstain altogether.