My husband smoked during our first pregnancy and often refused to change out of his clothes after that so the smells lingers. I do try to avoid him but there’s only so much I can do. I worry the same like you and thinking back, towards the later half of the pregnancy, this worry’s priority went down a lot. I think soon, you may worry about when the baby comes and second hand smoke situation remains. I didn’t like it and my fil carried my firstborn right after smoking. For peace, I didn’t say much. Thankfully, our firstborn is good and well, even now at 2.5yo. I have no solution for you but I do hope my experience brings you some comfort.
Sin Yee