If grandparents are taking care or looking after of baby to help you out as well then dont take it too heart. It is also good for babies so have some fresh air. But of course if you know that the downstairs like pollution and smoking then maybe talk to the grandparenta