Is it wrong to give your pet away during pregnancy?

I'm expecting baby #2. My husband has been the primary caretaker for my two cats but he just got a new job which requires him to be away from home most of the time. I would like to find a new home for my cats as I am not sure if I could manage the kids & the cats all by myself. I am feeling overwhelmed.