Just to share my personal experience, my ex husband was exactly like yours and said the exact same thing.
What happened? He ended up having an affair with one of them and I filed for a divorce 3 weeks after I gave birth.
Have a good talk with him and tell him why it concerns you very much. Otherwise, it's not healthy for your child to grow up in such an environment where his/her father thinks its perfectly okay to hide his real identity from other women.
Read more