You know the answer best. Whatever we suggest and advice, ultimately is your choice. Even drama series also showed all kinds of men who have marital affairs and the outcomes.
Whether or not he meant it when he hits you(for a good reason to initiate divorce, tho a guy should nv hit a woman) you should consider about your daughter, do you want her to continue to have such bad childhood memories?
Whatever decision you make, spare a thought for your daughter, she is the utmost innocent party in this. Think for your daughter before you think for yourself. If this guy ever returns, do you think the three of you can have a fresh restart and continue to be happy again?
Don make decision based on short run happiness! Divorce and raise up a child is no big deal. It is not a matter of can or cannot but a want or don't want.
Be strong for your daughter! Let her be proud of you!
*hugs*
Mannhy Sim