It's normal, I reckon. Baby's so used to being with mom and nanny that dad is sometimes not preferred. Don't give up! Eventually baby will get used to being cradled by you. Consistently hold baby for a few hours a day if possible and get baby used to the "feeling" of dad. Baby knows you're someone close and important, it's just that at his young age, baby is just more familiar with those who are always around him (mom for milk, nanny for many other occasions).
You're doing a stand-up job already -- helping whenever you can. Don't be disheartened and just keep being an awesome dad :)