It could just be his personality to be quite honest. Babies also don't quite understand stares or anything. Need to engage them with the other senses! Try stroking him or like cuddling him before your next activity? might work!
Your baby has a superb personality. Try talking to him and engaging him during nappy change.