Maybe he is overstimulated by his surroundings because they are "foreign" to him? At two months, everything must be really fascinating and frightening all at once, through his eyes. My friend's baby hated crowded places -- the noise and hustle-bustle used to overwhelm him so much. But his parents realised later that he did not react negatively when they took him to the park. Agree with the mommy above, holding a young baby close is better than placing him in a pram because at their age, they crave security and comfort as much as possible.
Is your baby using pacifier? Pacifier can let them self soothe themselves. It might help. Also, probably don't use the pram, baby carrier such as ergo with infant insert will be better since they stay close to your body and can feel sense of security.
Perhaps go out in the evenings when it is cooler? I bring my baby to the playground at night for him to settle down
Let the infant sleeps with something he or she likes.