When to remove baby mittens and booties?

Daytime I think is the best time when to remove baby mittens - so they don't scratch themselves while sleeping at night. I remember when I visited my OB with my baby and she was 2 months old - she said, remove the mittens already! So she can experience her world by touch :)