Reading books to toddlers

When do you start reading stories or introduce books to your child? I bought some simple story books for my 3yo. However my in law seems unhappy with it. She said children can't really understand it yet. Am I being too pussy on my child by introducing books at this age? I just want him to get familiar with books and do something else besides than just playing whole day.