How do I balance between respecting my child and not spoiling him or letting him think that he can get his way all the time?

I think being firm is the most important thing you can do. Being firm doesn't mean you are too negative or controlling. You can be firm while respecting your child's space and yet give him or her an indication that they are going against the line you've drawn earlier.