Are there any disadvantages of not to letting baby have a pacifier? Like affect their brain development?

Choosing to give your LO a pacifier or not is up to what you believe in. Does not cause any major harm whether or not you choose to give.