If there aren't any problems with your pregnancy, you may want to start doing it after your first trimester just to be really sure. Yogis who have been pregnant continue to exercise even while in their first trimester, though. It's best to assess yourself first or consult your OB before engaging in such activity.
My wife, who's a huge yoga enthusiast, did yoga all through her first and second trimester. She did mention that she slightly had to modify her routine when she got into her second trimester, so please take note of that if you're already in your second trimester.
When you're over your first trimester, it's usually safe to do some prenatal yoga exercises. Start with some basic moves and gradually level up if you believe you can handle it. But, of course, you must first consider your OB gyne's instructions.