Why not try sleeping at an angle on your back if you can't sleep on your sides? Like add a couple of pillows on your upper back so you don't sleep fully on your back. At least you won't be fully horizontal.
The idea is that your tummy is heavy so by sleeping on your back it may press and block blood circulation on a smaller vein on your back and affect you and your baby's blood oxygen level. I think different people will have different effect since different people have different body constitution, weight and blood pressure, so even if one has nothing bad happen, it may not apply to another. Personally, I don't think that a short while sleeping on your back or waking up on your back would be problematic.