Oh dear, I really feel you on this, the upper back burning at 23 weeks is so draining, especially when you are juggling work and home at the same time! When I was pregnant with my second, I had a similar burning sensation across my upper back and the broken sleep made everything feel ten times harder to manage.
What helped me a lot was doing the cat-cow yoga stretch before bed, you just go on all fours and gently arch and round your back slowly, and it really releases that tension along the spine. My neighbour who went through the same thing around the same stage swore by a warm (not hot) heat pad on her upper back for about fifteen minutes before sleeping, which gave her some relief without being too intense. For exercise, a shorter but more focused session of about twenty to thirty minutes of gentle stretching and posture work actually helped more than very long walks, because over-tiring yourself can sometimes make the aching worse the next day. Building a bit of core and back strength through simple movements like supported squats can also help your body carry the pregnancy weight better over time.
That said, since yours sounds quite persistent and is affecting your sleep and daily life, do check in with your gynae or a women's health physiotherapist, they can give advice that is specific to your body and how your pregnancy is progressing. Take care of yourself, mama, you are doing so well!