I feel you. I latched my baby so infrequently that eventually, he refused my breast too. And I brought it upon myself coz it was difficult to get a good latch, so I kept feeding with the bottle whenever I failed to latch him, until he only recognizes the bottle, and like your LO, would scream and cry when I try to bf him.
I comfort myself by telling myself tt at least I can pump and feed him bm, so he's still getting some. I hold my baby close to my breast when feeding him with a bottle just to feel closer to him. It'll be fine mama. Hugs.