I’ve never faced it but if you always put him down after singing, he may associate it as a bedtime cue, which may make him sad?
I only sing during the day to my LO, and she always smiles and squeals
Maybe he felt some emotional attachment to the song which brings sad feelings