 All throughout the song, Ruffin asks his lover why she left him, what he did wrong that drove her away, and professes to her that without her, his life is meaningless.Singing backup for Ruffin on the recording are The Originals, who the same year would score a hit of their own with "Baby I'm For Real".As to the distinct piccolo flute on the recording, Dayna Hartwick stated 

"My Whole World Ended (The Moment You Left Me)" was the focal point of Ruffin's debut solo LP, "My Whole World Ended".