6.4 When this Yogin, because of his natural disposition to the experience of the self, loses attachment, i.e., gets detached from sense-objects, i.e., things other than the self, and actions associated with them - then he has abandoned all desires and is said to have climbed the heights of Yoga. Therefore, for one wishing to climb to Yoga, but is still disposed to the experience of the sense-objects, Karma Yoga consisting of the practice of detachment to these objects, becomes the cause for success in Yoga. Therefore one who wishes to climb to Yoga must perform Karma Yoga consisting in the practice of detachment from sense-objects.

Sri Krsna further elucidates the same: