RNA transcripts that do not code for proteins have been long known to lie at the heart of many biological processes, such as splicing and translation. Yet their full potential has only been appreciated recently and non-coding RNAs (ncRNAs) are now attracting increasing attention. Pioneering work in yeast and plant systems has revealed that non-coding RNAs can have a major influence on the deposition of histone and DNA modifications. This can introduce heritable variation into gene expression and, thus, be the basis of epigenetic phenomena. Mechanistically, such processes have been studied extensively in the fission yeast Schizosaccharomyces pombe, providing an important conceptual framework for possible modes of action of ncRNAs also in other organisms. In this review, we highlight mechanistic insights into chromatin-associated ncRNA activities gained from work with fission yeast, and we draw parallels to studies in other eukaryotes that indicate evolutionary conservation.