Eukaryotes use several classes of small RNA molecules to guide diverse protein machineries to target messenger RNA. The role of small RNA in post-transcriptional regulation of mRNA stability and translation is now well established. Small RNAs can also guide sequence-specific modification of chromatin structure and thus contribute to establishment and maintenance of distinct chromatin domains. In this review we summarize the model for the inter-dependent interaction between small RNA and chromatin that has emerged from studies on fission yeast and plants. We focus on recent results that link a distinct class of small RNAs, the piRNAs, to chromatin regulation in animals.