It is well established that eukaryotic genomes are pervasively transcribed producing cryptic unstable transcripts (CUTs). However, the mechanisms regulating pervasive transcription are not well understood. Here, we report that the fission yeast CENP-B homolog Abp1 plays an important role in preventing pervasive transcription. We show that loss of abp1 results in the accumulation of CUTs, which are targeted for degradation by the exosome pathway. These CUTs originate from different types of genomic features, but the highest increase corresponds to Tf2 retrotransposons and rDNA repeats, where they map along the entire elements. In the absence of abp1, increased RNAPII-Ser5P occupancy is observed throughout the Tf2 coding region and, unexpectedly, RNAPII-Ser5P is enriched at rDNA repeats. Loss of abp1 also results in Tf2 derepression and increased nucleolus size. Altogether these results suggest that Abp1 prevents pervasive RNAPII transcription of repetitive DNA elements (i.e., Tf2 and rDNA repeats) from internal cryptic sites.