NLP Meets RNA: Unsupervised Embedding Learning for Ribozymes with
  Word2Vec
Andrew Kean Gao
Ribozymes, RNA molecules with distinct 3D structures and catalytic activity,
have widespread applications in synthetic biology and therapeutics. However,
relatively little research has focused on leveraging deep learning to enhance
our understanding of ribozymes. This study implements Word2Vec, an unsupervised
learning technique for natural language processing, to learn ribozyme
embeddings. Ribo2Vec was trained on over 9,000 diverse ribozymes, learning to
map sequences to 128 and 256-dimensional vector spaces. Using Ribo2Vec,
sequence embeddings for five classes of ribozymes (hatchet, pistol, hairpin,
hovlinc, and twister sister) were calculated. Principal component analysis
demonstrated the ability of these embeddings to distinguish between ribozyme
classes. Furthermore, a simple SVM classifier trained on ribozyme embeddings
showed promising results in accurately classifying ribozyme types. Our results
suggest that the embedding vectors contained meaningful information about
ribozymes. Interestingly, 256-dimensional embeddings behaved similarly to
128-dimensional embeddings, suggesting that a lower dimension vector space is
generally sufficient to capture ribozyme features. This approach demonstrates
the potential of Word2Vec for bioinformatics, opening new avenues for ribozyme
research. Future research includes using a Transformer-based method to learn
RNA embeddings, which can capture long-range interactions between nucleotides.