quietflamingo/dnaberts-no-flashattention

This repository is a fork of the DNABERT-S repository: https://huggingface.co/zhihan1996/DNABERT-S. It is modified to not load FlashAttention, which uses an outdated and no longer available version of Triton. While the standard protocol is to simply uninstall Trition when using DNABERT-S, this is not ideal for integrating DNABERT-S into a package.

This version and the original version provide identical outputs on simple test code.

If you use this model please provide attribution to:

@misc{zhou2024dnaberts,
      title={DNABERT-S: Learning Species-Aware DNA Embedding with Genome Foundation Models}, 
      author={Zhihan Zhou and Winmin Wu and Harrison Ho and Jiayi Wang and Lizhen Shi and Ramana V Davuluri and Zhong Wang and Han Liu},
      year={2024},
      eprint={2402.08777},
      archivePrefix={arXiv},
      primaryClass={q-bio.GN}
}

Original Github: https://github.com/MAGICS-LAB/DNABERT_S
Original HF repo: https://huggingface.co/zhihan1996/DNABERT-S
Original paper: https://arxiv.org/abs/2402.08777