---
tags:
- sentence-transformers
- sentence-similarity
- feature-extraction
- generated_from_trainer
- dataset_size:164619
- loss:TripletLoss
base_model: NovaSearch/stella_en_400M_v5
widget:
- source_sentence: 'Casado''s house bookshelf is so authentic Like your college degrees.
TURNER amazon.es Now Contil Hello Choose your address All Best Sellers Amazon
Basics Deals Latest News ELECTRONICS Mobile phones and telephony Photography and
video camera Electronics Photography and camcorders Accessories Photo studio [USER]
UNDE Audio and H TV, video and Home Cinema photographic backgrounds Funds library
for suckers that they want to pretend in a video call Visit the Store Price: €19.99
Y Of 586 FREE returns'
sentences:
- Quotes show Democrats supported riots “when BLM was BURNING down cities and killing
people in the streets!”
- This is how C5N lies with fake news fakenews cases of COVID Plaza de Mayo Argerich
- Pablo Casado has a fake library to appear on video calls
- source_sentence: 'Kendall Jenner. #BlackLivesMatter BLACK LIVES MATTER 79 GRID WILD'
sentences:
- Photo shows basketball legend Kobe Bryant’s body
- Kendall Jenner posted a photoshopped picture holding a "Black Lives Matter" sign
- Video shows the arrest of US military officer by Russian forces in 2022.
- source_sentence: 4 severe level one, fatigue Headache loss of smell -Cough Fever
- hoarseness Chest pain -Fatigue
sentences:
- THE INGREDIENTS OF THE VACCINES REVEALED
- The CROWN VIRUS from Wuhan. can be cured* with a bowl of freshly boiled garlic
water
- There are 6 "types" of COVID-19
- source_sentence: 'HUEN Airport Entebbe International Airport (IATA: EBB, ICAO: HUEN)
is the principal international airport of Uganda. ... It is the only international
airport of Uganda. Built: 1972-1973 (main terminal building) Location: Entebbe,
Uganda Hub for: Eagle Air; Uganda Airlines'
sentences:
- Uganda’s new police spokesman shoots catapult at journalist
- Uganda’s Entebbe Airport changes name to HUEN Airport
- Tweets from the Israeli prime minister’s official Twitter account show the country
was responsible for the Beirut explosion
- source_sentence: 'day I was acquitted 12/12/12 i hocus45th GP SERVICES USA CDC CENTERS
FOR DISH CONTROL AND P EXCLUSIVE: Per the CDC There Are Nearly Twice As Many Vaccine
Related Deaths SO FAR in 2021 (1,755) Than All the Vaccine Deaths this Past Decade
(994) For information about vaccines. visit who.int.'
sentences:
- New Zealand PM links booster dose to six months of freedom
- Side effects of the first published vaccine According to Pfizer documents, 1,200
deaths.
- “Thousands of COVID Vaccine Injuries and 13 U.S. Deaths Reported in December Alone”;
“In December, 3,916 COVID vaccine-related adverse events, including 13 deaths,
were reported to VAERS”
pipeline_tag: sentence-similarity
library_name: sentence-transformers
metrics:
- cosine_accuracy
model-index:
- name: SentenceTransformer based on NovaSearch/stella_en_400M_v5
results:
- task:
type: triplet
name: Triplet
dataset:
name: Unknown
type: unknown
metrics:
- type: cosine_accuracy
value: 0.9713522050783623
name: Cosine Accuracy
---
# SentenceTransformer based on NovaSearch/stella_en_400M_v5
This is a [sentence-transformers](https://www.SBERT.net) model finetuned from [NovaSearch/stella_en_400M_v5](https://huggingface.co/NovaSearch/stella_en_400M_v5) on the csv dataset. It maps sentences & paragraphs to a 1024-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.
## Model Details
### Model Description
- **Model Type:** Sentence Transformer
- **Base model:** [NovaSearch/stella_en_400M_v5](https://huggingface.co/NovaSearch/stella_en_400M_v5)
- **Maximum Sequence Length:** 512 tokens
- **Output Dimensionality:** 1024 dimensions
- **Similarity Function:** Cosine Similarity
- **Training Dataset:**
- csv
### Model Sources
- **Documentation:** [Sentence Transformers Documentation](https://sbert.net)
- **Repository:** [Sentence Transformers on GitHub](https://github.com/UKPLab/sentence-transformers)
- **Hugging Face:** [Sentence Transformers on Hugging Face](https://huggingface.co/models?library=sentence-transformers)
### Full Model Architecture
```
SentenceTransformer(
(0): Transformer({'max_seq_length': 512, 'do_lower_case': False}) with Transformer model: NewModel
(1): Pooling({'word_embedding_dimension': 1024, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
(2): Dense({'in_features': 1024, 'out_features': 1024, 'bias': True, 'activation_function': 'torch.nn.modules.linear.Identity'})
)
```
## Usage
### Direct Usage (Sentence Transformers)
First install the Sentence Transformers library:
```bash
pip install -U sentence-transformers
```
Then you can load this model and run inference.
```python
from sentence_transformers import SentenceTransformer
# Download from the 🤗 Hub
model = SentenceTransformer("Bharatdeep-H/stella_finetuned_en_dataset_to_mine_negatives_from")
# Run inference
sentences = [
'day I was acquitted 12/12/12 i hocus45th GP SERVICES USA CDC CENTERS FOR DISH CONTROL AND P EXCLUSIVE: Per the CDC There Are Nearly Twice As Many Vaccine Related Deaths SO FAR in 2021 (1,755) Than All the Vaccine Deaths this Past Decade (994) For information about vaccines. visit who.int.',
'“Thousands of COVID Vaccine Injuries and 13 U.S. Deaths Reported in December Alone”; “In December, 3,916 COVID vaccine-related adverse events, including 13 deaths, were reported to VAERS”',
'Side effects of the first published vaccine According to Pfizer documents, 1,200 deaths.',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 1024]
# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [3, 3]
```
## Evaluation
### Metrics
#### Triplet
* Evaluated with [TripletEvaluator](https://sbert.net/docs/package_reference/sentence_transformer/evaluation.html#sentence_transformers.evaluation.TripletEvaluator)
| Metric | Value |
|:--------------------|:-----------|
| **cosine_accuracy** | **0.9714** |
## Training Details
### Training Dataset
#### csv
* Dataset: csv
* Size: 164,619 training samples
* Columns: query, answer, and negative
* Approximate statistics based on the first 1000 samples:
| | query | answer | negative |
|:--------|:-----------------------------------------------------------------------------------|:-----------------------------------------------------------------------------------|:-----------------------------------------------------------------------------------|
| type | string | string | string |
| details | - min: 5 tokens
- mean: 83.66 tokens
- max: 512 tokens
| - min: 6 tokens
- mean: 19.17 tokens
- max: 141 tokens
| - min: 6 tokens
- mean: 18.69 tokens
- max: 120 tokens
|
* Samples:
| query | answer | negative |
|:--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:-------------------------------------------------------------------------------------------------------------------------------------------------|:-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| YEAST EXTRACK C DDees.co NO AFTER 40 YEARS OF RESEARCH, THERE IS A VACCINE AGAINST AIDS AFTER 76 YEARS OF RESEARCH, NO THERE IS A VACCINE AGAINST INFLUENZA AFTER 100 YEARS OF RESEARCH, NO THERE IS A VACCINE AGAINST CANCER BUT AFTER ONLY 6 MONTHS THERE IS A VACCINE AGAINST A "VIRUS" THAT APPEARED "FROM SUDDEN" AND THOSE WHO | After years of research there is no vaccine against AIDS, cancer and influenza, but in 6 months there is a vaccine against covid-19 | Chronology of decline in vaccine effectiveness shows that immunizers are not effective |
| Donald J. Trump [USER] Trump We have reports of NIGERIA supporting Iran ACTIVELY and PASSIVELY....just got of the phone with secretary Pompeo....the USA MUST and WILL respond to terrorist nations IMMEDIATELY! 12:11 AM - 08 Jan 2020 244,391 Retweets 319,284 Likes 167K [244K Follow 319K | US President Donald Trump threatens to attack Nigeria | Vice President of the United States Mike Pence unfollowed President Donald Trump on Twitter and put a picture of Joe Biden and Kamala Harris on the profile cover |
| 4 k 40 years worth of research... *no vaccine for HIV cancer MA At least 100 years of research...no vaccine for Ongoing research... no vaccine for the common cold Less than a year for a Covid vaccine? I think I'll pass on that shot!! PIC COLLAGE | Vaccines for HIV, cold and cancer should deter you from getting the Covid-19 vaccine | Anticovid vaccines cause deterioration of the immune system and AIDS |
* Loss: [TripletLoss](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#tripletloss) with these parameters:
```json
{
"distance_metric": "TripletDistanceMetric.EUCLIDEAN",
"triplet_margin": 5
}
```
### Evaluation Dataset
#### csv
* Dataset: csv
* Size: 164,619 evaluation samples
* Columns: query, answer, and negative
* Approximate statistics based on the first 1000 samples:
| | query | answer | negative |
|:--------|:----------------------------------------------------------------------------------|:-----------------------------------------------------------------------------------|:----------------------------------------------------------------------------------|
| type | string | string | string |
| details | - min: 5 tokens
- mean: 73.2 tokens
- max: 512 tokens
| - min: 5 tokens
- mean: 19.27 tokens
- max: 103 tokens
| - min: 5 tokens
- mean: 19.3 tokens
- max: 179 tokens
|
* Samples:
| query | answer | negative |
|:-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:-------------------------------------------------------------------------------------------------------------------------------------------------|:----------------------------------------------------------------------------------------------|
| --- Nora Al Riyadh Tik Tok Replying to [USER] | A restaurant in Riyadh in one of the malls, the opening of the meal is free, of course, Corona in the farthest corner is surprising | Video of Rakhi Sawant wearing a hijab to support protestors in Karnataka |
| SQUID FOR BRAZIL BELTER SALT Corumbau pasil Milk | Lula expelled from the city of Itanagrà in Bahia in May 2021. The Army had to provide security. | All workers in Gardenia Philippines bread factory COVID-19 positive in July 2020 |
| I just ran out of words William Barr, Attorney General of the America literally most important person of all the American court system just publicly denounced that there has been electoral fraud 2rad10 TM [USER].6h US Attorney General William Barr denounces Vote-by-mail fraud. OM BLITZER [USER] THE WITH THE WITH SITUATION WOLFOOTION WOLF S BLITZE ROOM OUTZER TH HE WITH DERNIE CNN EXCLUSIVE Jimmy Carter & James Baker WOLF ONE-ON-ONE WITH ATTORNEY GENERAL WILLIAM BARR CAN WELL DEEST | The US attorney general denounces that there has been electoral fraud | Hillary Clinton appeared before the US justice on June 2, 2020 |
* Loss: [TripletLoss](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#tripletloss) with these parameters:
```json
{
"distance_metric": "TripletDistanceMetric.EUCLIDEAN",
"triplet_margin": 5
}
```
### Training Hyperparameters
#### Non-Default Hyperparameters
- `eval_strategy`: steps
- `per_device_train_batch_size`: 2
- `learning_rate`: 3e-05
- `max_steps`: 4000
- `lr_scheduler_type`: cosine
- `warmup_ratio`: 0.2
- `bf16`: True
- `batch_sampler`: no_duplicates
#### All Hyperparameters
Click to expand
- `overwrite_output_dir`: False
- `do_predict`: False
- `eval_strategy`: steps
- `prediction_loss_only`: True
- `per_device_train_batch_size`: 2
- `per_device_eval_batch_size`: 8
- `per_gpu_train_batch_size`: None
- `per_gpu_eval_batch_size`: None
- `gradient_accumulation_steps`: 1
- `eval_accumulation_steps`: None
- `torch_empty_cache_steps`: None
- `learning_rate`: 3e-05
- `weight_decay`: 0.0
- `adam_beta1`: 0.9
- `adam_beta2`: 0.999
- `adam_epsilon`: 1e-08
- `max_grad_norm`: 1.0
- `num_train_epochs`: 3.0
- `max_steps`: 4000
- `lr_scheduler_type`: cosine
- `lr_scheduler_kwargs`: {}
- `warmup_ratio`: 0.2
- `warmup_steps`: 0
- `log_level`: passive
- `log_level_replica`: warning
- `log_on_each_node`: True
- `logging_nan_inf_filter`: True
- `save_safetensors`: True
- `save_on_each_node`: False
- `save_only_model`: False
- `restore_callback_states_from_checkpoint`: False
- `no_cuda`: False
- `use_cpu`: False
- `use_mps_device`: False
- `seed`: 42
- `data_seed`: None
- `jit_mode_eval`: False
- `use_ipex`: False
- `bf16`: True
- `fp16`: False
- `fp16_opt_level`: O1
- `half_precision_backend`: auto
- `bf16_full_eval`: False
- `fp16_full_eval`: False
- `tf32`: None
- `local_rank`: 0
- `ddp_backend`: None
- `tpu_num_cores`: None
- `tpu_metrics_debug`: False
- `debug`: []
- `dataloader_drop_last`: False
- `dataloader_num_workers`: 0
- `dataloader_prefetch_factor`: None
- `past_index`: -1
- `disable_tqdm`: False
- `remove_unused_columns`: True
- `label_names`: None
- `load_best_model_at_end`: False
- `ignore_data_skip`: False
- `fsdp`: []
- `fsdp_min_num_params`: 0
- `fsdp_config`: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
- `fsdp_transformer_layer_cls_to_wrap`: None
- `accelerator_config`: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
- `deepspeed`: None
- `label_smoothing_factor`: 0.0
- `optim`: adamw_torch
- `optim_args`: None
- `adafactor`: False
- `group_by_length`: False
- `length_column_name`: length
- `ddp_find_unused_parameters`: None
- `ddp_bucket_cap_mb`: None
- `ddp_broadcast_buffers`: False
- `dataloader_pin_memory`: True
- `dataloader_persistent_workers`: False
- `skip_memory_metrics`: True
- `use_legacy_prediction_loop`: False
- `push_to_hub`: False
- `resume_from_checkpoint`: None
- `hub_model_id`: None
- `hub_strategy`: every_save
- `hub_private_repo`: None
- `hub_always_push`: False
- `gradient_checkpointing`: False
- `gradient_checkpointing_kwargs`: None
- `include_inputs_for_metrics`: False
- `include_for_metrics`: []
- `eval_do_concat_batches`: True
- `fp16_backend`: auto
- `push_to_hub_model_id`: None
- `push_to_hub_organization`: None
- `mp_parameters`:
- `auto_find_batch_size`: False
- `full_determinism`: False
- `torchdynamo`: None
- `ray_scope`: last
- `ddp_timeout`: 1800
- `torch_compile`: False
- `torch_compile_backend`: None
- `torch_compile_mode`: None
- `dispatch_batches`: None
- `split_batches`: None
- `include_tokens_per_second`: False
- `include_num_input_tokens_seen`: False
- `neftune_noise_alpha`: None
- `optim_target_modules`: None
- `batch_eval_metrics`: False
- `eval_on_start`: False
- `use_liger_kernel`: False
- `eval_use_gather_object`: False
- `average_tokens_across_devices`: False
- `prompts`: None
- `batch_sampler`: no_duplicates
- `multi_dataset_batch_sampler`: proportional