tomaarsen/reranker-msmarco-MiniLM-L12-H384-uncased-lambdaloss

CrossEncoder based on microsoft/MiniLM-L12-H384-uncased

This is a Cross Encoder model finetuned from microsoft/MiniLM-L12-H384-uncased using the sentence-transformers library. It computes scores for pairs of texts, which can be used for text reranking and semantic search.

Model Details

Model Description

• Model Type: Cross Encoder
• Base model: microsoft/MiniLM-L12-H384-uncased
• Maximum Sequence Length: 512 tokens
• Number of Output Labels: 1 label

Model Sources

• Documentation: Sentence Transformers Documentation
• Documentation: Cross Encoder Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Cross Encoders on Hugging Face

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import CrossEncoder

# Download from the 🤗 Hub
model = CrossEncoder("tomaarsen/reranker-msmarco-MiniLM-L12-H384-uncased-lambdaloss")
# Get scores for pairs of texts
pairs = [
    ['How many calories in an egg', 'There are on average between 55 and 80 calories in an egg depending on its size.'],
    ['How many calories in an egg', 'Egg whites are very low in calories, have no fat, no cholesterol, and are loaded with protein.'],
    ['How many calories in an egg', 'Most of the calories in an egg come from the yellow yolk in the center.'],
]
scores = model.predict(pairs)
print(scores.shape)
# (3,)

# Or rank different texts based on similarity to a single text
ranks = model.rank(
    'How many calories in an egg',
    [
        'There are on average between 55 and 80 calories in an egg depending on its size.',
        'Egg whites are very low in calories, have no fat, no cholesterol, and are loaded with protein.',
        'Most of the calories in an egg come from the yellow yolk in the center.',
    ]
)
# [{'corpus_id': ..., 'score': ...}, {'corpus_id': ..., 'score': ...}, ...]

Evaluation

Metrics

Cross Encoder Reranking

• Datasets: NanoMSMARCO_R100, NanoNFCorpus_R100 and NanoNQ_R100
• Evaluated with CrossEncoderRerankingEvaluator with these parameters:

  {
      "at_k": 10,
      "always_rerank_positives": true
  }
  

Metric	NanoMSMARCO_R100	NanoNFCorpus_R100	NanoNQ_R100
map	0.6352 (+0.1456)	0.3389 (+0.0779)	0.7174 (+0.2978)
mrr@10	0.6298 (+0.1523)	0.5872 (+0.0874)	0.7283 (+0.3016)
ndcg@10	0.6981 (+0.1577)	0.4036 (+0.0786)	0.7584 (+0.2577)

Cross Encoder Nano BEIR

• Dataset: NanoBEIR_R100_mean
• Evaluated with CrossEncoderNanoBEIREvaluator with these parameters:

  {
      "dataset_names": [
          "msmarco",
          "nfcorpus",
          "nq"
      ],
      "rerank_k": 100,
      "at_k": 10,
      "always_rerank_positives": true
  }
  

Metric	Value
map	0.5638 (+0.1738)
mrr@10	0.6485 (+0.1805)
ndcg@10	0.6200 (+0.1647)

Training Details

Training Dataset

Unnamed Dataset

• Size: 399,282 training samples
• Columns: query, docs, and labels
• Approximate statistics based on the first 1000 samples:

  	query	docs	labels
  type	string	list	list
  details	min: 6 characters mean: 33.0 characters max: 154 characters	min: 6 elements mean: 13.23 elements max: 20 elements	min: 6 elements mean: 13.23 elements max: 20 elements

• Samples:

  query	docs	labels
  intel current gen core processors	["Identical or more capable versions of Core processors are also sold as Xeon processors for the server and workstation markets. As of 2017 the current lineup of Core processors included the Intel Core i7, Intel Core i5, and Intel Core i3, along with the Y - Series Intel Core CPU's.", "Most noticeably that Panasonic switched from Intel Core 2 Duo power to the latest Intel Core i3 and i5 processors. The three processors available in the new Toughbook 31, together with the new Mobile Intel QM57 Express chipset, are all part of Intel's Calpella platform.", 'The new 7th Gen Intel Core i7-7700HQ processor gives the 14-inch Razer Blade 2.8GHz of quad-core processing power and Turbo Boost speeds, which automatically increases the speed of active cores â\x80\x93 up to 3.8GHz.', 'Key difference: Intel Core i3 is a type of dual-core processor. i5 processors have 2 to 4 cores. A dual-core processor is a type of a central processing unit (CPU) that has two complete execution cores. Hence, it has t...	[1, 0, 0, 0, 0, ...]
  renovation definition	['Renovation is the act of renewing or restoring something. If your kitchen is undergoing a renovation, thereâ\x80\x99s probably plaster and paint all over the place and you should probably get take-out.', 'NEW GALLERY SPACES OPENING IN 2017. In early 2017, our fourth floor will be transformed into a new destination for historical education and innovation. During the current renovation, objects from our permanent collection are on view throughout the Museum.', 'A same level house extension in Australia will cost approximately $60,000 to $200,000+. Adding a room or extending your living area on the ground floor are affordable ways of creating more space.Here are some key points to consider that will help you keep your renovation costs in check.RTICLE Stephanie Matheson. A same level house extension in Australia will cost approximately $60,000 to $200,000+. Adding a room or extending your living area on the ground floor are affordable ways of creating more space. Here are some key points...	[1, 0, 0, 0, 0, ...]
  what is a girasol	['Girasol definition, an opal that reflects light in a bright luminous glow. See more.', 'Also, a type of opal from Mexico, referred to as Mexican water opal, is a colorless opal which exhibits either a bluish or golden internal sheen. Girasol opal is a term sometimes mistakenly and improperly used to refer to fire opals, as well as a type of transparent to semitransparent type milky quartz from Madagascar which displays an asterism, or star effect, when cut properly.', 'What is the meaning of Girasol? How popular is the baby name Girasol? Learn the origin and popularity plus how to pronounce Girasol', 'There are 5 basic types of opal. These types are Peruvian Opal, Fire Opal, Girasol Opal, Common opal and Precious Opal. There are 5 basic types of opal. These types are Peruvian Opal, Fire Opal, Girasol Opal, Common opal and Precious Opal.', 'girasol (Ë\x88dÊ\x92ɪrÉ\x99Ë\x8csÉ\x92l; -Ë\x8csÉ\x99Ê\x8al) , girosol or girasole n (Jewellery) a type of opal that has a red or pink glow in br...	[1, 0, 0, 0, 0, ...]

• Loss: LambdaLoss with these parameters:

  {
      "weighting_scheme": "sentence_transformers.cross_encoder.losses.LambdaLoss.NDCGLoss2PPScheme",
      "k": null,
      "sigma": 1.0,
      "eps": 1e-10,
      "reduction_log": "binary",
      "activation_fct": "torch.nn.modules.linear.Identity",
      "mini_batch_size": 16
  }
  

Evaluation Dataset

Unnamed Dataset

• Size: 1,000 evaluation samples
• Columns: query, docs, and labels
• Approximate statistics based on the first 1000 samples:

  	query	docs	labels
  type	string	list	list
  details	min: 10 characters mean: 33.63 characters max: 137 characters	min: 3 elements mean: 12.50 elements max: 20 elements	min: 3 elements mean: 12.50 elements max: 20 elements

• Samples:

  query	docs	labels
  can marijuana help dementia	["Cannabis 'could stop dementia in its tracks'. Cannabis may help keep Alzheimer's disease at bay. In experiments, a marijuana-based medicine triggered the formation of new brain cells and cut inflammation linked to dementia. The researchers say that using the information to create a pill suitable for people could help prevent or delay the onset of Alzheimer's.", 'Marijuana (cannabis): Marijuana in any form is not allowed on aircraft and is not allowed in the secure part of the airport (beyond the TSA screening areas). In addition it is illegal to import marijuana or marijuana-related items into the US.', 'Depakote and dementia - Can dementia be cured? Unfortunately, no. Dementia is a progressive disease. Even available treatments only slow progression or tame symptoms.', 'Marijuana Prices. The price of marijuana listed below is the typical price to buy marijuana on the black market in U.S. dollars. How much marijuana cost and the sale price of marijuana are based upon the United Natio...	[1, 0, 0, 0, 0, ...]
  what are carcinogen	['Written By: Carcinogen, any of a number of agents that can cause cancer in humans. They can be divided into three major categories: chemical carcinogens (including those from biological sources), physical carcinogens, and oncogenic (cancer-causing) viruses. 1 Most carcinogens, singly or in combination, produce cancer by interacting with DNA in cells and thereby interfering with normal cellular function.', 'Tarragon (Artemisia dracunculus) is a species of perennial herb in the sunflower family. It is widespread in the wild across much of Eurasia and North America, and is cultivated for culinary and medicinal purposes in many lands.One sub-species, Artemisia dracunculus var. sativa, is cultivated for use of the leaves as an aromatic culinary herb.arragon has an aromatic property reminiscent of anise, due to the presence of estragole, a known carcinogen and teratogen in mice. The European Union investigation revealed that the danger of estragole is minimal even at 100â\x80\x931,000 tim...	[1, 0, 0, 0, 0, ...]
  who played ben geller in friends	["Noelle and Cali aren't the only twins to have played one child character in Friends. Double vision: Ross' cheeky son Ben (pictured), from his first marriage to Carol, was also played by twins, Dylan and Cole Sprouse, who are now 22.", 'Update 7/29/06: There are now three â\x80\x9cTeaching Pastorsâ\x80\x9d at Applegate Christian Fellowship, according to their web site. Jon Courson is now back at Applegate. The other two listed as Teaching Pastors are Jonâ\x80\x99s two sons: Peter John and Ben Courson.on Courson has been appreciated over the years by many people who are my friends and whom I respect. I believe that he preaches the real Jesus and the true Gospel, for which I rejoice. I also believe that his ministry and church organization is a reasonable example with which to examine important issues together.', 'Ben 10 (Reboot) Ben 10: Omniverse is the fourth iteration of the Ben 10 franchise, and it is the sequel of Ben 10: Ultimate Alien. Ben was all set to be a solo hero with his n...	[1, 0, 0, 0, 0, ...]

• Loss: LambdaLoss with these parameters:

  {
      "weighting_scheme": "sentence_transformers.cross_encoder.losses.LambdaLoss.NDCGLoss2PPScheme",
      "k": null,
      "sigma": 1.0,
      "eps": 1e-10,
      "reduction_log": "binary",
      "activation_fct": "torch.nn.modules.linear.Identity",
      "mini_batch_size": 16
  }
  

Training Hyperparameters

Non-Default Hyperparameters

• eval_strategy: steps
• per_device_train_batch_size: 16
• per_device_eval_batch_size: 16
• learning_rate: 2e-05
• num_train_epochs: 1
• warmup_ratio: 0.1
• seed: 12
• bf16: True
• load_best_model_at_end: True

All Hyperparameters

Click to expand

• overwrite_output_dir: False
• do_predict: False
• eval_strategy: steps
• prediction_loss_only: True
• per_device_train_batch_size: 16
• per_device_eval_batch_size: 16
• per_gpu_train_batch_size: None
• per_gpu_eval_batch_size: None
• gradient_accumulation_steps: 1
• eval_accumulation_steps: None
• torch_empty_cache_steps: None
• learning_rate: 2e-05
• weight_decay: 0.0
• adam_beta1: 0.9
• adam_beta2: 0.999
• adam_epsilon: 1e-08
• max_grad_norm: 1.0
• num_train_epochs: 1
• max_steps: -1
• lr_scheduler_type: linear
• lr_scheduler_kwargs: {}
• warmup_ratio: 0.1
• warmup_steps: 0
• log_level: passive
• log_level_replica: warning
• log_on_each_node: True
• logging_nan_inf_filter: True
• save_safetensors: True
• save_on_each_node: False
• save_only_model: False
• restore_callback_states_from_checkpoint: False
• no_cuda: False
• use_cpu: False
• use_mps_device: False
• seed: 12
• data_seed: None
• jit_mode_eval: False
• use_ipex: False
• bf16: True
• fp16: False
• fp16_opt_level: O1
• half_precision_backend: auto
• bf16_full_eval: False
• fp16_full_eval: False
• tf32: None
• local_rank: 0
• ddp_backend: None
• tpu_num_cores: None
• tpu_metrics_debug: False
• debug: []
• dataloader_drop_last: False
• dataloader_num_workers: 0
• dataloader_prefetch_factor: None
• past_index: -1
• disable_tqdm: False
• remove_unused_columns: True
• label_names: None
• load_best_model_at_end: True
• ignore_data_skip: False
• fsdp: []
• fsdp_min_num_params: 0
• fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
• fsdp_transformer_layer_cls_to_wrap: None
• accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
• deepspeed: None
• label_smoothing_factor: 0.0
• optim: adamw_torch
• optim_args: None
• adafactor: False
• group_by_length: False
• length_column_name: length
• ddp_find_unused_parameters: None
• ddp_bucket_cap_mb: None
• ddp_broadcast_buffers: False
• dataloader_pin_memory: True
• dataloader_persistent_workers: False
• skip_memory_metrics: True
• use_legacy_prediction_loop: False
• push_to_hub: False
• resume_from_checkpoint: None
• hub_model_id: None
• hub_strategy: every_save
• hub_private_repo: None
• hub_always_push: False
• gradient_checkpointing: False
• gradient_checkpointing_kwargs: None
• include_inputs_for_metrics: False
• include_for_metrics: []
• eval_do_concat_batches: True
• fp16_backend: auto
• push_to_hub_model_id: None
• push_to_hub_organization: None
• mp_parameters:
• auto_find_batch_size: False
• full_determinism: False
• torchdynamo: None
• ray_scope: last
• ddp_timeout: 1800
• torch_compile: False
• torch_compile_backend: None
• torch_compile_mode: None
• dispatch_batches: None
• split_batches: None
• include_tokens_per_second: False
• include_num_input_tokens_seen: False
• neftune_noise_alpha: None
• optim_target_modules: None
• batch_eval_metrics: False
• eval_on_start: False
• use_liger_kernel: False
• eval_use_gather_object: False
• average_tokens_across_devices: False
• prompts: None
• batch_sampler: batch_sampler
• multi_dataset_batch_sampler: proportional

Training Logs

Click to expand

Epoch	Step	Training Loss	Validation Loss	NanoMSMARCO_R100_ndcg@10	NanoNFCorpus_R100_ndcg@10	NanoNQ_R100_ndcg@10	NanoBEIR_R100_mean_ndcg@10
-1	-1	-	-	0.0086 (-0.5318)	0.2639 (-0.0612)	0.0000 (-0.5006)	0.0908 (-0.3645)
0.0000	1	0.8912	-	-	-	-	-
0.0080	200	0.8246	-	-	-	-	-
0.0160	400	0.8117	-	-	-	-	-
0.0240	600	0.4376	-	-	-	-	-
0.0321	800	0.3271	-	-	-	-	-
0.0401	1000	0.2819	0.2442	0.6288 (+0.0884)	0.4289 (+0.1039)	0.7117 (+0.2111)	0.5898 (+0.1344)
0.0481	1200	0.24	-	-	-	-	-
0.0561	1400	0.2296	-	-	-	-	-
0.0641	1600	0.2244	-	-	-	-	-
0.0721	1800	0.2057	-	-	-	-	-
0.0801	2000	0.1947	0.1775	0.6251 (+0.0846)	0.4318 (+0.1068)	0.7111 (+0.2105)	0.5893 (+0.1340)
0.0882	2200	0.1939	-	-	-	-	-
0.0962	2400	0.1996	-	-	-	-	-
0.1042	2600	0.1938	-	-	-	-	-
0.1122	2800	0.1928	-	-	-	-	-
0.1202	3000	0.1915	0.1684	0.6226 (+0.0822)	0.4140 (+0.0890)	0.7063 (+0.2057)	0.5810 (+0.1256)
0.1282	3200	0.1898	-	-	-	-	-
0.1362	3400	0.1931	-	-	-	-	-
0.1443	3600	0.1834	-	-	-	-	-
0.1523	3800	0.1813	-	-	-	-	-
0.1603	4000	0.1722	0.1594	0.6584 (+0.1180)	0.4167 (+0.0916)	0.7031 (+0.2025)	0.5927 (+0.1374)
0.1683	4200	0.1759	-	-	-	-	-
0.1763	4400	0.187	-	-	-	-	-
0.1843	4600	0.1682	-	-	-	-	-
0.1923	4800	0.1813	-	-	-	-	-
0.2004	5000	0.1744	0.1541	0.6275 (+0.0871)	0.4591 (+0.1341)	0.7101 (+0.2095)	0.5989 (+0.1435)
0.2084	5200	0.164	-	-	-	-	-
0.2164	5400	0.1758	-	-	-	-	-
0.2244	5600	0.1715	-	-	-	-	-
0.2324	5800	0.1766	-	-	-	-	-
0.2404	6000	0.1633	0.1513	0.5947 (+0.0543)	0.4002 (+0.0751)	0.7161 (+0.2155)	0.5703 (+0.1150)
0.2484	6200	0.1675	-	-	-	-	-
0.2565	6400	0.1615	-	-	-	-	-
0.2645	6600	0.1697	-	-	-	-	-
0.2725	6800	0.1743	-	-	-	-	-
0.2805	7000	0.1781	0.1539	0.6461 (+0.1056)	0.4281 (+0.1030)	0.7288 (+0.2281)	0.6010 (+0.1456)
0.2885	7200	0.1796	-	-	-	-	-
0.2965	7400	0.1681	-	-	-	-	-
0.3045	7600	0.1746	-	-	-	-	-
0.3126	7800	0.1726	-	-	-	-	-
0.3206	8000	0.1625	0.1474	0.6162 (+0.0757)	0.4363 (+0.1113)	0.7352 (+0.2346)	0.5959 (+0.1405)
0.3286	8200	0.1574	-	-	-	-	-
0.3366	8400	0.1672	-	-	-	-	-
0.3446	8600	0.1766	-	-	-	-	-
0.3526	8800	0.1714	-	-	-	-	-
0.3606	9000	0.163	0.1497	0.6337 (+0.0933)	0.4559 (+0.1308)	0.7306 (+0.2300)	0.6067 (+0.1513)
0.3686	9200	0.1626	-	-	-	-	-
0.3767	9400	0.1638	-	-	-	-	-
0.3847	9600	0.1603	-	-	-	-	-
0.3927	9800	0.1689	-	-	-	-	-
0.4007	10000	0.1629	0.1500	0.6451 (+0.1046)	0.4330 (+0.1080)	0.7338 (+0.2332)	0.6040 (+0.1486)
0.4087	10200	0.1644	-	-	-	-	-
0.4167	10400	0.1596	-	-	-	-	-
0.4247	10600	0.1655	-	-	-	-	-
0.4328	10800	0.1596	-	-	-	-	-
0.4408	11000	0.1608	0.1416	0.6706 (+0.1302)	0.4425 (+0.1174)	0.7462 (+0.2455)	0.6197 (+0.1644)
0.4488	11200	0.1676	-	-	-	-	-
0.4568	11400	0.1642	-	-	-	-	-
0.4648	11600	0.1558	-	-	-	-	-
0.4728	11800	0.1582	-	-	-	-	-
0.4808	12000	0.1605	0.1471	0.6626 (+0.1222)	0.4141 (+0.0890)	0.7162 (+0.2156)	0.5976 (+0.1423)
0.4889	12200	0.1692	-	-	-	-	-
0.4969	12400	0.1592	-	-	-	-	-
0.5049	12600	0.1584	-	-	-	-	-
0.5129	12800	0.1613	-	-	-	-	-
0.5209	13000	0.1626	0.1436	0.6800 (+0.1396)	0.4200 (+0.0949)	0.7336 (+0.2329)	0.6112 (+0.1558)
0.5289	13200	0.1551	-	-	-	-	-
0.5369	13400	0.1622	-	-	-	-	-
0.5450	13600	0.1646	-	-	-	-	-
0.5530	13800	0.1642	-	-	-	-	-
0.5610	14000	0.1697	0.1396	0.6808 (+0.1403)	0.4255 (+0.1005)	0.7257 (+0.2250)	0.6107 (+0.1553)
0.5690	14200	0.1565	-	-	-	-	-
0.5770	14400	0.158	-	-	-	-	-
0.5850	14600	0.1497	-	-	-	-	-
0.5930	14800	0.1627	-	-	-	-	-
0.6011	15000	0.1599	0.1374	0.6647 (+0.1243)	0.4185 (+0.0935)	0.7465 (+0.2458)	0.6099 (+0.1545)
0.6091	15200	0.1586	-	-	-	-	-
0.6171	15400	0.1566	-	-	-	-	-
0.6251	15600	0.158	-	-	-	-	-
0.6331	15800	0.1693	-	-	-	-	-
0.6411	16000	0.157	0.1377	0.6844 (+0.1440)	0.4022 (+0.0771)	0.7715 (+0.2708)	0.6193 (+0.1640)
0.6491	16200	0.1508	-	-	-	-	-
0.6572	16400	0.1477	-	-	-	-	-
0.6652	16600	0.1589	-	-	-	-	-
0.6732	16800	0.148	-	-	-	-	-
0.6812	17000	0.153	0.1376	0.6835 (+0.1431)	0.4230 (+0.0980)	0.7471 (+0.2464)	0.6179 (+0.1625)
0.6892	17200	0.1599	-	-	-	-	-
0.6972	17400	0.152	-	-	-	-	-
0.7052	17600	0.1516	-	-	-	-	-
0.7133	17800	0.1537	-	-	-	-	-
0.7213	18000	0.1579	0.1386	0.6919 (+0.1514)	0.4111 (+0.0860)	0.7572 (+0.2565)	0.6200 (+0.1646)
0.7293	18200	0.1548	-	-	-	-	-
0.7373	18400	0.1492	-	-	-	-	-
0.7453	18600	0.1496	-	-	-	-	-
0.7533	18800	0.1514	-	-	-	-	-
0.7613	19000	0.1538	0.14	0.6981 (+0.1577)	0.4036 (+0.0786)	0.7584 (+0.2577)	0.6200 (+0.1647)
0.7694	19200	0.1504	-	-	-	-	-
0.7774	19400	0.146	-	-	-	-	-
0.7854	19600	0.1467	-	-	-	-	-
0.7934	19800	0.1542	-	-	-	-	-
0.8014	20000	0.1567	0.1365	0.6786 (+0.1382)	0.4081 (+0.0831)	0.7565 (+0.2559)	0.6144 (+0.1591)
0.8094	20200	0.1561	-	-	-	-	-
0.8174	20400	0.1444	-	-	-	-	-
0.8255	20600	0.15	-	-	-	-	-
0.8335	20800	0.1552	-	-	-	-	-
0.8415	21000	0.1548	0.1368	0.6786 (+0.1381)	0.4111 (+0.0860)	0.7544 (+0.2537)	0.6147 (+0.1593)
0.8495	21200	0.1533	-	-	-	-	-
0.8575	21400	0.1538	-	-	-	-	-
0.8655	21600	0.1486	-	-	-	-	-
0.8735	21800	0.1542	-	-	-	-	-
0.8816	22000	0.1536	0.1369	0.6670 (+0.1266)	0.4102 (+0.0851)	0.7504 (+0.2497)	0.6092 (+0.1538)
0.8896	22200	0.1604	-	-	-	-	-
0.8976	22400	0.1498	-	-	-	-	-
0.9056	22600	0.1563	-	-	-	-	-
0.9136	22800	0.154	-	-	-	-	-
0.9216	23000	0.1553	0.1363	0.6845 (+0.1441)	0.4134 (+0.0884)	0.7447 (+0.2441)	0.6142 (+0.1589)
0.9296	23200	0.1488	-	-	-	-	-
0.9377	23400	0.1489	-	-	-	-	-
0.9457	23600	0.1456	-	-	-	-	-
0.9537	23800	0.1561	-	-	-	-	-
0.9617	24000	0.1485	0.1374	0.6811 (+0.1407)	0.4111 (+0.0861)	0.7516 (+0.2510)	0.6146 (+0.1592)
0.9697	24200	0.1462	-	-	-	-	-
0.9777	24400	0.1472	-	-	-	-	-
0.9857	24600	0.1536	-	-	-	-	-
0.9937	24800	0.157	-	-	-	-	-
-1	-1	-	-	0.6981 (+0.1577)	0.4036 (+0.0786)	0.7584 (+0.2577)	0.6200 (+0.1647)

• The bold row denotes the saved checkpoint.

Environmental Impact

Carbon emissions were measured using CodeCarbon.

• Energy Consumed: 2.214 kWh
• Carbon Emitted: 0.861 kg of CO2
• Hours Used: 7.301 hours

Training Hardware

• On Cloud: No
• GPU Model: 1 x NVIDIA GeForce RTX 3090
• CPU Model: 13th Gen Intel(R) Core(TM) i7-13700K
• RAM Size: 31.78 GB

Framework Versions

• Python: 3.11.6
• Sentence Transformers: 3.5.0.dev0
• Transformers: 4.49.0
• PyTorch: 2.6.0+cu124
• Accelerate: 1.4.0
• Datasets: 3.3.2
• Tokenizers: 0.21.0

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

LambdaLoss

@inproceedings{wang2018lambdaloss,
  title={The lambdaloss framework for ranking metric optimization},
  author={Wang, Xuanhui and Li, Cheng and Golbandi, Nadav and Bendersky, Michael and Najork, Marc},
  booktitle={Proceedings of the 27th ACM international conference on information and knowledge management},
  pages={1313--1322},
  year={2018}
}