metadata
license: mit
datasets:
- mjjung/Charades-VTune
language:
- en
license: mit datasets: - ShuhuaiRen/TimeIT language: - en
TimeChat-7B-Charades-VTune Model
Model details
We trained TimeChat using VTune, a developed instruction-tuning method specifically designed to account for consistency.
For the tuning, we utilized 5K training videos from Charades-STA with 99K automatically generated annotations.
Evaluation
We evaluated the model on Charades-CON and Charades-STA.
Charades-CON
Metric Value Ground 76.2 R-Ground 69.2 (90.8) S-Ground 36.2 (47.5) H-Verify 44.8 (58.8) C-Verify 42.4 (55.7) Charades-STA
Metric Value R@1 IoU=0.3 72.74 R@1 IoU=0.5 58.47 R@1 IoU=0.7 34.70 mIoU 50.65
Paper and Code for more information: Paper, Code
Citation
If you find our research and codes useful, please consider starring our repository and citing our paper:
@article{jung2024consistency,
title={On the Consistency of Video Large Language Models in Temporal Comprehension},
author={Jung, Minjoon and Xiao, Junbin and Zhang, Byoung-Tak and Yao, Angela},
journal={arXiv preprint arXiv:2411.12951},
year={2024}
}