CephGPT-4: An Interactive Multimodal Cephalometric Measurement and
  Diagnostic System with Visual Large Language Model
Lei Ma Jincong Han Zhaoxin Wang Dian Zhang
Large-scale multimodal language models (LMMs) have achieved remarkable
success in general domains. However, the exploration of diagnostic language
models based on multimodal cephalometric medical data remains limited. In this
paper, we propose a novel multimodal cephalometric analysis and diagnostic
dialogue model. Firstly, a multimodal orthodontic medical dataset is
constructed, comprising cephalometric images and doctor-patient dialogue data,
with automatic analysis of cephalometric landmarks using U-net and generation
of diagnostic reports. Then, the cephalometric dataset and generated diagnostic
reports are separately fine-tuned on Minigpt-4 and VisualGLM. Results
demonstrate that the CephGPT-4 model exhibits excellent performance and has the
potential to revolutionize orthodontic measurement and diagnostic applications.
These innovations hold revolutionary application potential in the field of
orthodontics.