Peng, S., Xiong, T., Xu, C., & Chen, J. (2026). An end-to-end dense video captioning network with multimodal data fusion and multi-level visual feature enhancement via the ViFi-CLIP pretrained model. Journal on Image and Video Processing. https://doi.org/10.1186/s13640-026-00699-2
发布时间：2026-09-17
点击次数：
- 发布时间：
- 2026-09-17
- 所属单位：
- 人工智能教育学部
- 论文名称：
- Peng, S., Xiong, T., Xu, C., & Chen, J. (2026). An end-to-end dense video captioning network with multimodal data fusion and multi-level visual feature enhancement via the ViFi-CLIP pretrained model. Journal on Image and Video Processing. https://doi.org/10.1186/s13640-026-00699-2
- 发表刊物：
- Journal on Image and Video Processing
- 摘要：
- Dense video captioning is a multimodal task that involves visual feature extraction and natural language generation. Previous approaches face several challenges, such as reliance on complex "localize-then-describe" frameworks, limited contextual information, and the potential for inaccurate detection. In this paper, we propose an end-to-end dense video captioning model that leverages multimodal data and multi-level visual features, based on the pretrained ViFi-CLIP model. Our approach constructs an integrated network that unifies video segmentation and caption generation into a single end-to-end framework, effectively utilizing visual, audio, and textual modalities. In selecting visual features, we comprehensively consider the impact of grid and region-based features and employ ViFi-CLIP to map these features into a joint vision–language embedding space, thereby enhancing the semantic relevance of the visual representations. Extensive experiments conducted on the ActivityNet dataset demonstrate the effectiveness of our model. Results show that our approach outperforms PDVC by 8–10% across various metrics, and exceeds MDVC by 20–40% in METEOR and ROUGE-L scores, with over 100% improvements in BLEU-4 and CIDEr scores. Furthermore, our model consistently surpasses existing state-of-the-art dense video captioning methods, highlighting its superior performance.
- 合写作者：
- Xiong Ting,Xu Chenxin
- 第一作者：
- Peng Shixin
- 论文类型：
- 期刊论文
- 通讯作者：
- Chen Jingying
- 学科门类：
- 工学
- 一级学科：
- 计算机科学与技术
- 文献类型：
- J
- ISSN号：
- 3091-454X
- 是否译文：
- 否
- 发表时间：
- 2026-08-11