张宇洋,毛晋,孟婵媛,张煜,熊轲

• 1:北京交通大学
• 2:国网能源研究院有限公司

摘要(Abstract)：

针对6G时代多模态、多任务通信需求的不断增长与无线资源受限之间的矛盾，文章提出一种融合经典Transformer和Vision Transformer结构的多模态、多任务语义通信架构(Multimodal Multitask Semantic Communication,M2SC)。该架构以多模态语义编码器、掩码生成模块(Mask Generation Module,MGM)、语义解码器为核心组件，可实现文本、图像多模态语义数据的同时传输。为准确捕捉跨模态数据间关联信息，进一步降低数据冗余，在M2SC框架的编码端，设计基于交叉注意力机制的掩码生成模块，能够有效挖掘多模态数据间的关联关系，并结合信道状况动态筛选关键语义特征。在视觉问答等四项典型任务中进行实验，结果表明所提M2SC架构仅需依托一套训练完成的编解码器神经网络，即可逼近传统单模态、单任务定制的专用语义通信系统的传输性能，避免采用传统方案需部署多套单模态、单任务编解码神经网络才能完成多模态、多任务传输的系统开销问题。在实验室仿真环境下开展的视觉问答任务消融实验结果表明，将MGM模块嵌入M2SC后，在仅牺牲1.5%问答准确率的前提下，便实现了M2SC带宽开销近60%的大幅削减。

关键词(KeyWords)： 人工智能;语义通信;注意力机制;多模态多任务通信

Abstract:

Keywords:

基金项目(Foundation): 国家自然科学基金资助项目“基于信息年龄的射频能量采集无线网络设计理论与方法”（62071033）

作者(Author): 张宇洋,毛晋,孟婵媛,张煜,熊轲

参考文献(References)：

• [1]钱志鸿,肖琳,王雪.面向未来移动网络密集连接的关键技术综述[J].通信学报,2021,42(4):22-43
• [2]牛凯,戴金晟,张平,等.面向6G的语义通信[J].移动通信,2021,45(4):85-90
• [3]MAO J, XIONG K, LIU M, et al. A GAN-Based Semantic Communication for Text Without CSI[J]. IEEE Transactions on Wireless Communications, 2024,23(10):14498-14514
• [4]QIN Z, YING J, XIN G, et al. Ten challenges in semantic communications[J]. China Communications, 2025,22(6):24-43
• [5]杨立,王喜瑜,段向阳,等."智能体为中心"网络技术体系关键技术及趋势展望[J].信息通信技术, 2025,19(2):4-11
• [6]XIE H, QIN Z, LI G Y. Task-Oriented Multi-User Semantic Communications for VQA[J]. IEEE Wireless Communications Letters, 2022, 11(3):553-557
• [7]QIN Z, GAO F, LIN B, et al. A Generalized Semantic Communication System:From Sources to Channels[J].IEEE Wireless Communications, 2023, 30(3):18-26
• [8]JIANG F, DONG L, PENG Y, et al. Large AI Model Empowered Multimodal Semantic Communications[J].IEEE Communications Magazine, 2025, 63(1):76-82
• [9]GU J, ZHANG X, CUI Q, et al. Semantic Communication for Multi-modal Data Transmission[C]//2023International Conference on Wireless Communications and Signal Processing(WCSP). Hangzhou, China:IEEE, 2023:208-213
• [10]NI W, QIN Z, FENG Y, et al. Multi-Task Semantic Communications via Large Models[J]. IEEE Communications Standards Magazine, 2025, 9(4):16-23
• [11]WANG L, WU W, ZHOU F, et al. A Unified Hierarchical Semantic Knowledge Base for Multi-Task Semantic Communication[C]//ICC 2024-IEEE International Conference on Communications. Denver, CO, USA:IEEE, 2024:2937-2943
• [12]ZHU Z, ZHANG R, CHENG X, et al. Synesthesia of machines(SoM)-enabled multi-task semantic communication system[J]. IEEE Transactions on Mobile Computing, 2025, 25(4):4632-4647
• [13]ZHANG G, HU Q, QIN Z, et al. A Unified Multi-Task Semantic Communication System for Multimodal Data[J]. IEEE Transactions on Communications, 2024,72(7):4101-4116
• [14]HU J, WANG F, XU W, et al. Scalable Multi-Task Semantic Communication System with Feature Importance Ranking[C]//ICASSP 2023-2023 IEEE International Conference on Acoustics, Speech and Signal Processing(ICASSP). Rhodes Island, Greece:IEEE, 2023:1-5
• [15]SOCHER R, PERELYGIN A, WU J, et al. Recursive Deep Models for Semantic Compositionality Over a Sentiment Treebank[C]//Proceedings of the 2013Conference on Empirical Methods in Natural Language Processing. Seattle, Washington, USA:Association for Computational Linguistics, 2013:1631-1642
• [16]KRIZHEVSKY A , HINTON G.Learning multiple layers of features from tiny images[D]. 2009
• [17]ANTOL S, AGRAWAL A, LU J, et al. VQA:Visual Question Answering[C]//2015 IEEE International Conference on Computer Vision(ICCV). Santiago,Chile:IEEE, 2015:2425-2433
• [18]DEVLIN J, CHANG M W, LEE K, et al. Bert:Pre-training of deep bidirectional transformers for language understanding[C]//Proceedings of the 2019 conference of the North American chapter of the association for computational linguistics:human language technologies, volume 1(long and short papers). 2019:4171-4186