基于图文多模态融合推理的产品创新方案设计方法研究  

Innovative Product Design Schemes Based on Image-text Multi-modal Fusion Reasoning

作　　者：马进[1] 范明浩 马良山 胡洁 MA Jin;FAN Minghao;MA Liangshan;HU Jie(School of Sensing Science and Technology,Shanghai 200240,China;School of Design,Shanghai Jiao Tong University,Shanghai 200240,China;Shanghai China Software Computer Systems Engineering Co.,Ltd.,Shanghai 200001,China)

机构地区：[1]上海交通大学感知科学与工程学院,上海200240 [2]上海交通大学设计学院,上海200240 [3]上海中软计算机系统工程有限公司,上海200001

出　　处：《包装工程》2024年第8期21-28,共8页Packaging Engineering

基　　金：国家自然科学基金面上(52375254);上海交通大学医工交叉项目(21X010301670)。

摘　　要：目的针对当前产品创新设计领域中对基于图像-文本多模态知识支撑创新设计方法研究不足的问题,提出了一套基于图文多模态的产品创新方案设计方法。方法首先,对设计师的设计草图与文本要求进行预处理,然后引入产品设计知识图谱来促进设计思维的发散和创新;其次,通过微调的生成式预训练变换器模型和扩散模型生成产品方案及其概念图;最后,利用深度多模态设计评估模型对产品设计方案的可行性和市场潜力进行评估。结果通过产品设计知识图谱,及深度多模态设计评估模型的引入,该设计流程可以生成富有创新性且具备可行性的产品方案。结论基于图文多模态的产品创新方案设计流程结合了最新的深度学习技术,不仅提高了设计的效率,还为设计师提供了更广阔的创新视角和灵感来源。The work aims to propose a novel multi-modal process which integrates both image and text elements for innovative product design to address the issue of insufficient innovation and feasibility in product design schemes within the field of AI-assisted product design.The work begins with preprocessing the designer's sketches and textual require-ments,followed by the incorporation of a product design knowledge graph to facilitate divergent thinking and innovation.Subsequently,a fine-tuned generative pre-trained Transformer model and a diffusion model were employed to generate product schemes and their conceptual diagrams.Finally,a deep multi-modal design assessment model was adopted to evaluate the feasibility and market potential of the product design schemes.The results indicated that the introduction of the product design knowledge graph and the deep multi-modal design assessment model enabled the generation of inno-vative product schemes that also possessed feasibility.In conclusion,this multi-modal approach to innovative product scheme design,leveraging cutting-edge AI and deep learning technologies,not only enhances design efficiency but also provides designers with a broader perspective for innovation and inspiration sources.

关 键 词：图文多模态 深度生成模型 知识图谱 产品创新设计 

分 类 号：TB472[一般工业技术—工业设计]