基于大数据技术的AI岗位需求分析研究  

Research on AI Job Demand Analysis Based on Big Data Technology

作　　者：徐正丽 文博奚 谢梅英 蔡翔[1] XU Zhengli;WEN Boxi;XIE Meiying;CAI Xiang(Guilin University of Electronic Technology,Guilin,Guangxi,541004,China;Guangxi Polytechnic of Construction,Nanning,Guangxi,530007,China;Nanjing University of Information Science & Technology,Nanjing,Jiangsu,210044,China)

机构地区：[1]桂林电子科技大学,广西桂林541004 [2]广西建设职业技术学院,广西南宁530007 [3]南京信息工程大学,江苏南京210044

出　　处：《广西科学》2021年第3期321-329,共9页Guangxi Sciences

基　　金：国家自然科学基金项目(71463010);教育部人文社会科学研究专项(17JDGC022);江苏省研究生教育教学改革课题重点课题(JGZZ19_021)资助。

摘　　要：近年来,我国人才市场出现供需失配的结构性矛盾,尤其是在人工智能(AI)领域。准确感知并描述劳动力市场的需求是解决该问题的重要手段。本研究首先使用网络爬虫抓取智联招聘网站发布的AI岗位相关招聘信息,通过中文分词、K-means等大数据分析方法对招聘岗位名称进行聚类处理,识别出软件工程师、算法工程师、产品经理及产品架构师等4个岗位簇;然后利用概率主题模型(Latent Dirichlet Allocation,LDA)对招聘岗位要求继续进行聚类处理,得到数据库、机器学习、模式识别、大数据、程序设计等5个技能集;最后利用LDA求得岗位簇对其技能集的需求矩阵,以分析各岗位簇对其岗位技能的需求程度。结果表明:程序设计能力对AI软件工程师最重要,模式识别的理论与技术对算法工程师最重要;产品经理岗位对数据库、机器学习和大数据技术等均有较强的技能需求;机器学习的理论与技术对产品架构师最重要。本研究成果可为高校、企业常态化或实时准确感知并描述AI劳动力市场需求提供技术支持。In recent years,there has been a structural contradiction of mismatch between supply and demand in China's talent market,especially in the field of artificial intelligence(AI).Accurately perceiving and describing the demand of the labor market is an important method to solve this problem.This study first takes a web crawler to capture AI job-related recruitment information published by the recruitment website of Zhilian.Through Chinese word segmentation,K-means and other big data analysis methods,the recruitment job names are clustered to identify four job clusters:Software engineers,algorithm engineers,product managers and product architects.Then,the model of Latent Dirichlet Allocation(LDA)is used to cluster the job requirements,and five skill sets including database,machine learning,pattern recognition,big data and program design are obtained.Finally,LDA is used to obtain the demand matrix of job clusters for their skill sets to analyze the demand degree of each job cluster to their job skills.The results show that the programming ability is the most important for AI software engineers,and the theory and technology of pattern recognition are the most important for algorithm engineers.Product manager positions have strong skills requirements for databases,machine learning and big data technology.The theory and technology of machine learning are the most important to product architects.The results of this research can provide technical support for universities and enterprises to accurately perceive and describe the needs of the AI labor market in real time.

关 键 词：数据分析 人工智能 网络爬虫 岗位角色 岗位技能 岗位词典 

分 类 号：TP18[自动化与计算机技术—控制理论与控制工程]