A Semantic Understanding Oriented Evaluation of the Intelligent Q&A Service on Chinese Provincial Government Websites

WANG Fang, WEI Zhonghan, LIAN Zhixuan, KANG Jia

Scientific Information Research ›› 2023, Vol. 5 ›› Issue (3) : 67-84.

PDF(5254 KB)
PDF(5254 KB)
Scientific Information Research ›› 2023, Vol. 5 ›› Issue (3) : 67-84. DOI: 10.19809/j.cnki.kjqbyj.2023.03.006

A Semantic Understanding Oriented Evaluation of the Intelligent Q&A Service on Chinese Provincial Government Websites

  • WANG Fang1,2, WEI Zhonghan3, LIAN Zhixuan1,2, KANG Jia1,2
Author information +
History +

Abstract

[Purpose/significance]Intelligent question answering （Q&A） system has become an important facility for websites to provide consulting services. The complexity of government consultation issues poses higher requirements for the semantic understanding ability of intelligent Q&A systems on government websites.[Method/process]This study evaluates the Q&A systems of 30 Chinese provincial government websites from three aspects of problem solving quality,service interaction quality and basic construction quality by using the "Semantic Understanding based Evaluation Indicator System for Intelligent Q&A Service on Government Websites" developed by the Center for Network Society Governance of Nankai University and the supporting test sets.[Result/conclusion]The results show that Shanghai, Zhejiang and Beijing rank in the top three in terms of total scores. The Q&A systems on government websites have significant shortcomings in semantic understanding and scenario based services, with only 30% scoring above the passing line. Some Q&A systems have significant room for improvement in basic functionality and service interaction. Finally, this study proposes a few of suggestions to enhance the semantic understanding ability of government Q&A systems, such as expanding knowledge base, improving the accuracy of problem matching, and increasing humanistic care.

Key words

Government Website / intelligent customer service / semantic understanding ability / Q&A system / quality evaluation / dialogue robot

Cite this article

EndNote

Ris (Procite)

Bibtex

Download Citations
WANG Fang, WEI Zhonghan, LIAN Zhixuan, KANG Jia. A Semantic Understanding Oriented Evaluation of the Intelligent Q&A Service on Chinese Provincial Government Websites[J]. Scientific Information Research, 2023, 5(3): 67-84 https://doi.org/10.19809/j.cnki.kjqbyj.2023.03.006

References

[1] 中华人民共和国中央人民政府.国务院办公厅关于印发政府网站发展指引的通知:国办发[2017]47号[A/OL].(2017-06-08)[2021-12-31].www.gov.cn/zhengce/content/2017-06/08/content_5200760.htm.
[2] 周斐,李锦芝.基于人工智能的广州市政府智能服务机器人云平台设计及应用[J].数字技术与应用,2018,36(12):120-122.
[3] 央广网.商务部联手京东AI打造政务智能问答系统[EB/OL].(2019-07-15)[2022-02-22].https://baijiahao.baidu.com/s?id=1639129781659108953&wfr=spider&for=pc.
[4] 亓廷廷.济南市疾病预防控制中心两款智能问答机器人上线[EB/OL].(2020-01-29)[2022-02-22].https://baijiahao.baidu.com/s?id=1657048191273956690&wfr=spider&for=pc.
[5] 王友奎,张楠,赵雪娇.政务服务中的智能问答机器人:现状、机理和关键支撑[J].电子政务,2020(02):34-45.
[6] Sohu city-Tianjin."2022 Data enabling government governance" evaluation index was released,"Chengdu-Chongqing Twin Cities Economic Circle" had the strong momentum of development[EB/OL].(2022-06-11)[2022-12-21].https://3g.k.sohu.com/t/n611346354?serialId=5e31e1e9b574c173a3daf5a3380db33e&showType=.
[7] 肖路. 基于机器阅读理解的政务问答系统研究与实现[D].西安:西安电子科技大学,2021.
[8] 姬美琳. 面向政务领域的知识库问答系统研究[D].武汉:中南民族大学,2021.
[9] 李兆兆. 基于语义理解的智能问答系统关键技术研究[D].西安:西安邮电大学,2019.
[10] 张恺. 基于DEA和多元智能的教育信息管理理论与模型研究[D].天津:天津大学,2010.
[11] 郭艳华,周昌乐.自然语言理解研究综述[J].杭州电子工业学院学报,2000(01):58-65.
[12] 钟义信. 自然语言理解的全信息方法论[J].北京邮电大学学报,2004,27(04):1-12.
[13] NORASETA T,LOWPHANSIRIKUL L,Tuarob S.WabiQA:A Wikipedia-Based Thai Question-Answering System[J].Information Processing & Management,2020,58(01):209-216.
[14] STORKS S,GAO Q,CHAI J Y.Commonsense reasoning for natural language understanding:A survey of benchmarks,resources,and approaches[J].arXiv preprint arXiv:1904.01172,2019.
[15] 陈涛,冉龙亚,明承瀚.政务服务的人工智能应用研究[J].电子政务,2018(03):22-30.
[16] ADAN B G,LEONARDO M G.Question answering systems in e-government:a proposal for the Colombian context[C]//Proceedings of the International Conference on Advances in Information Processing and Communication Technology-IPCT:2014.
[17] CHAN H Y,TSAI M H.Question-answering dialogue system for emergency operations[J].International Journal of Disaster Risk Reduction,2019,41:1-13.
[18] ANDROUTSOPOULOU A,KARACAPILIDIS N,LOUKIS E,et al.Transforming the communication between citizens and government through AI-guided chatbots[J].Government Information Quarterly,2019,36(02):358-367.
[19] 谢炯坤. 面向人机互动的自然语言理解的研究[D].合肥:中国科学技术大学,2015.
[20] TURING A M.Computing machinery and intelligence[J].Mind,1950,59(236):433-460.
[21] 吴友政,赵军,段湘煜,等.问答式检索技术及评测研究综述[J].中文信息学报,2005(03):1-13.
[22] JURCICEK F,KEIZER S,GASIC M,et al.Real user evaluation of spoken dialogue systems using Amazon Mechanical Turk[C]//Proceedings of Interspeech:2011.
[23] 李菁,张海松,宋彦.面向非任务型对话系统的人工标注中文数据集[J].中文信息学报,2019,33(03):17-24.
[24] ROQUE G,SOUZA R,NASCIMENTO J,et al.Content validation and usability of a chatbot of guidelines for wound dressing[J].International Journal of Medical Informatics,2021,151(Jul):1-6.
[25] 李焱冬. 人工智能在电子政务建设中的实例应用初探[J].电子政务,2008(05):87-93.
[26] AOKI N.An experimental study of public trust in AI chatbots in the public sector[J].Government Information Quarterly,2020,37(04):1-10.
[27] 王友奎,张楠,赵雪娇.政务服务中的智能问答机器人:现状、机理和关键支撑[J].电子政务,2020(02):34-45.
[28] 王芳,阴雨轩,刘汪洋,等.我国城市政府运用大数据提升治理效能评价研究[J].图书与情报,2020(02):81-93.
[29] 程璇. 我国政府门户网站评价指标体系研究[D].武汉:华中师范大学,2007.
[30] 徐林明,李美娟.动态综合评价中的数据预处理方法研究[J].中国管理科学,2020,28(01):162-169.
[31] 新华网.“中国政务信息无障碍服务体系”获2017信息社会世界峰会项目大奖[EB/OL].(2017-06-14)[2022-02-22].http://it.people.com.cn/n1/2017/0614/c1009-29338648.html.
PDF(5254 KB)

75

Accesses

0

Citation

Detail
Sections
Recommended

/

〈 〉