干旱区生态环境知识资源中心(Arid): An Efficient Model-Free Approach for Controlling Large-Scale Canals via Hierarchical Reinforcement Learning

Arid

DOI	10.1109/TII.2020.3004857
	An Efficient Model-Free Approach for Controlling Large-Scale Canals via Hierarchical Reinforcement Learning
	Ren, Tao; Niu, Jianwei; Liu, Xuefeng; Wu, Jiyan; Lei, Xiaohui; Zhang, Zhao
通讯作者	Liu, XF (corresponding author), Beihang Univ, Sch Comp Sci & Engn, State Key Lab Virtual Real Technol & Syst, Beijing 100191, Peoples R China. ; Liu, XF (corresponding author), Beihang Univ, Sch Comp Sci & Engn, Beijing Adv Innovat Ctr Big Data & Brain Comp BDB, Beijing 100191, Peoples R China.
来源期刊	IEEE TRANSACTIONS ON INDUSTRIAL INFORMATICS
ISSN	1551-3203
EISSN	1941-0050
出版年	2021
卷号	17 期号:6 页码:4367-4378
英文摘要	Large-scale canals with cascaded pools are constructed wordwide to divert water from rich to arid areas to mitigate water shortages. Efficient control of canals is essential to improve water-diversion performance. Numerous model-based approaches have been proposed and made great progress for canal control. However, when the predictive model is unavailable or unpromising for long time step predictions, model-free approaches could be considered as a possible way to achieve efficient control. Since most existing model-free approaches are focused on control of small canals or reservoirs, this article proposes a new control approach named policy and action reinforcement learning (PARL) for large-scale canals. We leverage the idea of divide and conquer to decompose the control task of large-scale canals into policy learning and action learning subtasks, and develop PARL by means of hierarchical reinforcement learning. Extensive experiments are conducted via numerical simulation on the case study of Chinese South to North Water Transfer Project, and experimental results show that PARL can achieve desirable performance improvements over other model-free learning approaches.
英文关键词	Irrigation Water resources Logic gates Predictive models Reinforcement learning Genetic algorithms Numerical models Canal control hierarchical reinforcement learning (HRL) model-free multipool south to north water transfer project
类型	Article
语种	英语
收录类别	SCI-E
WOS记录号	WOS:000626556300063
WOS类目	Automation & Control Systems ; Computer Science, Interdisciplinary Applications ; Engineering, Industrial
WOS研究方向	Automation & Control Systems ; Computer Science ; Engineering
资源类型	期刊论文
条目标识符	http://119.78.100.177/qdio/handle/2XILL650/350539
作者单位	[Ren, Tao; Niu, Jianwei; Liu, Xuefeng; Wu, Jiyan] Beihang Univ, Sch Comp Sci & Engn, State Key Lab Virtual Real Technol & Syst, Beijing 100191, Peoples R China; [Ren, Tao; Niu, Jianwei; Liu, Xuefeng; Wu, Jiyan] Beihang Univ, Sch Comp Sci & Engn, Beijing Adv Innovat Ctr Big Data & Brain Comp BDB, Beijing 100191, Peoples R China; [Ren, Tao; Niu, Jianwei; Liu, Xuefeng; Wu, Jiyan] Beihang Univ, Hangzhou Innovat Inst, Hangzhou 310051, Peoples R China; [Niu, Jianwei] Zhengzhou Univ, Res Inst Ind Technol, Zhengzhou 450001, Peoples R China; [Lei, Xiaohui; Zhang, Zhao] China Inst Water Resources & Hydropower Res, Beijing 100038, Peoples R China
推荐引用方式 GB/T 7714	Ren, Tao,Niu, Jianwei,Liu, Xuefeng,et al. An Efficient Model-Free Approach for Controlling Large-Scale Canals via Hierarchical Reinforcement Learning[J],2021,17(6):4367-4378.
APA	Ren, Tao,Niu, Jianwei,Liu, Xuefeng,Wu, Jiyan,Lei, Xiaohui,&Zhang, Zhao.(2021).An Efficient Model-Free Approach for Controlling Large-Scale Canals via Hierarchical Reinforcement Learning.IEEE TRANSACTIONS ON INDUSTRIAL INFORMATICS,17(6),4367-4378.
MLA	Ren, Tao,et al."An Efficient Model-Free Approach for Controlling Large-Scale Canals via Hierarchical Reinforcement Learning".IEEE TRANSACTIONS ON INDUSTRIAL INFORMATICS 17.6(2021):4367-4378.