干旱区生态环境知识资源中心(Arid): Bulk Data Movement for Climate Dataset: Efficient Data Transfer Management with Dynamic Transfer Adjustment

Arid

DOI	10.2172/991958
报告编号	LBNL-3985E
来源ID	OSTI_ID: 991958
	Bulk Data Movement for Climate Dataset: Efficient Data Transfer Management with Dynamic Transfer Adjustment
	Sim, Alexander [Lawrence Berkeley National Lab. (LBNL), Berkeley, CA (United States)]; Balman, Mehmet [Lawrence Berkeley National Lab. (LBNL), Berkeley, CA (United States)]; Williams, Dean N. [Lawrence Livermore National Lab. (LLNL), Livermore, CA (United States)]; Shoshani, Arie [Lawrence Berkeley National Lab. (LBNL), Berkeley, CA (United States)]; Natarajan, Vijaya [Lawrence Berkeley National Lab. (LBNL), Berkeley, CA (United States)]
英文摘要	Many scientific applications and experiments, such as high energy and nuclear physics, astrophysics, climate observation and modeling, combustion, nano-scale material sciences, and computational biology, generate extreme volumes of data with a large number of files. These data sources are distributed among national and international data repositories, and are shared by large numbers of geographically distributed scientists. A large portion of data is frequently accessed, and a large volume of data is moved from one place to another for analysis and storage. One challenging issue in such efforts is the limited network capacity for moving large datasets to explore and manage. The Bulk Data Mover (BDM), a data transfer management tool in the Earth System Grid (ESG) community, has been managing the massive dataset transfers efficiently with the pre-configured transfer properties in the environment where the network bandwidth is limited. Dynamic transfer adjustment was studied to enhance the BDM to handle significant end-to-end performance changes in the dynamic network environment as well as to control the data transfers for the desired transfer performance. We describe the results from the BDM transfer management for the climate datasets. We also describe the transfer estimation model and results from the dynamic transfer adjustment.
出版年	2010
报告类型	Technical Report
语种	英语
国家	美国
来源学科分类	97 MATHEMATICS AND COMPUTING ; ASTROPHYSICS ; BIOLOGY ; CAPACITY ; CLIMATES ; COMBUSTION ; MANAGEMENT ; NUCLEAR PHYSICS ; PERFORMANCE ; SIMULATION ; STORAGE Bulk data movement, Climate datasets, Earth System Grid, Dynamic transfer adjustment, Transfer estimation model
URL	http://www.osti.gov/scitech/servlets/purl/991958
资源类型	科技报告
条目标识符	http://119.78.100.177/qdio/handle/2XILL650/270063
推荐引用方式 GB/T 7714	Sim, Alexander [Lawrence Berkeley National Lab. ,Balman, Mehmet [Lawrence Berkeley National Lab. ,Williams, Dean N. [Lawrence Livermore National Lab. ,et al. Bulk Data Movement for Climate Dataset: Efficient Data Transfer Management with Dynamic Transfer Adjustment,2010.