An Energy-Efficient and Heterogeneous Environment Adaptive Data Layout Strategy for MapReduce
返回论文页
|更新时间:2023-12-11
|
An Energy-Efficient and Heterogeneous Environment Adaptive Data Layout Strategy for MapReduce
Acta Scientiarum Naturalium Universitatis SunYatseniVol. 54, Issue 6, Pages: 55-66(2015)
作者机构:
1. 新疆财经大学统计与信息学院,新疆,乌鲁木齐,830012
2. 2 新疆大学软件学院,新疆,乌鲁木齐,830008
3.
4. 新疆医科大学医学工程技术学院,新疆,乌鲁木齐,830011
作者简介:
基金信息:
DOI:
CLC:
Published:2015,
Published Online:25 November 2015,
扫 描 看 全 文
LIAO Bin, ZHANG Tao, YU Jiong, et al. An Energy-Efficient and Heterogeneous Environment Adaptive Data Layout Strategy for MapReduce. [J]. Acta Scientiarum Naturalium Universitatis SunYatseni 54(6):55-66(2015)
DOI:
LIAO Bin, ZHANG Tao, YU Jiong, et al. An Energy-Efficient and Heterogeneous Environment Adaptive Data Layout Strategy for MapReduce. [J]. Acta Scientiarum Naturalium Universitatis SunYatseni 54(6):55-66(2015)DOI:
An Energy-Efficient and Heterogeneous Environment Adaptive Data Layout Strategy for MapReduce
The problem of high energy consumption producing from big data processing is an important issue that needs to be solved
especially under the background of data explosion. Based on analyzing problems of the existing data layout policy
the problems of the in adaptation of energy-saving mode based on storage area division and heterogeneous HDFS cluster
the inflexibility of data block segmentation algorithm
the randomness of storage node selection
proposing a data layout strategy orienting to energy conservation are analyzed. Firstly
the new strategy divides the cluster into two different storage areas to meet the needs of saving energy: Active-Zone and Sleep-Zone; secondly
the new strategy has made improvements on traditional data block computing method
proposes a minimum number of jobs calculation method to determine the number of data blocks; at last
the new strategy can increase the adaptability of the heterogeneous cluster environment and can choose the appropriate storage nodes according to different job types. Experimental results show that the new data layout strategy can adapt to the heterogeneous cluster environment and reach the goal of reducing energy consumption for MapReduce jobs.
关键词
绿色计算MapReduce异构环境数据布局
Keywords
green computingMapReduceheterogeneous environmentdata layout