| 发明名称 |
DYNAMIC NODE GROUP ALLOCATION |
| 摘要 |
Provided are techniques for improving data locality for parallel applications running in a big data distributed file system with a dynamic node group. In response to a consumer job starting to read one or more files in a big data distributed file system having multiple nodes, node group information for the one or more files to be read is retrieved, wherein the node group information identifies nodes from the multiple nodes on which a producer job wrote the one or more files, and the consumer job is assigned to the nodes identified by the node group information to allow for local reading of the one or more files by the consumer job. |
| 申请公布号 |
US2016366224(A1) |
申请公布日期 |
2016.12.15 |
| 申请号 |
US201514740050 |
申请日期 |
2015.06.15 |
| 申请人 |
International Business Machines Corporation |
发明人 |
Bonagiri Krishna K.;Jacobson Eric A.;Li Yong;Liu Ron E.;Pu Xiaoyan |
| 分类号 |
H04L29/08;G06F17/30;H04L12/923 |
主分类号 |
H04L29/08 |
| 代理机构 |
|
代理人 |
|
| 主权项 |
1. A method, comprising:
in response to a consumer job starting to read one or more files in a big data distributed file system having multiple nodes,
retrieving node group information for the one or more files to be read, wherein the node group information identifies nodes from the multiple nodes on which a producer job wrote the one or more files; andassigning the consumer job to the nodes identified by the node group information to allow for local reading of the one or more files by the consumer job. |
| 地址 |
Armonk NY US |
