FRAMEWORK TO IMPROVE PARALLEL JOB WORKFLOW

摘要

Embodiments of the inventive subject matter include receiving, in a distributed computing environment, a plurality of files for execution. Embodiments further include identifying, by parsing the plurality of files, code segments contained in each of the plurality of files. Embodiments further include determining, based on a comparison of the code segments and definitions contained in a distributed computing basic function library, a first group of the code segments that include configuration tasks and a second group of the code segments that include computational tasks. Embodiments further include combining the first group of the code segments to form a super configuration task. Embodiments further include creating an executable code, wherein the executable code comprises the super configuration task and the second group of code segments. Embodiments further include allocating the executable code to one or more nodes. Embodiments further include executing the executable code on the one or more nodes.

主权项

1. A method comprising:
receiving, in a distributed computing environment, a plurality of files for execution; identifying, by parsing the plurality of files, code segments contained in each of the plurality of files; determining, based on a comparison of the code segments and definitions contained in a distributed computing basic function library, a first group of the code segments that include configuration tasks and a second group of the code segments that include computational tasks; combining the first group of the code segments to form a super configuration task; creating an executable code, wherein the executable code comprises the super configuration task and the second group of the code segments; allocating the executable code to one or more nodes of the distributed computing environment; and executing the executable code on the one or more nodes of the distributed computing environment.