Workflow management is important to effectively and productively deploying complex computations, e.g., combining chains of dependent and complex tasks such as data gathering, data processing, simulation, training, inference, validation, and visualization. This Pathways to Open-Source Ecosystems (POSE) project seeks to “harmonize” Python programming language based workflow management, build sustainability, and better support complex computational workflows, both in research and commercial environments. Harmony will deliver a software ecosystem that is used by some of the most impactful science projects, from understanding the beginning of the universe to discovering new therapeutics for viruses. This Phase I project will engage in ecosystem discovery, organization and governance, and community building to build the Harmony Open-Source Ecosystem (OSE).Both small and large scale science require workflows to deliver science results. The Harmony OSE will provide new capabilities to integrate, interoperate, and interchange components to create high performance workflows. The project will lower the barrier to implement sophisticated workflows and manage their execution at scale all from a familiar Python interface. Harmony will bring together the community to consider how efforts can be more tightly integrated and managed and how an OSE can benefit all stakeholders.
POSE Phase I: HARMONY: Harmonizing the High Performance Python Workflow Ecosystem
PIs: Douglas Thain (Notre Dame), Kyle Chard (U-Chicago), and Shantenu Jha (BNL)
