Intro to TaskVine at GCASR 2023

Created on April 28, 2023 by Douglas Thain

2023

Prof. Thain gave the afternoon keynote ( Data Intensive Computing with TaskVine ) at the GCASR Workshop in Chicago on April 24th.  TaskVine is our latest workflow execution system, just released in March 2023.

The key idea of TaskVine is to exploit the local storage already embedded into clusters for use by complex workflows:


To accomplish this, each node runs a general purpose worker process that manages cached data under unique names, and runs each task within a private namespace:

You can write workflows directly to the TaskVine API, declaring files and tasks and connecting them together.  Or, TaskVine can be used as an executor under existing workflow systems:


Learn more and try it out here: http://ccl.cse.nd.edu/software/taskvine




Enjoy Reading This Article?

Here are some more articles you might like to read next:

  • Scaling SADE (Safety Aware Drone Ecosystem): A Hybrid UAV Simulation System for High-Fidelity Research
  • Wrangling Massive Tasks Graphs with Dynamic Hierarchical Composition
  • TaskVine Insights - Storage Management: Disk Load Shifting
  • Simulating Digital Agriculture in Near Real-Time with xGFabric
  • Undergraduate Researcher Showcases PLEDGE Project at APANAC 2025 in Panama