CCTools 3.5.0 Released

The Cooperative Computing Lab is pleased to announce the release of version 3.5.0 of the Cooperative Computing Tools, including Parrot, Chirp, Makeflow, WorkQueue, SAND, All-Pairs, and other software.

The software may be downloaded at http://www.nd.edu/~ccl/software/download .

This is a minor release which adds numerous features and fixes several bugs:

• Batch Job Hadoop module for submitting Hadoop jobs supports Hadoop 0.21.0. [Michael Albrecht, Patrick Donnelly]
  
• Improvements to WorkQueue for worker accounting and management. WorkQueue now does master capacity estimation and automatic worker removal. [Li Yu]
  
• WorkQueue now supports cancelling of a submitted task. [Dinesh Rajan]
  
• WorkQueue workers can now report task execution time. [Li Yu]
  
• Improved Batch Job local execution to fork and exec instead of fork and system.
  
• Swig detection improved. [Peter Bui]
  
• Improved and made consistent time formats for catalog server. [Li Yu]
  
• Various corrections to Parrot's directory handling. [Douglas Thain]
  
• Corrected numerous memory leaks and software bugs using
  Valgrind/ccpcheck. [Peter Bui]
  
• WorkQueue workers now check for low disk space. [Dinesh Rajan]
  
• Parrot now supports writable memory mapped files. [Douglas Thain]
  
• WorkQueue MOAB support improved. [Peter Bui, Michael Albrecht]
  
• WorkQueue now has prototype support for work_queue_pool resource management of multiple masters. work_queue_pool is now capable of automatically requesting resources from the underlying batch system as needed by the masters subject to a constraint file. [Li Yu]
  
• WorkQueue now supports FIFO and LIFO task dispatch to workers. [Dinesh Rajan]
  
• WorkQueue now has work_queue_version to differentiate versions of the library. [Peter Bui]
  
• Chirp client status output is now properly sent to stderr. [Patrick Donnelly]
  
• WorkQueue taskid assignment moved to submit from create. Submit now returns this unique id. [Dinesh Rajan]
  
• Makeflow/WorkQueue/Chirp now support selecting an arbitrary port in a range using environment variables TCP_LOW_PORT and TCP_HIGH_PORT. [Patrick Donnelly]
  
• Improved debug output for non-blocking tcp connections. [Li Yu]
  
• WorkQueue task status is now appropriately set to complete when tasks are moved to complete list. [Dinesh Rajan]
  
• Parrot now supports iRODS version 3.1. [Douglas Thain]
  
• Parrot now allows an identity-boxed process to write to a world-writable file. (such as /dev/null) [Douglas Thain]
  
• WorkQueue workers now have a tunable exponential backoff for reconnecting to masters. [Dinesh Rajan]
  
• Updated WorkQueue documentation and examples. [Dinesh Rajan]
  
• Various improvements to WorkQueue Python binding. [Peter Bui, Dinesh Rajan]
  
• Numerous API/code improvements made to WorkQueue. [Li Yu, Dinesh Rajan, Douglas Thain]
  
• Various compatibility improvements for building CCTools. [Douglas Thain, Patrick Donnelly]
  

Thanks goes to the contributors of other minor/bug fix corrections:
Michael Albrecht, Roger Barthelson, Dan Bradley, Peter Bui, Rory
Carmichael, Patrick Donnelly, Michael Hanke, Dinesh Rajan, Nathan
Regola, Douglas Thain, and Li Yu.

Enjoy!