xGFabric: Coupling Sensor Networks and HPC Facilities with Private 5G Wireless Networks for Real-Time Digital Agriculture
Liubov Kurafeeva, Alan Subedi, Ryan Hartung, Michael Fay, Avhishek Biswas, Shantenu Jha, Ozgur O. Kilic, Chandra Krintz, Andre Merzky, Douglas Thain, Mehmet C. Vuran, and Rich Wolski
In Workshop on Extreme-Scale Experiment-in-the-Loop Computing (XLOOP) at Supercomputing, 2025
@inproceedings{xgfabric-xloop-2025,author={Kurafeeva, Liubov and Subedi, Alan and Hartung, Ryan and Fay, Michael and Biswas, Avhishek and Jha, Shantenu and Kilic, Ozgur O. and Krintz, Chandra and Merzky, Andre and Thain, Douglas and Vuran, Mehmet C. and Wolski, Rich},title={{xGFabric: Coupling Sensor Networks and HPC Facilities with Private 5G Wireless Networks for Real-Time Digital Agriculture}},booktitle={{Workshop on Extreme-Scale Experiment-in-the-Loop Computing (XLOOP) at Supercomputing}},year={2025},cclpaperid={1005},keywords={xgfabric},}
Backpacks for Notebooks: Enabling Containerized Notebook Workflows in Distributed Environments
Md Saiful Islam, Talha Azaz, Raza Ahmad, A D M Shahadat Hossain, Furqan Baig, Shaowen Wang, Kevin Lannon, Tanu Malik, and Douglas Thain
@inproceedings{floability-escience-2025,author={Islam, Md Saiful and Azaz, Talha and Ahmad, Raza and Hossain, A D M Shahadat and Baig, Furqan and Wang, Shaowen and Lannon, Kevin and Malik, Tanu and Thain, Douglas},title={{Backpacks for Notebooks: Enabling Containerized Notebook Workflows in Distributed Environments}},booktitle={{IEEE Conference on eScience}},pages={9},year={2025},cclpaperid={1003},keywords={floability}}
Liberating the Data Aware Scheduler to Achieve Locality in Layered Scientific Workflow Systems
@inproceedings{liberating-escience-2025,author={Thomas, Colin and Thain, Douglas},title={{Liberating the Data Aware Scheduler to Achieve Locality in Layered Scientific Workflow Systems}},booktitle={{IEEE Conference on eScience}},pages={9},year={2025},cclpaperid={1004}}
2024
Reshaping High Energy Physics Applications for Near-Interactive Execution Using TaskVine
Barry Sly-Delgado, Ben Tovar, Jin Zhou, and Douglas Thain
@inproceedings{reshaping-sc-2024,author={Sly-Delgado, Barry and Tovar, Ben and Zhou, Jin and Thain, Douglas},title={{Reshaping High Energy Physics Applications for Near-Interactive Execution Using TaskVine}},booktitle={{ACM/IEEE Supercomputing}},pages={1-11},year={2024},cclpaperid={996},keywords={taskvine},doi={10.1109/SC41406.2024.00068}}
Shepherd: Seamless Integration of Service Workflows into Task-Based Workflows through Log Monitoring
Saiful Islam and Douglas Thain
In Workshop on Workflows at ACM Supercomputing, 2024
@inproceedings{shepherd-works-2024,author={Islam, Saiful and Thain, Douglas},title={{Shepherd: Seamless Integration of Service Workflows into Task-Based Workflows through Log Monitoring}},booktitle={{Workshop on Workflows at ACM Supercomputing}},pages={1-8},year={2024},cclpaperid={997},keywords={shepherd},}
Accelerating Function-Centric Applications by Discovering, Distributing, and Retaining Reusable Context in Workflow Systems
Thanh Son Phung, Colin Thomas, Logan Ward, Kyle Chard, and Douglas Thain
In ACM International Symposium on High-Performance Parallel and Distributed Computing (HPDC), 2024
@inproceedings{function-context-hpdc-2024,author={Phung, Thanh Son and Thomas, Colin and Ward, Logan and Chard, Kyle and Thain, Douglas},title={{Accelerating Function-Centric Applications by Discovering, Distributing, and Retaining Reusable Context in Workflow Systems}},booktitle={{ACM International Symposium on High-Performance Parallel and Distributed Computing (HPDC)}},year={2024},cclpaperid={995},}
Poster: Leveraging Intermediate Data Management with Parsl/TaskVine
Colin Thomas and Douglas Thain
In Greater Chicago Area Systems Research Workshop, 2024
@inproceedings{data-gcasr-2024,author={Thomas, Colin and Thain, Douglas},title={{Poster: Leveraging Intermediate Data Management with Parsl/TaskVine}},booktitle={{Greater Chicago Area Systems Research Workshop}},pages={1},year={2024},cclpaperid={998},keywords={taskvine},}
Poster: Toward High-Performance Simulation of 1000 Drones
Md Saiful Islam and Douglas Thain
In Greater Chicago Area Systems Research Workshop, 2024
@inproceedings{drones-gcasr-2024,author={Islam, Md Saiful and Thain, Douglas},title={{Poster: Toward High-Performance Simulation of 1000 Drones}},booktitle={{Greater Chicago Area Systems Research Workshop}},pages={1},year={2024},cclpaperid={999},}
Poster: Adaptive Task-Oriented Resource Allocation for Large Dynamic Workflows on Opportunistic Resources
Thanh Son Phùng and Douglas Thain
In Greater Chicago Area Systems Research Workshop, 2024
@inproceedings{alloc-gcasr-2024,author={Phùng, Thanh Son and Thain, Douglas},title={{Poster: Adaptive Task-Oriented Resource Allocation for Large Dynamic Workflows on Opportunistic Resources}},booktitle={{Greater Chicago Area Systems Research Workshop}},pages={1},year={2024},cclpaperid={1000},}
Poster: Import Hoisting for Serverless Libraries
Jin Zhou and Douglas Thain
In Greater Chicago Area Systems Research Workshop, 2024
@inproceedings{hoisting-gcasr-2024,author={Zhou, Jin and Thain, Douglas},title={{Poster: Import Hoisting for Serverless Libraries}},booktitle={{Greater Chicago Area Systems Research Workshop}},pages={1},year={2024},cclpaperid={1001},}
Poster: Reshaping High Energy Physics Applications for Near-Interactive Execution Using TaskVine
Barry Sly-Delgado and Douglas Thain
In Greater Chicago Area Systems Research Workshop, 2024
@inproceedings{reshaping-gcasr-2024,author={Sly-Delgado, Barry and Thain, Douglas},title={{Poster: Reshaping High Energy Physics Applications for Near-Interactive Execution Using TaskVine}},booktitle={{Greater Chicago Area Systems Research Workshop}},pages={1},year={2024},cclpaperid={1002},keywords={taskvine},}
Adaptive Task-Oriented Resource Allocation for Large Dynamic Workflows on Opportunistic Resources
Thanh Son Phung and Douglas Thain
In IEEE International Parallel and Distributed Processing Symposium (IPDPS), 2024
@inproceedings{adaptive-ipdps-2024,author={Phung, Thanh Son and Thain, Douglas},title={{Adaptive Task-Oriented Resource Allocation for Large Dynamic Workflows on Opportunistic Resources}},booktitle={{IEEE International Parallel and Distributed Processing Symposium (IPDPS)}},pages={12},year={2024},cclpaperid={994},}
2023
Maximizing Data Utility for HPC Python Workflow Execution
Thanh Son Phung, Ben Clifford, Kyle Chard, and Douglas Thain
In SC23 Workshop: High Performance Python for Science at Scale (HPPSS), 2023
@inproceedings{utility-hppss-2023,author={Phung, Thanh Son and Clifford, Ben and Chard, Kyle and Thain, Douglas},title={{Maximizing Data Utility for HPC Python Workflow Execution}},booktitle={{SC23 Workshop: High Performance Python for Science at Scale (HPPSS)}},year={2023},cclpaperid={990},}
TaskVine: Managing In-Cluster Storage for High-Throughput Data Intensive Workflows
Barry Sly-Delgado, Thanh Son Phung, Colin Thomas, David Simonetti, Andrew Hennessee, Ben Tovar, and Douglas Thain
In 18th Workshop on Workflows in Support of Large-Scale Science, 2023
@inproceedings{taskvine-works-2023,author={Sly-Delgado, Barry and Phung, Thanh Son and Thomas, Colin and Simonetti, David and Hennessee, Andrew and Tovar, Ben and Thain, Douglas},title={{TaskVine: Managing In-Cluster Storage for High-Throughput Data Intensive Workflows}},booktitle={{ 18th Workshop on Workflows in Support of Large-Scale Science}},year={2023},cclpaperid={991},keywords={taskvine},}
Poster: Minimizing Data Movement Using Distant Futures
@inproceedings{futures-sc-2023,author={Sly-Delgado, Barry and Thain, Douglas},title={{Poster: Minimizing Data Movement Using Distant Futures}},booktitle={{ACM/IEEE Supercomputing}},year={2023},cclpaperid={993},}
Poster: TaskVine: A User-Level Framework for Data Intensive Scientific Applications
@inproceedings{taskvine-cssi-2023,author={Thain, Douglas},title={{Poster: TaskVine: A User-Level Framework for Data Intensive Scientific Applications}},booktitle={{CSSI PI Meeting}},year={2023},cclpaperid={989},keywords={taskvine},}
Poster: Mixed Modality Workflows in TaskVine
David Simonetti, Ben Tovar, and Douglas Thain
In ACM High Performance Distributed Computing, 2023
@inproceedings{mixed-hpdc-2023,author={Simonetti, David and Tovar, Ben and Thain, Douglas},title={{Poster: Mixed Modality Workflows in TaskVine}},booktitle={{ACM High Performance Distributed Computing}},pages={331-332},year={2023},cclpaperid={988},keywords={taskvine},}
LANDLORD: Coordinating Dynamic Software Environments to Reduce Container Sprawl
Tim Shaffer, Thanh Son Phung, Kyle Chard, and Douglas Thain
IEEE Transactions on Parallel and Distributed Systems, 2023
@article{landlord-tpds-2023,author={Shaffer, Tim and Phung, Thanh Son and Chard, Kyle and Thain, Douglas},title={{LANDLORD: Coordinating Dynamic Software Environments to Reduce Container Sprawl}},journal={{IEEE Transactions on Parallel and Distributed Systems}},volume={34},number={5},pages={1376-1389},year={2023},note={{doi: 10.1109/TPDS.2023.3241598}},cclpaperid={985},}
2022
Poster: Robust Meta-Workflow Management with Mufasa
Ben Lyons and Douglas Thain
In IEEE International Conference on eScience, 2022
@inproceedings{mufasa-escience-2022,author={Lyons, Ben and Thain, Douglas},title={{Poster: Robust Meta-Workflow Management with Mufasa}},booktitle={{IEEE International Conference on eScience}},year={2022},cclpaperid={983},}
Dynamic Task Shaping for High Throughput Data Analysis Applications in High Energy Physics
Ben Tovar, Ben Lyons, Kelci Mohrman, Barry Sly-Delgado, Kevin Lannon, and Douglas Thain
In IEEE International Parallel and Distributed Processing Symposium, 2022
@inproceedings{topeft-ipdps-2022,author={Tovar, Ben and Lyons, Ben and Mohrman, Kelci and Sly-Delgado, Barry and Lannon, Kevin and Thain, Douglas},title={{Dynamic Task Shaping for High Throughput Data Analysis Applications in High Energy Physics}},booktitle={{IEEE International Parallel and Distributed Processing Symposium}},year={2022},note={{doi: 10.1109/IPDPS53621.2022.00041}},cclpaperid={979},keywords={workqueue},}
PONCHO: Dynamic Package Synthesis for Distributed and Serverless Python Applications
Barry Sly-Delgado, Nick Locascio, David Simonetti, Brett Wiseman, Ben Tovar, and Douglas Thain
In Workshop on High Performance Serverless Computing, 2022
@inproceedings{poncho-hips-2022,author={Sly-Delgado, Barry and Locascio, Nick and Simonetti, David and Wiseman, Brett and Tovar, Ben and Thain, Douglas},title={{PONCHO: Dynamic Package Synthesis for Distributed and Serverless Python Applications}},booktitle={{Workshop on High Performance Serverless Computing}},year={2022},note={{doi: 10.1145/3526060.3535459}},cclpaperid={982},}
Analysis Cyberinfrastructure: Challenges and Opportunities
Kevin Lannon, Paul Brenner, Michael Hildreth, Kenya Hurtado Anampa, Alan Malta, Rodrigues, Kelci Mohrman, Douglas Thain, and Ben Tovar
@inproceedings{analysis-snowmass-2022,author={Lannon, Kevin and Brenner, Paul and Hildreth, Michael and Anampa, Kenya Hurtado and Malta, Alan and Rodrigues and Mohrman, Kelci and Thain, Douglas and Tovar, Ben},title={{Analysis Cyberinfrastructure: Challenges and Opportunities}},booktitle={{Snowmass}},pages={1-14},year={2022},cclpaperid={986},}
VisDict: A Visual Dictionary in a Science Gateway
Sanda Gesing, Ewa Deelman, Rafael Ferreira Silva, Michael Hildreth, Mary Ann McDowell, Natalie K. Meyers, and Douglas Thain
@inproceedings{visdict-gateways-2022,author={Gesing, Sanda and Deelman, Ewa and da Silva, Rafael Ferreira and Hildreth, Michael and McDowell, Mary Ann and Meyers, Natalie K. and Thain, Douglas},title={{VisDict: A Visual Dictionary in a Science Gateway}},booktitle={{Gateways}},year={2022},cclpaperid={987},}
2021
Not All Tasks Are Created Equal: Adaptive Resource Allocation for Heterogeneous Tasks in Dynamic Workflows
Thanh Son Phung, Logan Ward, Kyle Chard, and Douglas Thain
In WORKS Workshop on Workflows at Supercomputing, 2021
@inproceedings{tasks-works-2021,author={Phung, Thanh Son and Ward, Logan and Chard, Kyle and Thain, Douglas},title={{Not All Tasks Are Created Equal: Adaptive Resource Allocation for Heterogeneous Tasks in Dynamic Workflows}},booktitle={{WORKS Workshop on Workflows at Supercomputing}},year={2021},cclpaperid={978},}
@inproceedings{regulating-sc-2021,author={Gao, Kevin and Shaffer, Tim and Chard, Kyle},title={{Regulating Traffic in Crowded Cache}},booktitle={{Student Poster Session at Supercomputing}},year={2021},cclpaperid={981},}
An Empirical Study of Package Dependencies and Lifetimes in Binder Python Containers
Tim Shaffer, Kyle Chard, and Douglas Thain
In IEEE International Conference on e-Science, 2021
@inproceedings{binder-escience-2021,author={Shaffer, Tim and Chard, Kyle and Thain, Douglas},title={{An Empirical Study of Package Dependencies and Lifetimes in Binder Python Containers}},booktitle={{IEEE International Conference on e-Science}},year={2021},cclpaperid={977},}
@misc{binder-dataset,author={Shaffer, Tim and Chard, Kyle and Thain, Douglas},title={{Binder Software Environments}},year={2021},note={{doi: 10.5281/zenodo.4891790}},doi={10.5281/zenodo.4891790},cclpaperid={976},}
Lightweight Function Monitors for Fine-Grained Management in Large Scale Python Applications
Tim Shaffer, Zhuozhao Li, Ben Tovar, Yadu Babuji, TJ Dasso, Zoe Surma, Kyle Chard, Ian Foster, and Douglas Thain
In IEEE International Parallel and Distributed Processing Symposium, 2021
@inproceedings{lfm-ipdps-2021,author={Shaffer, Tim and Li, Zhuozhao and Tovar, Ben and Babuji, Yadu and Dasso, TJ and Surma, Zoe and Chard, Kyle and Foster, Ian and Thain, Douglas},title={{Lightweight Function Monitors for Fine-Grained Management in Large Scale Python Applications}},booktitle={{IEEE International Parallel and Distributed Processing Symposium}},year={2021},note={{doi: 10.1109/IPDPS49936.2021.00088}},cclpaperid={968},}
Harnessing HPC resources for CMS jobs using a Virtual Private Network
Benjamin Tovar, Brian Bockelman, Michael Hildreth, Kevin Lannon, and Douglas Thain
In 25th International Conference on Computing in High Energy and Nuclear Physics (CHEP), 2021
@inproceedings{vpn-chep-2021,author={Tovar, Benjamin and Bockelman, Brian and Hildreth, Michael and Lannon, Kevin and Thain, Douglas},title={{Harnessing HPC resources for CMS jobs using a Virtual Private Network}},booktitle={{25th International Conference on Computing in High Energy and Nuclear Physics (CHEP)}},year={2021},note={{doi: 10.1051/epjconf/202125102032}},cclpaperid={973},}
Log Discovery, Log Custody, and the Web Inspired Approach for Open Distributed Systems Troubleshooting
@thesis{nkremerh-dissertation,author={Kremer-Herman, Nathaniel},title={{Log Discovery, Log Custody, and the Web Inspired Approach for Open Distributed Systems Troubleshooting}},editor={Thesis, Ph.D.},booktitle={{University of Notre Dame}},year={2021},cclpaperid={971},}
A Community Roadmap for Scientific Workflows Research and Development
R. F. al.
In IEEE Workshop on Workflows in Support of Large-Scale Science (WORKS), 2021
@inproceedings{roadmap-works-2021,author={da Silva et al., R. F.},title={{A Community Roadmap for Scientific Workflows Research and Development}},booktitle={{IEEE Workshop on Workflows in Support of Large-Scale Science (WORKS)}},pages={81-90},year={2021},note={{doi: 10.1109/WORKS54523.2021.00016}},cclpaperid={984},}
2020
Autoscaling High Throughput Workloads on Container Orchestrators
Chao Zheng, Nathaniel Kremer-Herman, Tim Shaffer, and Douglas Thain
@inproceedings{autoscaling-cluster-2020,author={Zheng, Chao and Kremer-Herman, Nathaniel and Shaffer, Tim and Thain, Douglas},title={{Autoscaling High Throughput Workloads on Container Orchestrators }},booktitle={{IEEE Conference on Cluster Computing}},pages={1-10},year={2020},note={{doi: 10.1109/CLUSTER49012.2020.00024}},cclpaperid={967},}
Log Discovery for Troubleshooting Open Distributed Systems with TLQ
Nathaniel Kremer-Herman and Douglas Thain
In Practice and Experience of Advanced Research Computing (PEARC), 2020
@inproceedings{tlq-pearc-2020,author={Kremer-Herman, Nathaniel and Thain, Douglas},title={{Log Discovery for Troubleshooting Open Distributed Systems with TLQ}},booktitle={{Practice and Experience of Advanced Research Computing (PEARC)}},year={2020},note={{doi: 10.1145/3311790.3396633}},cclpaperid={966},}
Solving the Container Explosion Problem for Distributed High Throughput Computing
Tim Shaffer, Nicholas Hazekamp, Jakob Blomer, and Douglas Thain
In International Parallel and Distributed Processing Symposium, 2020
@inproceedings{explosion-ipdps-2020,author={Shaffer, Tim and Hazekamp, Nicholas and Blomer, Jakob and Thain, Douglas},title={{Solving the Container Explosion Problem for Distributed High Throughput Computing}},booktitle={{International Parallel and Distributed Processing Symposium}},year={2020},note={{doi: 10.1109/IPDPS47924.2020.00048}},cclpaperid={964},}
2019
Methods Enabling Portability of Scientific Workflows
@thesis{czheng-dissertation,author={Zheng, Chao},title={{The Challenges of Scaling Up High-Throughput Workflow with Container Technologies}},editor={Thesis, Ph.D.},booktitle={{University of Notre Dame}},year={2019},cclpaperid={963},}
Dynamic Sizing of Continuously Divisible Jobs for Heterogeneous Resources
Nick Hazekamp, Ben Tovar, and Douglas Thain
In IEEE International Conference on e-Science, 2019
@inproceedings{sizing-escience-2019,author={Hazekamp, Nick and Tovar, Ben and Thain, Douglas},title={{Dynamic Sizing of Continuously Divisible Jobs for Heterogeneous Resources}},booktitle={{IEEE International Conference on e-Science}},year={2019},note={{doi: 10.1109/eScience.2019.00026}},cclpaperid={962},}
Flexible Partitioning of Scientific Workflows Using the JX Workflow Language
Tim Shaffer, Nathaniel Kremer-Herman, and Douglas Thain
In Practice and Experience in Advanced Research Computing (PEARC), 2019
@inproceedings{jx-pearc19,author={Shaffer, Tim and Kremer-Herman, Nathaniel and Thain, Douglas},title={{Flexible Partitioning of Scientific Workflows Using the JX Workflow Language}},booktitle={{Practice and Experience in Advanced Research Computing (PEARC)}},year={2019},note={{doi: 10.1145/3332186.3338100}},cclpaperid={961},keywords={makeflow},}
2018
A Lightweight Model for Right-Sizing Master-Worker Applications
Nathaniel Kremer-Herman, Benjamin Tovar, and Douglas Thain
@inproceedings{capacity-sc18,author={Kremer-Herman, Nathaniel and Tovar, Benjamin and Thain, Douglas},title={{A Lightweight Model for Right-Sizing Master-Worker Applications}},booktitle={{ACM/IEEE Supercomputing (SC)}},year={2018},note={{doi: 10.1109/SC.2018.00042}},cclpaperid={955},}
Reduction of Workflow Resource Consumption Using a Density-based Clustering Model
Qimin Zhang, Ben Tovar, Nate Kremer-Herman, and Douglas Thain
@inproceedings{clustering-works-2018,author={Zhang, Qimin and Tovar, Ben and Kremer-Herman, Nate and Thain, Douglas},title={{Reduction of Workflow Resource Consumption Using a Density-based Clustering Model}},booktitle={{WORKS Workshop at Supercomputing}},year={2018},cclpaperid={956},}
An Algebra for Robust Workflow Transformations
Nicholas Hazekamp and Douglas Thain
In IEEE International Conference on e-Science, 2018
@inproceedings{transformation-escience-2018,author={Hazekamp, Nicholas and Thain, Douglas},title={{An Algebra for Robust Workflow Transformations}},booktitle={{IEEE International Conference on e-Science}},pages={12},year={2018},note={{doi: 10.1109/eScience.2018.00031}},cclpaperid={953},}
Poster: A First Look at the JX Workflow Language
Tim Shaffer, Kyle M.D. Sweeney, Nathaniel Kremer-Herman, and Douglas Thain
In IEEE International Conference on e-Science, 2018
@inproceedings{jx-escience18,author={Shaffer, Tim and Sweeney, Kyle M.D. and Kremer-Herman, Nathaniel and Thain, Douglas},title={{Poster: A First Look at the JX Workflow Language}},booktitle={{IEEE International Conference on e-Science}},year={2018},note={{doi: 10.1109/eScience.2018.00094}},cclpaperid={954},}
Wharf: Sharing Docker Images in a Distributed File System
Chao Zheng, Lukas Rupprecht, Vasily Tarasov, Douglas Thain, Mohamed Mohamed, Dimitrios Skourtis, Amit S. Warke, and Dean Hildrebarnd
@inproceedings{wharf-socc-2018,author={Zheng, Chao and Rupprecht, Lukas and Tarasov, Vasily and Thain, Douglas and Mohamed, Mohamed and Skourtis, Dimitrios and Warke, Amit S. and Hildrebarnd, Dean},title={{Wharf: Sharing Docker Images in a Distributed File System}},booktitle={{ACM Symposium on Cloud Computing}},pages={12},year={2018},note={{doi: 10.1145/3267809.3267836}},cclpaperid={957},}
VC3: A Virtual Cluster Service for Community Computation
Lincoln Bryant, Jeremy Van, Benedikt Riedel, Robert Gardner, Jose Caballero, John Hover, Ben Tovar, and Douglas Thain
In PEARC: Practice and Experience in Advanced Research Computing, 2018
@inproceedings{vc3-pearc-2018,author={Bryant, Lincoln and Van, Jeremy and Riedel, Benedikt and Gardner, Robert and Caballero, Jose and Hover, John and Tovar, Ben and Thain, Douglas},title={{VC3: A Virtual Cluster Service for Community Computation}},booktitle={{PEARC: Practice and Experience in Advanced Research Computing}},pages={1-8},year={2018},note={{doi: 10.1145/3219104.3219125}},cclpaperid={949},}
@article{repro-survey,author={Ivie, Peter and Thain, Douglas},title={{Reproducibility in Scientific Computing}},journal={{ACM Computing Surveys}},volume={51},number={3},year={2018},note={{doi: 10.1145/3186266}},cclpaperid={952},}
Early Experience Using Amazon Batch for Scientific Workflows
@inproceedings{batch-sciencecloud-2018,author={Sweeney, Kyle and Thain, Douglas},title={{Early Experience Using Amazon Batch for Scientific Workflows}},booktitle={{ScienceCloud Workshop at HPDC }},year={2018},note={{doi: 10.1145/3217880.3217885}},cclpaperid={950},}
Efficient Integration of Containers into Scientific Workflows
@inproceedings{containers-sciencecloud-2018,author={Sweeney, Kyle and Thain, Douglas},title={{Efficient Integration of Containers into Scientific Workflows}},booktitle={{Science Cloud Workshop at HPDC}},year={2018},note={{doi: 10.1145/3217880.3217887}},cclpaperid={951},}
Automatic Dependency Management for Scientific Applications on Clusters
Benjamin Tovar, Nicholas Hazekamp, Nathaniel Kremer-Herman, and Douglas Thain
In IEEE International Conference on Cloud Engineering (IC2E) , 2018
@inproceedings{tovar-ic2e-2018,author={Tovar, Benjamin and Hazekamp, Nicholas and Kremer-Herman, Nathaniel and Thain, Douglas},title={{Automatic Dependency Management for Scientific Applications on Clusters}},booktitle={{IEEE International Conference on Cloud Engineering (IC2E) }},year={2018},note={{doi: 10.1109/IC2E.2018.00026}},cclpaperid={945},}
MAKER as a Service: Moving HPC applications to Jetstream Cloud
Nicholas Hazekamp, Upendra Kumar Devisetty, Nirav Merchant, and Douglas Thain
In IEEE International Conference on Cloud Engineering, 2018
@inproceedings{maker-service-ic2e2018,author={Hazekamp, Nicholas and Devisetty, Upendra Kumar and Merchant, Nirav and Thain, Douglas},title={{MAKER as a Service: Moving HPC applications to Jetstream Cloud}},booktitle={{IEEE International Conference on Cloud Engineering}},pages={6},year={2018},note={{doi: 10.1109/IC2E.2018.00029}},cclpaperid={946},}
A Workflow Management System to Facilitate Reproducibility of Scientific Computing Applications
@thesis{ivie-thesis,author={Ivie, Peter},title={{A Workflow Management System to Facilitate Reproducibility of Scientific Computing Applications}},editor={Thesis, Ph.D.},booktitle={{University of Notre Dame}},year={2018},cclpaperid={960},}
@inproceedings{shadho-wacv-2018,author={Kinnison, Jeffrey and Kremer-Herman, Nathaniel and Thain, Douglas and Scheirer, Walter},title={{SHADHO: Massively Scalable Hardware-Aware Distributed Hyperparameter Optimization}},booktitle={{IEEE Winter Conference on Applications of Computer Vision}},pages={1-10},year={2018},note={{doi: 10.1109/WACV.2018.00086}},cclpaperid={947},}
A Job Sizing Strategy for High-Throughput Scientific Workflows
Benjamin Tovar, Rafael Ferreira Silva, Gideon Juve, Ewa Deelman, William Allcock, Douglas Thain, and Miron Livny
IEEE Transactions on Parallel and Distributed Systems, 2018
@article{tovar-tpds-2017,author={Tovar, Benjamin and da Silva, Rafael Ferreira and Juve, Gideon and Deelman, Ewa and Allcock, William and Thain, Douglas and Livny, Miron},title={{A Job Sizing Strategy for High-Throughput Scientific Workflows}},journal={{IEEE Transactions on Parallel and Distributed Systems}},volume={29},number={2},pages={240-253},year={2018},note={{doi: 10.1109/TPDS.2017.2762310}},cclpaperid={941},}
Combining Static and Dynamic Storage Management for Data Intensive Scientific Workflows
Nicholas Hazekamp, Nathaniel Kremer-Herman, Benjamin Tovar, Haiyan Meng, Olivia Choudhury, Scott Emrich, and Douglas Thain
IEEE Transactions on Parallel and Distributed Systems, 2018
@article{mf-storage-tpds17,author={Hazekamp, Nicholas and Kremer-Herman, Nathaniel and Tovar, Benjamin and Meng, Haiyan and Choudhury, Olivia and Emrich, Scott and Thain, Douglas},title={{Combining Static and Dynamic Storage Management for Data Intensive Scientific Workflows}},journal={{IEEE Transactions on Parallel and Distributed Systems}},volume={29},number={2},pages={338-350},year={2018},note={{doi: 10.1109/TPDS.2017.2764897}},cclpaperid={942},}
GPU acceleration of Eulerian–Lagrangian particle-laden turbulent flow simulations
@article{gpu-ijmf,author={Sweet, James and Richter, David and Thain, Douglas},title={{GPU acceleration of Eulerian–Lagrangian particle-laden turbulent flow simulations}},journal={{International Journal of Multiphase Flow}},volume={99},pages={437-435},year={2018},note={{doi: 10.1016/j.ijmultiphaseflow.2017.11.010}},cclpaperid={944},}
2017
Poster: Wharf: Sharing Docker Images across Hosts from a Distributed Filesystem
Chao Zheng, Lukas Rupprecht, Vasily Tarasov, Mohamed Mohamed, Dimitrios Skourtis, Amit S. Warke, and Dean Hildebrand Douglas Thain
In IEEE International Conference for High Performance Computing, Networking, Storage and Analysis, 2017
@inproceedings{wharf-sc17,author={Zheng, Chao and Rupprecht, Lukas and Tarasov, Vasily and Mohamed, Mohamed and Skourtis, Dimitrios and Warke, Amit S. and Thain, Dean Hildebrand Douglas},title={{Poster: Wharf: Sharing Docker Images across Hosts from a Distributed Filesystem}},booktitle={{IEEE International Conference for High Performance Computing, Networking, Storage and Analysis}},year={2017},cclpaperid={943},}
Taming Metadata Storms in Parallel Filesystems with MetaFS
Tim Shaffer and Douglas Thain
In Workshop on Parallel Data Storage and Data Intensive Scalable Computing Systems, 2017
@inproceedings{metafs-pdsw-2017,author={Shaffer, Tim and Thain, Douglas},title={{Taming Metadata Storms in Parallel Filesystems with MetaFS}},booktitle={{Workshop on Parallel Data Storage and Data Intensive Scalable Computing Systems}},pages={25-30},year={2017},note={{doi: 10.1145/3149393.3149401}},cclpaperid={948},}
Enabling Implementation and Optimization of Scientific Algorithms via Graphics Processing Units
@thesis{sweet-thesis,author={Sweet, James},title={{Enabling Implementation and Optimization of Scientific Algorithms via Graphics Processing Units}},editor={Thesis, Ph.D.},booktitle={{University of Notre Dame}},year={2017},cclpaperid={959},}
Facilitating the Reproducibility of Scientific Workflows with Execution Environment Specifications
Haiyan Meng and Douglas Thain
In The 17th International Conference on Computational Science (ICCS), 2017
@inproceedings{PAPER937,author={Meng, Haiyan and Thain, Douglas},title={{Facilitating the Reproducibility of Scientific Workflows with Execution Environment Specifications}},booktitle={{The 17th International Conference on Computational Science (ICCS)}},year={2017},note={{doi: 10.1016/j.procs.2017.05.116}},cclpaperid={937},}
Towards Scalable and Dynamic Social Sensing Using A Distributed Computing Framework
Daniel (Yue) Zhang, Charles (Chao) Zheng, Dong Wang, Doug Thain, Chao Huang, Xin Mu, and Greg Madey
In The 37th IEEE International Conference on Distributed Computing Systems (ICDCS 2017), 2017
@inproceedings{social-icdcs-2017,author={Zhang, Daniel (Yue) and Zheng, Charles (Chao) and Wang, Dong and Thain, Doug and Huang, Chao and Mu, Xin and Madey, Greg},title={{Towards Scalable and Dynamic Social Sensing Using A Distributed Computing Framework}},booktitle={{The 37th IEEE International Conference on Distributed Computing Systems (ICDCS 2017)}},year={2017},note={{doi: 10.1109/ICDCS.2017.196}},cclpaperid={938},}
Deploying High Throughput Scientific Workflows on Container Schedulers with Makeflow and Mesos
Charles (Chao) Zheng, Ben Tovar, and Douglas Thain
In 17th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (CCGrid 2017), 2017
@inproceedings{makeflow-mesos-ccgrid17.pdf,author={Zheng, Charles (Chao) and Tovar, Ben and Thain, Douglas},title={{Deploying High Throughput Scientific Workflows on Container Schedulers with Makeflow and Mesos}},booktitle={{17th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (CCGrid 2017)}},year={2017},note={{doi: 10.1109/CCGRID.2017.9}},cclpaperid={939},}
Improving the Reproducibility of Scientific Applications with Execution Environment Specifications
@thesis{meng-thesis,author={Meng, Haiyan},title={{Improving the Reproducibility of Scientific Applications with Execution Environment Specifications}},editor={Thesis, Ph.D.},booktitle={{University of Notre Dame}},year={2017},cclpaperid={940},}
Designing Self-Tuning Split-Map-Merge Applications for High Cost-Efficiency in the Cloud
@article{tuning-tcc-2015,author={Rajan, Dinesh and Thain, Douglas},title={{Designing Self-Tuning Split-Map-Merge Applications for High Cost-Efficiency in the Cloud}},journal={{IEEE Transactions on Cloud Computing}},volume={5},number={2},pages={303-316},year={2017},note={{doi: 10.1109/TCC.2015.2415780}},cclpaperid={909},}
2016
PRUNE: A Preserving Run Environment for Reproducible Computing
@inproceedings{prune-escience-2016,author={Ivie, Peter and Thain, Douglas},title={{PRUNE: A Preserving Run Environment for Reproducible Computing}},booktitle={{IEEE Conference on e-Science}},year={2016},note={{doi: 10.1109/eScience.2016.7870886}},cclpaperid={930},}
Conducting Reproducible Research with Umbrella: Tracking, Creating, and Preserving Execution Environments
Haiyan Meng, Douglas Thain, Alexander Vyushkov, Matthias Wolf, and Anna Woodard
@inproceedings{umbrella-escience-2016,author={Meng, Haiyan and Thain, Douglas and Vyushkov, Alexander and Wolf, Matthias and Woodard, Anna},title={{Conducting Reproducible Research with Umbrella: Tracking, Creating, and Preserving Execution Environments}},booktitle={{IEEE Conference on e-Science}},year={2016},note={{doi: 10.1109/eScience.2016.7870889}},cclpaperid={931},}
A First Look at Reproducibility and Non-Determinism in CMS Codes and ROOT Data
Peter Ivie, Charles (Chao) Zheng, and Douglas Thain
@techreport{repro-tr-2016,author={Ivie, Peter and Zheng, Charles (Chao) and Thain, Douglas},title={{A First Look at Reproducibility and Non-Determinism in CMS Codes and ROOT Data}},institution={{University of Notre Dame, Computer Science and Engineering Department}},number={2016-01},year={2016},cclpaperid={933},}
Opportunistic Computing with Lobster: Lessons Learned from Scaling up to 25K Non-Dedicated Cores
Matthias Wolf, Anna Woodard, Wenzhao Li, Kenyi Hurtado Anampa, Yannakopoulos, Benjamin Tovar, Patrick Donnelly, Paul Brenner, Kevin Lannon, Mike Hildreth, and Douglas Thain
In International Conference on Computing in High Energy Physics, 2016
@inproceedings{oppo-chep2016,author={Wolf, Matthias and Woodard, Anna and Li, Wenzhao and Anampa, Kenyi Hurtado and Yannakopoulos and Tovar, Benjamin and Donnelly, Patrick and Brenner, Paul and Lannon, Kevin and Hildreth, Mike and Thain, Douglas},title={{Opportunistic Computing with Lobster: Lessons Learned from Scaling up to 25K Non-Dedicated Cores}},booktitle={{International Conference on Computing in High Energy Physics}},year={2016},note={{doi: 10.1088/1742-6596/898/5/052036}},cclpaperid={934},}
Scaling Up a CMS Tier-3 Site with Campus Resources and a 100Gb/s Network Connection: What Could Go Wrong?
Matthias Wolf, Anna Woodard, Wenzhao Li, Kenyi Hurtado Anampa, Benjamin Tovar, Paul Brenner, Kevin Lannon, Mike Hildreth, and Douglas Thain
In International Conference on Computing in High Energy Physics, 2016
@inproceedings{scaling-chep2016,author={Wolf, Matthias and Woodard, Anna and Li, Wenzhao and Anampa, Kenyi Hurtado and Tovar, Benjamin and Brenner, Paul and Lannon, Kevin and Hildreth, Mike and Thain, Douglas},title={{Scaling Up a CMS Tier-3 Site with Campus Resources and a 100Gb/s Network Connection: What Could Go Wrong?}},booktitle={{International Conference on Computing in High Energy Physics}},year={2016},note={{doi: 10.1088/1742-6596/898/8/082041}},cclpaperid={935},}
An Analysis of Reproducibility and Non-Determinism in HEP Software and ROOT Data
Peter Ivie, Charles (Chao) Zheng, and Douglas Thain
In International Conference on Computing in High Energy and Nuclear Physics, 2016
@inproceedings{PAPER936,author={Ivie, Peter and Zheng, Charles (Chao) and Thain, Douglas},title={{An Analysis of Reproducibility and Non-Determinism in HEP Software and ROOT Data}},booktitle={{International Conference on Computing in High Energy and Nuclear Physics}},year={2016},note={{doi: 10.1088/1742-6596/898/10/102007}},cclpaperid={936},}
DistIA: A Cost-Effective Dynamic Impact Analysis for Distributed Prorgams
Haipeng Cai and Douglas thain
In IEEE/ACM International Conference on Automated Software Engineering, 2016
@inproceedings{distia-ase-2016,author={Cai, Haipeng and thain, Douglas},title={{DistIA: A Cost-Effective Dynamic Impact Analysis for Distributed Prorgams}},booktitle={{IEEE/ACM International Conference on Automated Software Engineering}},year={2016},note={{doi: 10.1145/2970276.2970352}},doi={10.1145/2970276.2970352},cclpaperid={932},}
DIAPRO: Unifying Dynamic Impact Analyses for Improved and Variable Cost-Effectiveness
Haipeng Cai, Raul Santelices, and Douglas Thain
ACM Transactions on Software Engineering and Methodology, 2016
@article{diapro-tosem-2016,author={Cai, Haipeng and Santelices, Raul and Thain, Douglas},title={{DIAPRO: Unifying Dynamic Impact Analyses for Improved and Variable Cost-Effectiveness}},journal={{ACM Transactions on Software Engineering and Methodology}},volume={25},number={2},year={2016},note={{doi: 10.1145/2894751}},cclpaperid={927},}
Balancing push and pull in Confuga, an active storage cluster file system for scientific workflows
Patrick Donnelly and Douglas Thain
Concurrency and Computation: Practice and Experience, 2016
@article{ccpe-confuga,author={Donnelly, Patrick and Thain, Douglas},title={{Balancing push and pull in Confuga, an active storage cluster file system for scientific workflows}},journal={{Concurrency and Computation: Practice and Experience}},volume={29},number={4},year={2016},note={{doi: 10.1002/cpe.3834}},cclpaperid={929},}
Data Locality Techniques in an Active Cluster Filesystem for Scientific Workflows
@thesis{pdonnelly-thesis,author={Donnelly, Patrick},title={{Data Locality Techniques in an Active Cluster Filesystem for Scientific Workflows}},editor={Thesis, Ph.D.},booktitle={{University of Notre Dame}},year={2016},cclpaperid={928},}
2015
A Case Study in Preserving a High Energy Physics Application with Parrot
Haiyan Meng, Matthias Wolf, Peter Ivie, Anna Woodard, Michael Hildreth, and Douglas Thain
In Journal of Physics: Conference Series (CHEP 2015), 2015
@inproceedings{tauroast-chep-2015,author={Meng, Haiyan and Wolf, Matthias and Ivie, Peter and Woodard, Anna and Hildreth, Michael and Thain, Douglas},title={{A Case Study in Preserving a High Energy Physics Application with Parrot}},booktitle={{Journal of Physics: Conference Series (CHEP 2015)}},year={2015},note={{doi: 10.1088/1742-6596/664/3/032022}},cclpaperid={925},}
The Evolution of Global Scale Filesystems for Scientific Software Distribution
Jakob Blomer, Predrag Buncic, Rene Meusel, Gerardo Ganis, Igor Sfiligoi, and Douglas Thain
IEEE/AIP Computing in Science and Engineering, 2015
@article{globalfs-cise-2015,author={Blomer, Jakob and Buncic, Predrag and Meusel, Rene and Ganis, Gerardo and Sfiligoi, Igor and Thain, Douglas},title={{The Evolution of Global Scale Filesystems for Scientific Software Distribution}},journal={{IEEE/AIP Computing in Science and Engineering}},volume={17},number={6},pages={61-71},year={2015},note={{doi: 10.1109/MCSE.2015.111}},cclpaperid={926},}
Techniques for Preserving Scientific Software Executions: Preserve the Mess or Encourage Cleanliness?
Douglas Thain, Peter Ivie, and Haiyan Meng
In 12th International Conference on Digital Preservation (iPres), 2015
@inproceedings{techniques-ipres-2015,author={Thain, Douglas and Ivie, Peter and Meng, Haiyan},title={{Techniques for Preserving Scientific Software Executions: Preserve the Mess or Encourage Cleanliness?}},booktitle={{12th International Conference on Digital Preservation (iPres)}},year={2015},note={{doi: 10.7274/R0CZ353M}},cclpaperid={921},}
DAGViz: A DAG Visualization Tool for Analzying Task Parallel Program Traces
An Huynh, Douglas Thain, Miquel Pericas, and Kenjiro Taura
In Workshop on Visual Performance Analysis (VPA) at ACM Supercomputing (SC), 2015
@inproceedings{dagviz-vpa-2015,author={Huynh, An and Thain, Douglas and Pericas, Miquel and Taura, Kenjiro},title={{DAGViz: A DAG Visualization Tool for Analzying Task Parallel Program Traces}},booktitle={{Workshop on Visual Performance Analysis (VPA) at ACM Supercomputing (SC)}},year={2015},cclpaperid={924},}
Scaling Data Intensive Physics Applications to 10k Cores on Non-Dedicated Clusters with Lobster
Anna Woodard, Matthias Wolf, Charles Mueller, Nil Valls, Ben Tovar, Patrick Donnelly, Peter Ivie, Kenyi Hurtado Anampa, Paul Brenner, Douglas Thain, Kevin Lannon, and Michael Hildreth
@inproceedings{lobster-cluster-2015,author={Woodard, Anna and Wolf, Matthias and Mueller, Charles and Valls, Nil and Tovar, Ben and Donnelly, Patrick and Ivie, Peter and Anampa, Kenyi Hurtado and Brenner, Paul and Thain, Douglas and Lannon, Kevin and Hildreth, Michael},title={{Scaling Data Intensive Physics Applications to 10k Cores on Non-Dedicated Clusters with Lobster}},booktitle={{IEEE Conference on Cluster Computing}},year={2015},cclpaperid={915},}
Adapting Collaborative Software Development Techniques to Structural Engineering
Peter Sempolinski, Daniel Wei, Ahsan Kareem, and Douglas Thain
IEEE/AIP Computing in Science and Engineering, 2015
@article{collab-cise-2015,author={Sempolinski, Peter and Wei, Daniel and Kareem, Ahsan and Thain, Douglas},title={{Adapting Collaborative Software Development Techniques to Structural Engineering}},journal={{IEEE/AIP Computing in Science and Engineering}},volume={17},number={51},pages={27-34},year={2015},note={{doi: 10.1109/MCSE.2015.88}},doi={10.1109/MCSE.2015.88},cclpaperid={916},}
Balancing Thread-level and Task-level Parallelism for Data-Intensive Workloads on Clusters and Clouds
Olivia Choudhury, Dinesh Rajan, Nicholas Hazekamp, Sandra Gesing, Douglas Thain, and Scott Emrich
@inproceedings{balancing-cluster-2015,author={Choudhury, Olivia and Rajan, Dinesh and Hazekamp, Nicholas and Gesing, Sandra and Thain, Douglas and Emrich, Scott},title={{Balancing Thread-level and Task-level Parallelism for Data-Intensive Workloads on Clusters and Clouds}},booktitle={{IEEE Conference on Cluster Computing}},year={2015},cclpaperid={919},}
Practical Resource Monitoring for Robust High Throughput Computing
Gideon Juve, Benjamin Tovar, Rafael Ferreira Silva, Dariusz Krol, Douglas Thain, Ewa Deelman, William Allcock, and Miron Livny
In Workshop on Monitoring and Analysis for High Performance Computing Systems Plus Applications at IEEE Cluster Computing, 2015
@inproceedings{monitoring-hpcmaspa-2015,author={Juve, Gideon and Tovar, Benjamin and da Silva, Rafael Ferreira and Krol, Dariusz and Thain, Douglas and Deelman, Ewa and Allcock, William and Livny, Miron},title={{Practical Resource Monitoring for Robust High Throughput Computing}},booktitle={{Workshop on Monitoring and Analysis for High Performance Computing Systems Plus Applications at IEEE Cluster Computing}},year={2015},cclpaperid={922},}
Lessons Learned from Crowdsourcing Complex Engineering Tasks
Matthew Staffelbach, Peter Sempolinski, Tracy Kijewski-Correa, Douglas Thain, Daniel Wei, Ahsan Kareem, and Gregory Madey
@article{crowdsourcing-plosone-2015,author={Staffelbach, Matthew and Sempolinski, Peter and Kijewski-Correa, Tracy and Thain, Douglas and Wei, Daniel and Kareem, Ahsan and Madey, Gregory},title={{Lessons Learned from Crowdsourcing Complex Engineering Tasks}},journal={{PLOS One}},volume={10},number={9},year={2015},note={{doi: 10.1371/journal.pone.0134978}},doi={10.1371/journal.pone.0134978},cclpaperid={923},}
Scaling Up Bioinformatics Workflows with Dynamic Job Expansion
Nicholas Hazekamp, Joseph Sarro, Olivia Choudhury, Sandra Gesing, Scott Emrich, and Douglas Thain
In IEEE International Conference on e-Science, 2015
@inproceedings{scaling-escience-2015,author={Hazekamp, Nicholas and Sarro, Joseph and Choudhury, Olivia and Gesing, Sandra and Emrich, Scott and Thain, Douglas},title={{Scaling Up Bioinformatics Workflows with Dynamic Job Expansion}},booktitle={{IEEE International Conference on e-Science}},year={2015},cclpaperid={920},}
An Invariant Framework for Conducting Reproducible Computational Science
Haiyan Meng, Rupa Kommineni, Quan Pham, Robert Gardner, Tanu Malik, and Douglas Thain
@article{invariant-jocs-2015,author={Meng, Haiyan and Kommineni, Rupa and Pham, Quan and Gardner, Robert and Malik, Tanu and Thain, Douglas},title={{An Invariant Framework for Conducting Reproducible Computational Science}},journal={{Journal of Computational Science}},volume={9},pages={137-142},year={2015},note={{doi: 10.1016/j.jocs.2015.04.012}},cclpaperid={914},}
Integrating Containers into Workflows: A Case Study Using Makeflow, Work Queue, and Docker
Charles (Chao) Zheng and Douglas Thain
In Workshop on Virtualization Technologies in Distributed Computing (VTDC), 2015
@inproceedings{wq-docker-vtdc15,author={Zheng, Charles (Chao) and Thain, Douglas},title={{Integrating Containers into Workflows: A Case Study Using Makeflow, Work Queue, and Docker}},booktitle={{Workshop on Virtualization Technologies in Distributed Computing (VTDC)}},year={2015},note={{doi: 10.1145/2755979.2755984}},cclpaperid={910},}
Umbrella: A Portable Environment Creator for Reproducible Computing on Clusters, Clouds, and Grids
Haiyan Meng and Douglas Thain
In Workshop on Virtualization Technologies in Distributed Computing (VTDC) at HPDC, 2015
@inproceedings{umbrella-vtdc15,author={Meng, Haiyan and Thain, Douglas},title={{Umbrella: A Portable Environment Creator for Reproducible Computing on Clusters, Clouds, and Grids}},booktitle={{Workshop on Virtualization Technologies in Distributed Computing (VTDC) at HPDC}},year={2015},note={{doi: 10.1145/2755979.2755982}},cclpaperid={912},}
Confuga: Scalable Data Intensive Computing for POSIX Workflows
Patrick Donnelly, Nicholas Hazekamp, and Douglas Thain
In IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing, 2015
@inproceedings{confuga-ccgrid2015,author={Donnelly, Patrick and Hazekamp, Nicholas and Thain, Douglas},title={{Confuga: Scalable Data Intensive Computing for POSIX Workflows}},booktitle={{IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing}},pages={392-401},year={2015},note={{doi: 10.1109/CCGrid.2015.95}},cclpaperid={908},}
Principles for the Design and Operation of Elastic Scientific Applications on Distributed Systems
@thesis{dpandiar-dissertation,author={Rajan, Dinesh},title={{Principles for the Design and Operation of Elastic Scientific Applications on Distributed Systems}},editor={Thesis, Ph.D.},booktitle={{University of Notre Dame}},year={2015},cclpaperid={913},}
Software in Science: a Report of Outcomes of the 2014 National Science Foundation Software Infrastructure for Sustained Innovation (SI2) Meeting
@techreport{software-nsf-2014,author={Plale, Beth and Jones, Matt and Thain, Douglas},title={{Software in Science: a Report of Outcomes of the 2014 National Science Foundation Software Infrastructure for Sustained Innovation (SI2) Meeting}},year={2015},cclpaperid={911},}
A Case Study in Preserving a High Energy Physics Application
Haiyan Meng, Matthias Wolf, Anna Woodard, Peter Ivie, Michael Hildreth, and Douglas Thain
@inproceedings{casestudy-poster-chep-2015,author={Meng, Haiyan and Wolf, Matthias and Woodard, Anna and Ivie, Peter and Hildreth, Michael and Thain, Douglas},title={{A Case Study in Preserving a High Energy Physics Application}},booktitle={{Computing in High Energy Physics}},year={2015},cclpaperid={917},}
Exploiting Volatile Opportunistic Computing Resources with Lobster
Anna Woodard, Matthias Wolf, Charles Nicholas Mueller, Ben Tovar, Patrick Donnelly, Kenyi Hurtado Anampa, Paul Brenner, Kevin Lannon, and Michael Hildreth
@inproceedings{lobster-poster-chep-2015,author={Woodard, Anna and Wolf, Matthias and Mueller, Charles Nicholas and Tovar, Ben and Donnelly, Patrick and Anampa, Kenyi Hurtado and Brenner, Paul and Lannon, Kevin and Hildreth, Michael},title={{Exploiting Volatile Opportunistic Computing Resources with Lobster}},booktitle={{Computing in High Energy Physics}},year={2015},cclpaperid={918},}
2014
AWE-WQ: Fast-Forwarding Molecular Dynamics using the Accelerated Weighted Ensemble
Badi Abdul-Wahid, Haoyun Feng, Dinesh Rajan, Ronan Costaouec, Eric Darve, Douglas Thain, and Jesus A. Izaguirre
Journal of Chemical Information and Modeling, 2014
@article{awe-jcim-2014,author={Abdul-Wahid, Badi and Feng, Haoyun and Rajan, Dinesh and Costaouec, Ronan and Darve, Eric and Thain, Douglas and Izaguirre, Jesus A.},title={{AWE-WQ: Fast-Forwarding Molecular Dynamics using the Accelerated Weighted Ensemble}},journal={{Journal of Chemical Information and Modeling}},volume={54},number={10},pages={3033-3043},year={2014},note={{doi: 10.1021/ci500321g}},cclpaperid={907},}
Scaling Up Genome Annotation with MAKER and Work Queue
Andrew Thrasher, Zachary Musgrave, Brian Kachmark, Douglas Thain, and Scott Emrich
International Journal of Bioinformatics Research and Applications, 2014
@article{maker-wq-ijbra,author={Thrasher, Andrew and Musgrave, Zachary and Kachmark, Brian and Thain, Douglas and Emrich, Scott},title={{Scaling Up Genome Annotation with MAKER and Work Queue}},journal={{International Journal of Bioinformatics Research and Applications}},volume={10},number={4-5},pages={447-460},year={2014},note={{doi: 10.1504/IJBRA.2014.062994}},cclpaperid={904},}
DeltaDB: A Scalable Database Design for Time-Varying Schema-Free Data
Peter Ivie and Douglas Thain
In IEEE International Congress on Big Data (BigData 2014), 2014
@inproceedings{pivie-deltadb-2014,author={Ivie, Peter and Thain, Douglas},title={{DeltaDB: A Scalable Database Design for Time-Varying Schema-Free Data}},booktitle={{IEEE International Congress on Big Data (BigData 2014)}},pages={104-111},year={2014},note={{doi: 10.1109/BigData.Congress.2014.24 }},cclpaperid={906},}
Opportunistic High Energy Physics Computing in User Space with Parrot
Dillon Skeehan, Paul Brenner, Ben Tovar, Douglas Thain, Nil Valls, Anna Woodard, Matthew Wolf, Tessa Pearson, Sean Lynch, and Kevin Lannon
In IEEE Symposium on Cluster, Cloud, and Grid Computing (CCGrid), 2014
@inproceedings{oppo-ccgrid14,author={Skeehan, Dillon and Brenner, Paul and Tovar, Ben and Thain, Douglas and Valls, Nil and Woodard, Anna and Wolf, Matthew and Pearson, Tessa and Lynch, Sean and Lannon, Kevin},title={{Opportunistic High Energy Physics Computing in User Space with Parrot}},booktitle={{IEEE Symposium on Cluster, Cloud, and Grid Computing (CCGrid)}},pages={170-175},year={2014},note={{doi: 10.1109/CCGrid.2014.34}},cclpaperid={902},}
Accelerating Comparative Genomics Workflows in a Distributed Environment with Optimized Data Partitioning
Olivia Choudhury, Nicholas L. Hazekamp, Douglas Thain, and Scott Emrich
In C4BIO Workshop at IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (CCGRID), 2014
@inproceedings{bio-partition-c4bio-grid14,author={Choudhury, Olivia and Hazekamp, Nicholas L. and Thain, Douglas and Emrich, Scott},title={{Accelerating Comparative Genomics Workflows in a Distributed Environment with Optimized Data Partitioning}},booktitle={{C4BIO Workshop at IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (CCGRID)}},year={2014},cclpaperid={903},}
Lessons Learned from an Experiment in Crowdsourcing Complex Citizen Engineering Tasks with Amazon Mechanical Turk
Matthew Staffelbach, Peter Sempolinski, David Hachen, Ahsan Kareem, Tracy Kijewski-Correa, Douglas Thain, Daniel Wei, and Gregory Madey
@inproceedings{abstract-mechturk,author={Staffelbach, Matthew and Sempolinski, Peter and Hachen, David and Kareem, Ahsan and Kijewski-Correa, Tracy and Thain, Douglas and Wei, Daniel and Madey, Gregory},title={{Lessons Learned from an Experiment in Crowdsourcing Complex Citizen Engineering Tasks with Amazon Mechanical Turk}},booktitle={{Collective Intelligence}},year={2014},cclpaperid={905},}
Poster: Expanding Tasks of Logical Workflows into Independent Workflows for Improved Scalability
Nicholas Hazekamp, Olivia Choudhury, Sandra Gesing, Scott Emrich, and Douglas Thain
In IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing, 2014
@inproceedings{workflow-expand-grid14,author={Hazekamp, Nicholas and Choudhury, Olivia and Gesing, Sandra and Emrich, Scott and Thain, Douglas},title={{Poster: Expanding Tasks of Logical Workflows into Independent Workflows for Improved Scalability}},booktitle={{IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing}},pages={548-549},year={2014},note={{doi: 10.1109/CCGrid.2014.84}},cclpaperid={901},}
2013
Automated Packaging of Bioinformatics Workflows for Portability and Durability Using Makeflow
Casey Robinson and Douglas Thain
In Workshop on Workflows in Support of Large-Scale Science (WORKS), 2013
@inproceedings{automated-packaging-works13,author={Robinson, Casey and Thain, Douglas},title={{Automated Packaging of Bioinformatics Workflows for Portability and Durability Using Makeflow}},booktitle={{Workshop on Workflows in Support of Large-Scale Science (WORKS)}},year={2013},note={{doi: 10.1145/2534248.2534258}},cclpaperid={899},}
Design of an Active Storage Cluster File System for DAG Workflows
Patrick Donnelly and Douglas Thain
In International Workshop on Data-Intensive Scalable Computing Systems, 2013
@inproceedings{confuga-discs2013,author={Donnelly, Patrick and Thain, Douglas},title={{Design of an Active Storage Cluster File System for DAG Workflows}},booktitle={{International Workshop on Data-Intensive Scalable Computing Systems}},pages={37-42},year={2013},note={{doi: 10.1145/2534645.2534656}},cclpaperid={900},}
Toward Fine Grained Online Task Chracteristics Estimation in Scientific Workflows
Rafael Ferreira Silva, Gideon Juve, Ewa Deelman, Tristan Glatard, Frederic Desprez, Douglas Thain, Benjamin Tovar, and Miron Livny
In Workshop on Workflows in Support of Large Scale Science (WORKS), 2013
@inproceedings{tasks-works-2013,author={da Silva, Rafael Ferreira and Juve, Gideon and Deelman, Ewa and Glatard, Tristan and Desprez, Frederic and Thain, Douglas and Tovar, Benjamin and Livny, Miron},title={{Toward Fine Grained Online Task Chracteristics Estimation in Scientific Workflows}},booktitle={{Workshop on Workflows in Support of Large Scale Science (WORKS)}},pages={58-67},year={2013},note={{doi: 10.1145/2534248.2534254}},cclpaperid={980},}
Making Work Queue Cluster-Friendly for Data Intensive Scientific Applications
Michael Albrecht, Dinesh Rajan, and Douglas Thain
In IEEE International Conference on Cluster Computing, 2013
@inproceedings{wqh-cluster13,author={Albrecht, Michael and Rajan, Dinesh and Thain, Douglas},title={{Making Work Queue Cluster-Friendly for Data Intensive Scientific Applications}},booktitle={{IEEE International Conference on Cluster Computing}},year={2013},note={{doi: 10.1109/CLUSTER.2013.6702628}},cclpaperid={898},}
Right-sizing Resource Allocations for Scientific Applications in Clusters, Grids, and Clouds
@thesis{PAPER896,author={Yu, Li},title={{Right-sizing Resource Allocations for Scientific Applications in Clusters, Grids, and Clouds}},editor={Thesis, Ph.D.},booktitle={{University of Notre Dame}},year={2013},cclpaperid={896},}
Case Studies in Designing Elastic Applications
Dinesh Rajan, Andrew Thrasher, Badi Abdul-Wahid, Jesus A Izaguirre, Scott Emrich, and Douglas Thain
In 13th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (CCGrid), 2013
@inproceedings{casestudies-ccgrid13,author={Rajan, Dinesh and Thrasher, Andrew and Abdul-Wahid, Badi and Izaguirre, Jesus A and Emrich, Scott and Thain, Douglas},title={{Case Studies in Designing Elastic Applications}},booktitle={{ 13th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (CCGrid)}},year={2013},note={{doi: 0.1109/CCGrid.2013.46}},cclpaperid={893},}
Search Should be a System Call
Brenden Kokoszka, Patrick Donnelly, and Douglas Thain
@techreport{search-tr,author={Kokoszka, Brenden and Donnelly, Patrick and Thain, Douglas},title={{Search Should be a System Call}},institution={{University of Notre Dame, Computer Science and Engineering Department}},number={2013-03},year={2013},cclpaperid={894},}
2012
A Framework for Scalable Genome Assembly on Clusters, Clouds, and Grids
Christopher Moretti, Andrew Thrasher, Li Yu, Michael Olson, Scott Emrich, and Douglas Thain
IEEE Transactions on Parallel and Distributed Systems, 2012
@article{assembly-tpds,author={Moretti, Christopher and Thrasher, Andrew and Yu, Li and Olson, Michael and Emrich, Scott and Thain, Douglas},title={{A Framework for Scalable Genome Assembly on Clusters, Clouds, and Grids}},journal={{IEEE Transactions on Parallel and Distributed Systems}},volume={23},number={12},year={2012},note={{doi: 10.1109/TPDS.2012.80}},cclpaperid={100},}
Folding Proteins at 500 ns/hour with Work Queue
Badi Abdul-Wahid, Li Yu, Dinesh Rajan, Haoyun Feng, Eric Darve, Douglas Thain, and Jesus A. Izaguirre
In 8th IEEE International Conference on eScience (eScience 2012), 2012
@inproceedings{folding-escience12,author={Abdul-Wahid, Badi and Yu, Li and Rajan, Dinesh and Feng, Haoyun and Darve, Eric and Thain, Douglas and Izaguirre, Jesus A.},title={{Folding Proteins at 500 ns/hour with Work Queue}},booktitle={{8th IEEE International Conference on eScience (eScience 2012)}},year={2012},note={{doi: 10.1109/eScience.2012.6404429}},cclpaperid={891},}
A System for Management of Computational Fluid Dynamics Simulations for Civil Engineering
Peter Sempolinski, Daniel Wei, Douglas Thain, and Ahsan Kareem
In 8th IEEE International Conference on eScience, 2012
@inproceedings{psempoli-escience12,author={Sempolinski, Peter and Wei, Daniel and Thain, Douglas and Kareem, Ahsan},title={{A System for Management of Computational Fluid Dynamics Simulations for Civil Engineering}},booktitle={{8th IEEE International Conference on eScience}},pages={1-8},year={2012},note={{doi: 10.1109/eScience.2012.6404433 }},cclpaperid={892},}
ROARS: A Robust Object Archival System for Data Intensive Scientific Computing
Hoang Bui, Peter Bui, Patrick Flynn, and Douglas Thain
@article{roars-pddb-2012,author={Bui, Hoang and Bui, Peter and Flynn, Patrick and Thain, Douglas},title={{ROARS: A Robust Object Archival System for Data Intensive Scientific Computing}},journal={{Distributed and Parallel Databases}},volume={30},number={5-6},publisher={Springer},year={2012},note={{doi: 10.1007/s10619-012-7103-5}},cclpaperid={890},}
A Compiler Toolchain For Data Intensive Scientific Workflows
@thesis{pbui-dissertation.pdf,author={Bui, Peter},title={{A Compiler Toolchain For Data Intensive Scientific Workflows}},editor={Thesis, Ph.D.},booktitle={{University of Notre Dame}},year={2012},cclpaperid={889},}
Fine-Grained Access Control in the Chirp Distributed File System
Patrick Donnelly and Douglas Thain
In IEEE/ACM International Symposium on Cluster, Cloud, and Grid Computing, 2012
@inproceedings{chirp-tickets-ccgrid12,author={Donnelly, Patrick and Thain, Douglas},title={{Fine-Grained Access Control in the Chirp Distributed File System}},booktitle={{IEEE/ACM International Symposium on Cluster, Cloud, and Grid Computing}},year={2012},note={{doi: 10.1109/CCGrid.2012.128}},cclpaperid={101},}
Makeflow: A Portable Abstraction for Data Intensive Computing on Clusters, Clouds, and Grids
Michael Albrecht, Patrick Donnelly, Peter Bui, and Douglas Thain
In Workshop on Scalable Workflow Enactment Engines and Technologies (SWEET) at ACM SIGMOD, 2012
@inproceedings{makeflow-sweet12,author={Albrecht, Michael and Donnelly, Patrick and Bui, Peter and Thain, Douglas},title={{Makeflow: A Portable Abstraction for Data Intensive Computing on Clusters, Clouds, and Grids}},booktitle={{Workshop on Scalable Workflow Enactment Engines and Technologies (SWEET) at ACM SIGMOD}},year={2012},note={{doi: 10.1145/2443416.2443417}},cclpaperid={104},}
@inproceedings{maker-iccabs12,author={Thrasher, Andrew and Musgrave, Zachary and Thain, Douglas and Emrich, Scott},title={{Shifting the Bioinformatics Computing Paradigm: A Case Study in Parallelizing Genome Annotation Using Maker and Work Queue}},booktitle={{IEEE International Conference on Computational Advances in Bio and Medical Sciences}},year={2012},cclpaperid={102},}
Data Intensive Computing with Clustered Chirp Servers
Douglas Thain, Michael Albrecht, Hoang Bui, Peter Bui, Rory Carmichael, Scott Emrich, and Patrick Flynn
In Data Intensive Distributed Computing: Challenges and Solutions for Large Scale Information Management, 2012
@incollection{chirp-didc-chapter,author={Thain, Douglas and Albrecht, Michael and Bui, Hoang and Bui, Peter and Carmichael, Rory and Emrich, Scott and Flynn, Patrick},title={{Data Intensive Computing with Clustered Chirp Servers}},editor={Kosar, Tevfik},booktitle={{Data Intensive Distributed Computing: Challenges and Solutions for Large Scale Information Management}},pages={140-154},publisher={IGI},year={2012},note={{isbn: 9781615209712}},doi={10.4018/978-1-61520-971-2.ch009},cclpaperid={99}}
2011
Biocompute 2.0: An Improved Collaborative Workspace for Data Intensive Bio-Science.
Rory Carmichael, Patrick Braga-Henebry, Douglas Thain, and Scott Emrich
Concurrency and Computation: Practice and Experience, 2011
@article{biocompute-ccpe,author={Carmichael, Rory and Braga-Henebry, Patrick and Thain, Douglas and Emrich, Scott},title={{Biocompute 2.0: An Improved Collaborative Workspace for Data Intensive Bio-Science.}},journal={{Concurrency and Computation: Practice and Experience}},volume={23},number={17},pages={2305-2314},year={2011},note={{doi: 10.1002/cpe.1782}},cclpaperid={96},}
Converting a High Performance Application to an Elastic Cloud Application
Dinesh Rajan, Anthony Canino, Jesus A Izaguirre, and Douglas Thain
In The 3rd IEEE International Conference on Cloud Computing Technology and Science (CloudCom 2011), 2011
@inproceedings{elasticrepex-cloudcom11,author={Rajan, Dinesh and Canino, Anthony and Izaguirre, Jesus A and Thain, Douglas},title={{Converting a High Performance Application to an Elastic Cloud Application}},booktitle={{The 3rd IEEE International Conference on Cloud Computing Technology and Science (CloudCom 2011)}},year={2011},cclpaperid={93},}
Work Queue + Python: A Framework For Scalable Scientific Ensemble Applications
Peter Bui, Dinesh Rajan, Badi Abdul-Wahid, Jesus Izaguirre, and Douglas Thain
In Workshop on Python for High Performance and Scientific Computing (PyHPC) at the ACM/IEEE International Conference for High Performance Computing, Networking, Storage, and Analysis (Supercomputing) , 2011
@inproceedings{wq-python-pyhpc2011,author={Bui, Peter and Rajan, Dinesh and Abdul-Wahid, Badi and Izaguirre, Jesus and Thain, Douglas},title={{Work Queue + Python: A Framework For Scalable Scientific Ensemble Applications}},booktitle={{Workshop on Python for High Performance and Scientific Computing (PyHPC) at the ACM/IEEE International Conference for High Performance Computing, Networking, Storage, and Analysis (Supercomputing) }},year={2011},cclpaperid={95},}
Scripting distributed scientific workflows using Weaver
Peter Bui, Li Yu, Andrew Thrasher, Rory Carmichael, Irena Lanc, Patrick Donnelly, and Douglas Thain
Concurrency and Computation: Practice and Experience, 2011
@article{weaver-ccpe,author={Bui, Peter and Yu, Li and Thrasher, Andrew and Carmichael, Rory and Lanc, Irena and Donnelly, Patrick and Thain, Douglas},title={{Scripting distributed scientific workflows using Weaver}},journal={{Concurrency and Computation: Practice and Experience}},volume={24},number={15},year={2011},note={{doi: 10.1002/cpe.1871}},cclpaperid={98},}
Adapting Bioinformatics Applications for Heterogeneous Systems: A Case Study
Irena Lanc, Peter Bui, Douglas Thain, and Scott Emrich
In Emerging Computational Methods for the Life Sciences Workshop at ACM HPDC, 2011
@inproceedings{adapting-ecmls11,author={Lanc, Irena and Bui, Peter and Thain, Douglas and Emrich, Scott},title={{Adapting Bioinformatics Applications for Heterogeneous Systems: A Case Study}},booktitle={{Emerging Computational Methods for the Life Sciences Workshop at ACM HPDC}},pages={7-13},year={2011},note={{doi: 10.1145/1996023.1996025}},cclpaperid={94},}
An Introduction to Open-Source IaaS Cloud Middleware
Peter Sempolinski and Douglas Thain
In Cloud Computing: Methodology, Systems, and Applications, 2011
@incollection{cloud-chapter,author={Sempolinski, Peter and Thain, Douglas},title={{An Introduction to Open-Source IaaS Cloud Middleware}},editor={Benatallah, Boualem},booktitle={{Cloud Computing: Methodology, Systems, and Applications}},pages={133-149},publisher={CRC Press},year={2011},note={{isbn: 978-1-4398-5641-3}},doi={10.1201/b11149-7},cclpaperid={895}}
2010
A Comparison and Critique of Eucalyptus, OpenNebula and Nimbus
Peter Sempolinski and Douglas Thain
In IEEE International Conference on Cloud Computing Technology and Science, 2010
@inproceedings{survey-cloudCom2010,author={Sempolinski, Peter and Thain, Douglas},title={{A Comparison and Critique of Eucalyptus, OpenNebula and Nimbus}},booktitle={{IEEE International Conference on Cloud Computing Technology and Science}},pages={417-426},year={2010},note={{doi: 10.1109/CloudCom.2010.42}},cclpaperid={89},}
Attaching Cloud Storage to a Campus Grid Using Parrot, Chirp, and Hadoop
Patrick Donnelly, Peter Bui, and Douglas Thain
In IEEE International Conference on Cloud Computing Technology and Science, 2010
@inproceedings{chirp+parrot+hdfs,author={Donnelly, Patrick and Bui, Peter and Thain, Douglas},title={{Attaching Cloud Storage to a Campus Grid Using Parrot, Chirp, and Hadoop}},booktitle={{IEEE International Conference on Cloud Computing Technology and Science}},pages={488-495},year={2010},note={{doi: 10.1109/CloudCom.2010.74}},cclpaperid={90},}
Taming Complex Bioinformatics Workflows with Weaver, Makeflow, and Starch
Andrew Thrasher, Rory Carmichael, Peter Bui, Li Yu, Douglas Thain, and Scott Emrich
In Workshop on Workflows in Support of Large Scale Science, 2010
@inproceedings{taming-works10.pdf,author={Thrasher, Andrew and Carmichael, Rory and Bui, Peter and Yu, Li and Thain, Douglas and Emrich, Scott},title={{Taming Complex Bioinformatics Workflows with Weaver, Makeflow, and Starch}},booktitle={{Workshop on Workflows in Support of Large Scale Science}},pages={1-6},year={2010},note={{doi: 10.1109/WORKS.2010.5671858}},cclpaperid={92},}
Harnessing Parallelism in Multicore Clusters with the All-Pairs, Wavefront, and Makeflow Abstractions
Li Yu, Christopher Moretti, Andrew Thrasher, Scott Emrich, Kenneth Judd, and Douglas Thain
@article{abstr-jcc,author={Yu, Li and Moretti, Christopher and Thrasher, Andrew and Emrich, Scott and Judd, Kenneth and Thain, Douglas},title={{Harnessing Parallelism in Multicore Clusters with the All-Pairs, Wavefront, and Makeflow Abstractions}},journal={{Journal of Cluster Computing}},volume={13},number={3},pages={243-256},year={2010},note={{doi: 10.1007/s10586-010-0134-7}},cclpaperid={83},}
Environmentally Opportunistic Computing: Transforming the Data Center for Economic and Environmental Sustainability
Paul Brenner, Ryan Jansen, David Go, and Douglas Thain
@inproceedings{eoc-green,author={Brenner, Paul and Jansen, Ryan and Go, David and Thain, Douglas},title={{Environmentally Opportunistic Computing: Transforming the Data Center for Economic and Environmental Sustainability}},booktitle={{IEEE Green Computing Conference}},pages={383-388},year={2010},note={{doi: 10.1109/GREENCOMP.2010.5598289 }},cclpaperid={97},}
Abstractions for Cloud Computing with Condor
Douglas Thain and Christopher Moretti
In Cloud Computing and Software Services: Theory and Techniques, 2010
@incollection{abstr-cloudbook,author={Thain, Douglas and Moretti, Christopher},title={{Abstractions for Cloud Computing with Condor}},editor={Ahson, Syed and Ilyas, Mohammad},booktitle={{Cloud Computing and Software Services: Theory and Techniques}},pages={153-171},publisher={CRC Press},year={2010},note={{isbn: 9781439803158}},cclpaperid={78},}
ROARS: A Scalable Repository for Data Intensive Scientific Computing
Hoang Bui, Peter Bui, Patrick Flynn, and Douglas Thain
In The Third International Workshop on Data Intensive Distributed Computing at ACM HPDC 2010, 2010
@inproceedings{roars-didc10,author={Bui, Hoang and Bui, Peter and Flynn, Patrick and Thain, Douglas},title={{ROARS: A Scalable Repository for Data Intensive Scientific Computing}},booktitle={{The Third International Workshop on Data Intensive Distributed Computing at ACM HPDC 2010}},year={2010},note={{doi: 10.1145/1851476.1851587}},cclpaperid={85},}
Weaver: Integrating Distributed Computing Abstractions into Scientific Workflows using Python
Peter Bui, Li Yu, and Douglas Thain
In Challenges of Large Applications in Distributed Environments at ACM HPDC 2010, 2010
@inproceedings{weaver-clade10,author={Bui, Peter and Yu, Li and Thain, Douglas},title={{Weaver: Integrating Distributed Computing Abstractions into Scientific Workflows using Python}},booktitle={{Challenges of Large Applications in Distributed Environments at ACM HPDC 2010}},year={2010},note={{doi: 10.1145/1851476.1851570}},cclpaperid={86},}
Towards Long Term Data Quality in a Large Scale Biometrics Experiment
Hoang Bui, Diane Wright, Clarence Helm, Rachel Witty, Patrick Flynn, and Douglas Thain
In Managing Data Quality for Collaborative Science at ACM HPDC 2010, 2010
@inproceedings{quality-didc10,author={Bui, Hoang and Wright, Diane and Helm, Clarence and Witty, Rachel and Flynn, Patrick and Thain, Douglas},title={{Towards Long Term Data Quality in a Large Scale Biometrics Experiment}},booktitle={{Managing Data Quality for Collaborative Science at ACM HPDC 2010}},year={2010},note={{doi: 10.1145/1851476.1851559}},cclpaperid={87},}
Abstractions for Scientific Computing on Campus Grids
@thesis{moretti-dissertation,author={Moretti, Christopher},title={{Abstractions for Scientific Computing on Campus Grids}},editor={Thesis, Ph.D.},booktitle={{University of Notre Dame}},year={2010},cclpaperid={88},}
Biocompute: Toward a Collaborative Workspace for Data Intensive Bio-Science
Rory Carmichael, Patrick Braga-Henebry, Douglas Thain, and Scott Emrich
In Workshop on Emerging Computational Methods for Life Sciences at ACM HPDC 2010, 2010
@inproceedings{biocompute-ecmls,author={Carmichael, Rory and Braga-Henebry, Patrick and Thain, Douglas and Emrich, Scott},title={{Biocompute: Toward a Collaborative Workspace for Data Intensive Bio-Science}},booktitle={{Workshop on Emerging Computational Methods for Life Sciences at ACM HPDC 2010}},pages={489-498},year={2010},note={{doi: 10.1145/1851476.1851547}},cclpaperid={91},}
All-Pairs: An Abstraction for Data Intensive Computing on Campus Grids
Christopher Moretti, Hoang Bui, Karen Hollingsworth, Brandon Rich, Patrick Flynn, and Douglas Thain
IEEE Transactions on Parallel and Distributed Systems, 2010
@article{allpairs-tpds,author={Moretti, Christopher and Bui, Hoang and Hollingsworth, Karen and Rich, Brandon and Flynn, Patrick and Thain, Douglas},title={{All-Pairs: An Abstraction for Data Intensive Computing on Campus Grids}},journal={{IEEE Transactions on Parallel and Distributed Systems}},volume={21},number={1},pages={33-46},year={2010},note={{doi: 10.1109/TPDS.2009.49}},cclpaperid={12},}
Visualizing Massively Multithreaded Applications with ThreadScope
Kyle Wheeler and Douglas Thain
Concurrency and Computation: Practice and Experience, 2010
@article{threadscope-ccpe09,author={Wheeler, Kyle and Thain, Douglas},title={{Visualizing Massively Multithreaded Applications with ThreadScope}},journal={{Concurrency and Computation: Practice and Experience}},volume={22},number={1},pages={45-67},year={2010},note={{doi: 10.1002/cpe.1469}},cclpaperid={59},}
2009
Towards Data Intensive Many Task Computing
Ioan Raicu, Ian Foster, Yong Zhao, Alex Szalay, Philip Little, Christopher Moretti, Amitabh Chaudhary, and Douglas Thain
In Data Intensive Distributed Computing: Challenges and Solutions for Large-Scale Information Management, 2009
@incollection{raicu-manytask-chapter,author={Raicu, Ioan and Foster, Ian and Zhao, Yong and Szalay, Alex and Little, Philip and Moretti, Christopher and Chaudhary, Amitabh and Thain, Douglas},title={{Towards Data Intensive Many Task Computing}},editor={Kosar, Tevfik},booktitle={{Data Intensive Distributed Computing: Challenges and Solutions for Large-Scale Information Management}},publisher={IGI},year={2009},doi={10.4018/978-1-61520-971-2.ch010},cclpaperid={80}}
Exploiting Locality with QThreads for Portable Parallel Performance
@thesis{wheeler-thesis,author={Wheeler, Kyle},title={{Exploiting Locality with QThreads for Portable Parallel Performance}},editor={Thesis, Ph.D.},booktitle={{University of Notre Dame}},year={2009},cclpaperid={81},}
Highly Scalable Genome Assembly on Campus Grids
Christopher Moretti, Michael Olson, Scott Emrich, and Douglas Thain
In Many-Task Computing on Grids and Supercomputers (MTAGS), 2009
@inproceedings{assembly-mtags09,author={Moretti, Christopher and Olson, Michael and Emrich, Scott and Thain, Douglas},title={{Highly Scalable Genome Assembly on Campus Grids}},booktitle={{Many-Task Computing on Grids and Supercomputers (MTAGS)}},year={2009},note={{doi: 10.1145/1646468.1646480}},cclpaperid={82},}
Reflections on the Virtues of Modularity: A Case Study in Linux Security Modules
@article{modularity-spe,author={Blaich, Andrew and Thain, Douglas and Striegel, Aaron},title={{Reflections on the Virtues of Modularity: A Case Study in Linux Security Modules}},journal={{Software: Practice and Experience}},volume={39},number={15},pages={1235-1251},year={2009},note={{doi: 10.1002/spe.933}},cclpaperid={11},}
Cooperative Localization in GPS-Limited Urban Environments
Jeffrey Hemmes, Douglas Thain, and Christian Poellabauer
@inproceedings{teamtrak-adhocnets,author={Hemmes, Jeffrey and Thain, Douglas and Poellabauer, Christian},title={{Cooperative Localization in GPS-Limited Urban Environments}},booktitle={{ICST AdHocNets}},year={2009},cclpaperid={61},}
Scheduling Grid Workloads on Multicore Clusters to Minimize Energy and Maximize Performance
@inproceedings{multicore-grid09,author={Lammie, Michael and Thain, Douglas and Brenner, Paul},title={{Scheduling Grid Workloads on Multicore Clusters to Minimize Energy and Maximize Performance}},booktitle={{IEEE Grid Computing}},year={2009},cclpaperid={75},}
Portable Performance from Workstation to Supercomputer: Distributing Data Structures with Qthreads
Kyle Wheeler, Douglas Thain, and Rich Murphy
In Workshop on Programming Models for Emerging Architectures, 2009
@inproceedings{qthreads-pmea09,author={Wheeler, Kyle and Thain, Douglas and Murphy, Rich},title={{Portable Performance from Workstation to Supercomputer: Distributing Data Structures with Qthreads}},booktitle={{Workshop on Programming Models for Emerging Architectures}},pages={1-8},year={2009},cclpaperid={76},}
Coordination of Access to Large-scale Datasets in Distributed Environments
T. Kosar, A. Hutanu, J. McLaren, and D. Thain
In Scientific Data Management: Challenges, Existing Technology, and Deployment, 2009
@incollection{kosar-databook,author={Kosar, T. and Hutanu, A. and McLaren, J. and Thain, D.},title={{Coordination of Access to Large-scale Datasets in Distributed Environments}},editor={Shoshani, A. and Rotem, D.},booktitle={{Scientific Data Management: Challenges, Existing Technology, and Deployment}},publisher={Chapman and Hall / CRC Press},year={2009},note={{isbn: 978-1420069808}},doi={10.1201/9781420069815-12},cclpaperid={8}}
Scalable Modular Genome Assembly on Campus Grids
Christopher Moretti, Michael Olson, Scott Emrich, and Douglas Thain
@techreport{assembly-tr,author={Moretti, Christopher and Olson, Michael and Emrich, Scott and Thain, Douglas},title={{Scalable Modular Genome Assembly on Campus Grids}},institution={{University of Notre Dame, Computer Science and Engineering Department}},number={2009-04},year={2009},cclpaperid={77},}
Improving Data Availability in Mobile Applications Through Enhanced Cooperative Localization
@thesis{jhemmes-thesis,author={Hemmes, Jeffrey},title={{Improving Data Availability in Mobile Applications Through Enhanced Cooperative Localization}},editor={Thesis, Ph.D.},booktitle={{University of Notre Dame}},year={2009},cclpaperid={79},}
Harnessing Parallelism in Multicore Clusters with the All-Pairs and Wavefront Abstractions
Li Yu, Christopher Moretti, Scott Emrich, Kenneth Judd, and Douglas Thain
In IEEE High Performance Distributed Computing, 2009
@inproceedings{abstr-hpdc09,author={Yu, Li and Moretti, Christopher and Emrich, Scott and Judd, Kenneth and Thain, Douglas},title={{Harnessing Parallelism in Multicore Clusters with the All-Pairs and Wavefront Abstractions}},booktitle={{IEEE High Performance Distributed Computing}},pages={1-10},year={2009},note={{doi: 10.1145/1551609.1551613}},cclpaperid={5},}
Grid Heating Clusters: Transforming Cooling Constraints Into Thermal Benefits
Paul Brenner, Douglas Thain, and Dewitt Latimer
In The Uptime Institute Green Enterprise IT Award Paper, 2009
@inproceedings{gheat-uptime09,author={Brenner, Paul and Thain, Douglas and Latimer, Dewitt},title={{Grid Heating Clusters: Transforming Cooling Constraints Into Thermal Benefits}},booktitle={{The Uptime Institute Green Enterprise IT Award Paper}},pages={1-7},year={2009},cclpaperid={60},}
CDF Software Distribution on the Grid using Parrot
Gabrielle Compostella, Simone Pagan Griso, Donatella Lucchesi, Igor Sfiligoi, and Douglas Thain
@inproceedings{parrot-chep09,author={Compostella, Gabrielle and Griso, Simone Pagan and Lucchesi, Donatella and Sfiligoi, Igor and Thain, Douglas},title={{CDF Software Distribution on the Grid using Parrot}},booktitle={{Computing in High Energy Physics}},year={2009},note={{doi: 10.1088/1742-6596/219/6/062009}},cclpaperid={22},}
Biocompute: Harnessing Distributed Systems for Bioinformatics
@thesis{pbh-thesis,author={Braga-Henebry, Patrick},title={{Biocompute: Harnessing Distributed Systems for Bioinformatics}},editor={Thesis, B.S. Honors},booktitle={{University of Notre Dame}},year={2009},cclpaperid={64},}
Experience with BXGrid: A Data Repository and Computing Grid for Biometrics Research
Hoang Bui, Michael Kelly, Christopher Lyon, Mark Pasquier, Deborah Thomas, Patrick Flynn, and Douglas Thain
@article{bxgrid-jcc,author={Bui, Hoang and Kelly, Michael and Lyon, Christopher and Pasquier, Mark and Thomas, Deborah and Flynn, Patrick and Thain, Douglas},title={{Experience with BXGrid: A Data Repository and Computing Grid for Biometrics Research}},journal={{Journal of Cluster Computing}},volume={12},number={4},pages={373},year={2009},note={{doi: 10.1007/s10586-009-0098-7}},cclpaperid={1},}
Chirp: A Practical Global Filesystem for Cluster and Grid Computing
Douglas Thain, Christopher Moretti, and Jeffrey Hemmes
@article{chirp-jgc,author={Thain, Douglas and Moretti, Christopher and Hemmes, Jeffrey},title={{Chirp: A Practical Global Filesystem for Cluster and Grid Computing}},journal={{Journal of Grid Computing}},volume={7},number={1},pages={51-72},year={2009},note={{doi: 10.1007/s10723-008-9100-5}},cclpaperid={14},}
2008
Poster: BXGrid: A Data Repository and Workflow Abstraction for Biometrics Research
Hoang Bui, Deborah Thomas, Michael Kelly, Christopher Lyon, Douglas Thain, and Patrick J. Flynn
@article{bxgrid-escience09,author={Bui, Hoang and Thomas, Deborah and Kelly, Michael and Lyon, Christopher and Thain, Douglas and Flynn, Patrick J.},title={{Poster: BXGrid: A Data Repository and Workflow Abstraction for Biometrics Research}},booktitle={{IEEE International Conference on e-Science}},pages={394-395},year={2008},note={{isbn: 10.1109/eScience.2008.135}},cclpaperid={24},}
Scaling Up Classifiers to Cloud Computers
Christopher Moretti, Karsten Steinhaeuser, Douglas Thain, and Nitesh V. Chawla
In IEEE International Conference on Data Mining (ICDM), 2008
@inproceedings{classify-icdm08,author={Moretti, Christopher and Steinhaeuser, Karsten and Thain, Douglas and Chawla, Nitesh V.},title={{Scaling Up Classifiers to Cloud Computers}},booktitle={{IEEE International Conference on Data Mining (ICDM)}},pages={472-481},year={2008},note={{doi: 10.1109/ICDM.2008.99}},cclpaperid={25},}
Biomolecular Committor Probability Calculation Enabled by Processing in Network Storage
Paul Brenner, Justin Wozniak, Douglas Thain Aaron Striegel, Jeffrey Peng, and Jesus Izaguirre
@article{pins-jpc,author={Brenner, Paul and Wozniak, Justin and Striegel, Douglas Thain Aaron and Peng, Jeffrey and Izaguirre, Jesus},title={{Biomolecular Committor Probability Calculation Enabled by Processing in Network Storage}},journal={{Journal of Parallel Computing}},volume={34},number={11},pages={652-660},year={2008},note={{doi: 10.1016/j.parco.2008.08.001}},doi={10.1016/j.parco.2008.08.001},cclpaperid={13},}
@inproceedings{enavis-lisa08,author={Liao, Qi and Blaich, Andrew and Striegel, Aaron and Thain, Douglas},title={{ENAVis: Enterprise Network Activities Visualization}},booktitle={{USENIX Large Installation System Administration (LISA)}},pages={59-74},year={2008},cclpaperid={23},}
Troubleshooting Thousands of Jobs on Production Grids Using Data Mining Techniques
@inproceedings{debug-grid08,author={Cieslak, David and Chawla, Nitesh and Thain, Douglas},title={{Troubleshooting Thousands of Jobs on Production Grids Using Data Mining Techniques}},booktitle={{IEEE Grid Computing}},pages={217-224},year={2008},note={{doi: 10.1109/GRID.2008.4662802}},cclpaperid={26},}
Poster: DataLab: Transactional Data Parallel Computing on an Active Storage Cloud
Brandon Rich and Douglas Thain
In IEEE/ACM High Performance Distributed Computing, 2008
@inproceedings{datalab-hpdc08,author={Rich, Brandon and Thain, Douglas},title={{Poster: DataLab: Transactional Data Parallel Computing on an Active Storage Cloud}},booktitle={{IEEE/ACM High Performance Distributed Computing}},pages={233-234},year={2008},note={{isbn: 10.1145/1383422.1383461}},cclpaperid={27},}
All-Pairs: An Abstraction for Data Intensive Cloud Computing
Christopher Moretti, Jared Bulosan, Douglas Thain, and Patrick Flynn
In IEEE International Parallel and Distributed Processing Symposium (IPDPS), 2008
@inproceedings{allpairs-ipdps08,author={Moretti, Christopher and Bulosan, Jared and Thain, Douglas and Flynn, Patrick},title={{All-Pairs: An Abstraction for Data Intensive Cloud Computing}},booktitle={{IEEE International Parallel and Distributed Processing Symposium (IPDPS)}},pages={1-11},year={2008},note={{doi: 10.1109/IPDPS.2008.4536311 }},cclpaperid={28},}
QThreads: An API for Programming with Millions of Lightweight Threads
Kyle Wheeler, Richard Murphy, and Douglas thain
In Workshop on Multithreaded Architectures and Applications at IEEE IPDPS, 2008
@inproceedings{qthreads-mtaap08,author={Wheeler, Kyle and Murphy, Richard and thain, Douglas},title={{QThreads: An API for Programming with Millions of Lightweight Threads}},booktitle={{Workshop on Multithreaded Architectures and Applications at IEEE IPDPS}},pages={1-8},year={2008},note={{doi: 10.1109/IPDPS.2008.4536359}},cclpaperid={29},}
Towards a Data Analysis Grid for Biometrics Research
@techreport{bxgrid-tr,author={Thain, Douglas and Flynn, Patrick},title={{Towards a Data Analysis Grid for Biometrics Research}},institution={{University of Notre Dame, Computer Science and Engineering Department}},number={2008-05},year={2008},cclpaperid={62},}
Making the Best of a Bad Situation: Prioritized Storage Management in GEMS
Justin Wozniak, Paul Brenner, Douglas Thain, Aaron Striegel, and Jesus Izaguirre
@article{priority-fgcs,author={Wozniak, Justin and Brenner, Paul and Thain, Douglas and Striegel, Aaron and Izaguirre, Jesus},title={{Making the Best of a Bad Situation: Prioritized Storage Management in GEMS}},journal={{Future Generation Computing Systems}},volume={24},number={1},pages={10-16},year={2008},note={{doi: 10.1016/j.future.2007.04.003}},cclpaperid={15},}
Grid Heating: Transforming Cooling Constraints Into Thermal Benefits
@techreport{gheat-tr,author={Brenner, Paul and Thain, Douglas and Latimer, Dewitt},title={{Grid Heating: Transforming Cooling Constraints Into Thermal Benefits}},institution={{University of Notre Dame, Computer Science and Engineering Department}},number={2008-09},year={2008},cclpaperid={7},}
2007
On Demand Transient Storage and Backup in Mobile Systems
Jeffrey Hemmes, Christian Poellabauer, and Douglas Thain
@inproceedings{teamtrak-milcom07,author={Hemmes, Jeffrey and Poellabauer, Christian and Thain, Douglas},title={{On Demand Transient Storage and Backup in Mobile Systems}},booktitle={{IEEE Military Communications Conference}},pages={1-7},year={2007},note={{doi: 10.1109/MILCOM.2007.4454917}},cclpaperid={30},}
Lessons Learned Building TeamTrak: An Urban/Outdoor Mobile Testbed
Jeffrey Hemmes, Douglas Thain, Christian Poellabauer, Christopher Moretti, Phil Snowberger, and Brendan McNutt
In International Conference on Wireless Architectures Systems and Applications, 2007
@inproceedings{teamtrak-wasa07,author={Hemmes, Jeffrey and Thain, Douglas and Poellabauer, Christian and Moretti, Christopher and Snowberger, Phil and McNutt, Brendan},title={{Lessons Learned Building TeamTrak: An Urban/Outdoor Mobile Testbed}},booktitle={{International Conference on Wireless Architectures Systems and Applications}},pages={219-224},year={2007},note={{doi: 10.1109/WASA.2007.35}},cclpaperid={33},}
Work in Progress: Integrating Undergraduate Research and Education via the TeamTrak Mobile Computing Framework
Jeffrey Hemmes, Douglas Thain, and Christian Poellabauer
@inproceedings{teamtrak-fie07,author={Hemmes, Jeffrey and Thain, Douglas and Poellabauer, Christian},title={{Work in Progress: Integrating Undergraduate Research and Education via the TeamTrak Mobile Computing Framework}},booktitle={{IEEE Frontiers in Education}},pages={1-2},year={2007},note={{doi: 10.1109/FIE.2007.4418007}},cclpaperid={35},}
Efficient Access to Many Small Files in a Filesystem for Grid Computing
@inproceedings{small-grid07,author={Thain, Douglas and Moretti, Christopher},title={{Efficient Access to Many Small Files in a Filesystem for Grid Computing}},booktitle={{IEEE Grid Computing}},pages={243-250},year={2007},note={{doi: 10.1109/GRID.2007.4354139}},cclpaperid={31},}
Poster: Lockdown: Distributed Policy Analysis and Enforcement within the Enterprise Network
Andrew Blaich, Qi Liao, Greg Allan, Aaron Striegel, and Douglas Thain
@inproceedings{lockdown-security07,author={Blaich, Andrew and Liao, Qi and Allan, Greg and Striegel, Aaron and Thain, Douglas},title={{Poster: Lockdown: Distributed Policy Analysis and Enforcement within the Enterprise Network}},booktitle={{USENIX Security Symposium}},year={2007},cclpaperid={32},}
Poster: All-Pairs: An Abstraction for Data Intensive Computing
Christopher Moretti, Jared Bulosan, Douglas Thain, and Patrick J. Flynn
@inproceedings{allpairs-grid07,author={Moretti, Christopher and Bulosan, Jared and Thain, Douglas and Flynn, Patrick J.},title={{Poster: All-Pairs: An Abstraction for Data Intensive Computing}},booktitle={{IEEE/ACM Grid Computing}},year={2007},cclpaperid={63},}
Challenges in Executing Data Intensive Biometric Workloads on a Desktop Grid
Christopher Moretti, Timothy Faltemier, Douglas Thain, and Patrick J. Flynn
In Workshop on Large Scale and Volatile Desktop Grids at IEEE IPDPS, 2007
@inproceedings{challenges-pcgrid07,author={Moretti, Christopher and Faltemier, Timothy and Thain, Douglas and Flynn, Patrick J.},title={{Challenges in Executing Data Intensive Biometric Workloads on a Desktop Grid}},booktitle={{Workshop on Large Scale and Volatile Desktop Grids at IEEE IPDPS}},pages={481-489},year={2007},note={{doi: 10.1109/IPDPS.2007.370671}},cclpaperid={34},}
Biomolecular Path Sampling Enabled by Processing in Network Storage
Paul Brenner, Justin Wozniak, Douglas Thain, Aaron Striegel, Jeffrey Peng, and Jesus Izaguirre
In Workshop on High Performance Computational Biology at IEEE IPDPS, 2007
@inproceedings{pins-bio07,author={Brenner, Paul and Wozniak, Justin and Thain, Douglas and Striegel, Aaron and Peng, Jeffrey and Izaguirre, Jesus},title={{Biomolecular Path Sampling Enabled by Processing in Network Storage}},booktitle={{Workshop on High Performance Computational Biology at IEEE IPDPS}},pages={1-6},year={2007},note={{doi: 10.1109/IPDPS.2007.370446}},cclpaperid={36},}
Flexible Object Based Filesystems for Scientific Computing
@thesis{moretti-ms-thesis,author={Moretti, Christopher},title={{Flexible Object Based Filesystems for Scientific Computing}},editor={Thesis, M.S.},booktitle={{University of Notre Dame}},year={2007},cclpaperid={65},}
Sub-Identities: A Hierarchical Identity Model for Practical Containment
@thesis{snowberger-ms-thesis,author={Snowberger, Philip},title={{Sub-Identities: A Hierarchical Identity Model for Practical Containment}},editor={Thesis, M.S.},booktitle={{University of Notre Dame}},year={2007},cclpaperid={66},}
2006
How to Measure a Large Open Source Distributed System
Douglas Thain and Miron Livny
Concurrency and Computation: Practice and Experience, 2006
@article{measure-ccpe,author={Thain, Douglas and Livny, Miron},title={{How to Measure a Large Open Source Distributed System}},journal={{Concurrency and Computation: Practice and Experience}},volume={18},number={15},pages={1989-2019},year={2006},note={{doi: 10.1002/cpe.v18:15}},cclpaperid={16},}
Access Control for a Replica Management Database
Justin Wozniak, Paul Brenner, Douglas Thain, Aaron Striegel, and Jesus Izaguirre
In ACM Workshop on Storage Security and Survivability at ACM CCS, 2006
@inproceedings{gems-sss06,author={Wozniak, Justin and Brenner, Paul and Thain, Douglas and Striegel, Aaron and Izaguirre, Jesus},title={{Access Control for a Replica Management Database}},booktitle={{ACM Workshop on Storage Security and Survivability at ACM CCS}},pages={41-46},year={2006},note={{doi: 10.1145/1179559.1179567}},cclpaperid={40},}
Grid Deployment of Legacy Bioinformatics Applications with Transparent Data Access
Christophe Blanchet, Remi Mollon, Douglas Thain, and Gilbert Deleage
@inproceedings{bio-grid06,author={Blanchet, Christophe and Mollon, Remi and Thain, Douglas and Deleage, Gilbert},title={{Grid Deployment of Legacy Bioinformatics Applications with Transparent Data Access}},booktitle={{IEEE Grid Computing}},pages={120-127},year={2006},note={{doi: 10.1109/ICGRID.2006.311006}},cclpaperid={37},}
Operating System Support for Space Allocation in Grid Storage Systems
@inproceedings{alloc-grid06,author={Thain, Douglas},title={{Operating System Support for Space Allocation in Grid Storage Systems}},booktitle={{IEEE Grid Computing}},pages={104-111},year={2006},note={{doi: 10.1109/ICGRID.2006.311004}},cclpaperid={41},}
Cacheable Decentralized Groups for Grid Resource Access Control
@inproceedings{groups-grid06,author={Hemmes, Jeffrey and Thain, Douglas},title={{Cacheable Decentralized Groups for Grid Resource Access Control}},booktitle={{IEEE Grid Computing}},pages={192-199},year={2006},note={{doi: 10.1109/ICGRID.2006.311015}},cclpaperid={42},}
Positioning Dynamic Storage Caches for Transient Data
Sudharshan Vazhkudai, Douglas Thain, Xiaosong Ma, and Vincent Freeh
In Workshop on High-Performance I/O Techniques and Deployment of Very Large Scale I/O Systems at IEEE Cluster Computing, 2006
@inproceedings{caches-hyperio06,author={Vazhkudai, Sudharshan and Thain, Douglas and Ma, Xiaosong and Freeh, Vincent},title={{Positioning Dynamic Storage Caches for Transient Data}},booktitle={{Workshop on High-Performance I/O Techniques and Deployment of Very Large Scale I/O Systems at IEEE Cluster Computing}},pages={1-9},year={2006},note={{doi: 10.1109/CLUSTR.2006.311900}},cclpaperid={43},}
@inproceedings{subid-sec06,author={Snowberger, Phil and Thain, Douglas},title={{Poster: Sub-Identities: Security for Mere Mortals}},booktitle={{USENIX Security}},year={2006},cclpaperid={45},}
Experience with a Literate Approach to Computer Science
@inproceedings{literate-fie06,author={Thain, Douglas and Poellabauer, Christian},title={{Experience with a Literate Approach to Computer Science}},booktitle={{IEEE Frontiers in Education}},pages={8-13},year={2006},note={{doi: 10.1109/FIE.2006.322405}},cclpaperid={44},}
iDIBS: An Improved Distributed Internet Backup System
Faruck Morcos, Thidapat Chantem, Philip Little, Tiago Gasiba, and Douglas Thain
In IEEE International Conference on Parallel and Distributed Systems (ICPADS), 2006
@inproceedings{idibs-icpads06,author={Morcos, Faruck and Chantem, Thidapat and Little, Philip and Gasiba, Tiago and Thain, Douglas},title={{iDIBS: An Improved Distributed Internet Backup System}},booktitle={{IEEE International Conference on Parallel and Distributed Systems (ICPADS)}},pages={58-67},year={2006},note={{doi: 10.1109/ICPADS.2006.52}},cclpaperid={46},}
Transparent Access to Grid Resources for User Software
Sander Klous, Jamie Frey, Se-Chang Son, Douglas Thain, Alain Roy, Miron Livny, and Jo Brand
Concurrency and Computation: Practice and Experience, 2006
@article{transparent-ccpe,author={Klous, Sander and Frey, Jamie and Son, Se-Chang and Thain, Douglas and Roy, Alain and Livny, Miron and van den Brand, Jo},title={{Transparent Access to Grid Resources for User Software}},journal={{Concurrency and Computation: Practice and Experience}},volume={18},number={7},pages={787-801},year={2006},note={{doi: 10.1002/cpe.961 }},cclpaperid={17},}
Short Paper: Troubleshooting Distributed Systems via Data Mining
David Cieslak, Douglas Thain, and Nitesh Chawla
In IEEE Symposium on High Performance Distributed Computing (HPDC), 2006
@inproceedings{debug-hpdc15,author={Cieslak, David and Thain, Douglas and Chawla, Nitesh},title={{Short Paper: Troubleshooting Distributed Systems via Data Mining}},booktitle={{IEEE Symposium on High Performance Distributed Computing (HPDC)}},pages={309-312},year={2006},cclpaperid={47},}
Applying Feedback Control to a Replica Management System
Justin Wozniak, Paul Brenner, Douglas Thain, Aaron Striegel, and Jesus Izaguirre
In IEEE Southeastern Symposium on System Theory, 2006
@inproceedings{gems-ssst06,author={Wozniak, Justin and Brenner, Paul and Thain, Douglas and Striegel, Aaron and Izaguirre, Jesus},title={{Applying Feedback Control to a Replica Management System}},booktitle={{IEEE Southeastern Symposium on System Theory}},pages={472-476},year={2006},note={{doi: 10.1109/SSST.2006.1619125}},cclpaperid={48},}
Using Condor Glide-Ins and Parrot to Move from Dedicated Resources to the Grid
Stefano Belforte, Matthew Normal, Subir Sarkar, Ifor Sfiligoi, Douglas Thain, and Frank Wuerthwein
@article{parrot-lni06,author={Belforte, Stefano and Normal, Matthew and Sarkar, Subir and Sfiligoi, Ifor and Thain, Douglas and Wuerthwein, Frank},title={{Using Condor Glide-Ins and Parrot to Move from Dedicated Resources to the Grid}},journal={{Lecture Notes in Informatics}},volume={81},pages={285-292},year={2006},cclpaperid={49},}
Transparently Distributing CDF Software with Parrot
Douglas Thain, Christopher Moretti, and Igor Sfiligoi
@inproceedings{parrot-chep06,author={Thain, Douglas and Moretti, Christopher and Sfiligoi, Igor},title={{Transparently Distributing CDF Software with Parrot}},booktitle={{Computing in High Energy Physics}},pages={1-4},year={2006},cclpaperid={50},}
2005
The Consequences of Decentralized Security in a Cooperative Storage System
Douglas Thain, Christopher Moretti, Paul Madrid, Phil Snowberger, and Jeff Hemmes
In Workshop on Security in Storage at IEEE FAST, 2005
@inproceedings{cons-sisw05,author={Thain, Douglas and Moretti, Christopher and Madrid, Paul and Snowberger, Phil and Hemmes, Jeff},title={{The Consequences of Decentralized Security in a Cooperative Storage System}},booktitle={{Workshop on Security in Storage at IEEE FAST}},pages={82-94},year={2005},note={{doi: 10.1109/SISW.2005.11}},cclpaperid={51},}
Separating Abstractions from Resources in a Tactical Storage System
Douglas Thain, Sander Klous, Justin Wozniak, Paul Brenner, Aaron Striegel, and Jesus Izaguirre
@inproceedings{tactical-sc05,author={Thain, Douglas and Klous, Sander and Wozniak, Justin and Brenner, Paul and Striegel, Aaron and Izaguirre, Jesus},title={{Separating Abstractions from Resources in a Tactical Storage System}},booktitle={{IEEE/ACM Supercomputing}},pages={55-67},year={2005},note={{doi: 10.1109/SC.2005.64}},cclpaperid={52},}
Identity Boxing: A New Technique for Consistent Global Identity
@inproceedings{ibox-sc05,author={Thain, Douglas},title={{Identity Boxing: A New Technique for Consistent Global Identity}},booktitle={{IEEE/ACM Supercomputing}},pages={51-61},year={2005},note={{doi: 10.1109/SC.2005.34}},cclpaperid={53},}
Work in Progress: A Literate Approach to Graduate Computer Science Education
@inproceedings{literate-fie05,author={Thain, Douglas and Poellabauer, Christian},title={{Work in Progress: A Literate Approach to Graduate Computer Science Education}},booktitle={{IEEE Frontiers in Education}},pages={1-2},year={2005},note={{doi: 10.1109/FIE.2005.1612087}},cclpaperid={54},}
Sub-Identities: Towards Operating System Support for Distributed System Security
@techreport{subid-tr,author={Snowberger, Philip and thain, Douglas},title={{Sub-Identities: Towards Operating System Support for Distributed System Security}},institution={{University of Notre Dame, Computer Science and Engineering Department}},number={2005-18},year={2005},cclpaperid={84},}
Parrot: An Application Environment for Data-Intensive Computing
@article{parrot-scpe05,author={Thain, Douglas and Livny, Miron},title={{Parrot: An Application Environment for Data-Intensive Computing}},journal={{Scalable Computing: Practice and Experience}},volume={6},number={3},pages={9-18},year={2005},cclpaperid={18},}
Poster: Identity Boxing: Secure User-Level Containment for the Grid
@article{ibox-hpdc05,author={Thain, Douglas},title={{Poster: Identity Boxing: Secure User-Level Containment for the Grid}},booktitle={{IEEE Symposium on High Performance Distributed Computing}},pages={299-300},year={2005},note={{isbn: 10.1109/HPDC.2005.1520984}},cclpaperid={55},}
Generosity and Gluttony in GEMS: Grid Enabled Molecular Simulations
Justin Wozniak, Paul Brenner, Douglas Thain, Aaron Striegel, and Jesus Izaguirre
In IEEE Symposium on High Performance Distributed Computing, 2005
@inproceedings{gems-hpdc05,author={Wozniak, Justin and Brenner, Paul and Thain, Douglas and Striegel, Aaron and Izaguirre, Jesus},title={{Generosity and Gluttony in GEMS: Grid Enabled Molecular Simulations}},booktitle={{IEEE Symposium on High Performance Distributed Computing}},pages={191-200},year={2005},note={{doi: 10.1109/HPDC.2005.1520959}},cclpaperid={56},}
Patisserie: Support for Parameter Sweeps in a Fault-Tolerant, Massively Parallel, Peer-to-Peer Simulation Environment
Timothy Schoenharl, Scott Christley, and Douglas Thain
In Workshop on Agent Directed Simulation (ADS), 2005
@inproceedings{patisserie-ads05,author={Schoenharl, Timothy and Christley, Scott and Thain, Douglas},title={{Patisserie: Support for Parameter Sweeps in a Fault-Tolerant, Massively Parallel, Peer-to-Peer Simulation Environment}},booktitle={{Workshop on Agent Directed Simulation (ADS)}},pages={1-8},year={2005},cclpaperid={57},}
Distributed Computing in Practice: The Condor Experience
Douglas Thain, Todd Tannenbaum, and Miron Livny
Concurrency and Computation: Practice and Experience, 2005
@article{condor-ccpe,author={Thain, Douglas and Tannenbaum, Todd and Livny, Miron},title={{Distributed Computing in Practice: The Condor Experience}},journal={{Concurrency and Computation: Practice and Experience}},volume={17},number={2-4},pages={323-356},year={2005},note={{doi: 10.1002/cpe.v17:2/4}},cclpaperid={19},}
2004
Building Reliable Clients and Servers
Douglas Thain and Miron Livny
In Grid: Blueprint for a New Computing Infrastructure, 2004
@incollection{condor-reliable,author={Thain, Douglas and Livny, Miron},title={{Building Reliable Clients and Servers}},editor={Foster, Ian and Kesselman, Carl},booktitle={{Grid: Blueprint for a New Computing Infrastructure}},publisher={Morgan Kaufmann},year={2004},note={{isbn: 1-55860-933-4}},cclpaperid={10},}
Coordinating Access to Computation and Data in Distributed Systems
@thesis{thain-dissertation,author={Thain, Douglas},title={{Coordinating Access to Computation and Data in Distributed Systems}},editor={Thesis, Ph.D.},booktitle={{University of Wisconsin}},year={2004},cclpaperid={4},}
Explicit Control in a Batch Aware Distributed File System
John Bent, Douglas Thain, Andrea Arpaci-Dusseau, Remzi Arpaci-Dusseau, and Miron Livny
In USENIX Networked Systems Design and Implementation (NSDI), 2004
@inproceedings{badfs-nsdi-04,author={Bent, John and Thain, Douglas and Arpaci-Dusseau, Andrea and Arpaci-Dusseau, Remzi and Livny, Miron},title={{Explicit Control in a Batch Aware Distributed File System}},booktitle={{USENIX Networked Systems Design and Implementation (NSDI)}},pages={365-378},year={2004},cclpaperid={58},}
2003
Condor and the Grid
Douglas Thain, Todd Tannenbaum, and Miron Livny
In Grid Computing: Making the Global Infrastructure a Reality, 2003
@incollection{condor-grid,author={Thain, Douglas and Tannenbaum, Todd and Livny, Miron},title={{Condor and the Grid}},editor={Berman, Fran and Hey, Anthony and Fox, Geoffrey},booktitle={{Grid Computing: Making the Global Infrastructure a Reality}},publisher={John Wiley},year={2003},note={{isbn: 0-470-85319-0}},cclpaperid={9},}
Pipeline and Batch Sharing in Grid Workloads
Douglas Thain, John Bent, Andrea Arpaci-Dusseau, Remzi Arpaci-Dusseau, and Miron Livny
In IEEE High Performance Distributed Computing, 2003
@inproceedings{workload-hpdc12,author={Thain, Douglas and Bent, John and Arpaci-Dusseau, Andrea and Arpaci-Dusseau, Remzi and Livny, Miron},title={{Pipeline and Batch Sharing in Grid Workloads}},booktitle={{IEEE High Performance Distributed Computing}},pages={152-161},year={2003},note={{doi: 10.1109/HPDC.2003.1210025}},cclpaperid={68},}
The Ethernet Approach to Grid Computing
Douglas Thain and Miron Livny
In IEEE High Performance Distributed Computing, 2003
@inproceedings{ethernet-hpdc12,author={Thain, Douglas and Livny, Miron},title={{The Ethernet Approach to Grid Computing}},booktitle={{IEEE High Performance Distributed Computing}},pages={138-},year={2003},note={{doi: 10.1109/HPDC.2003.1210024}},cclpaperid={69},}
Parrot: Transparent User-Level Middleware for Data Intensive Computing
Douglas Thain and Miron Livny
In Workshop on Adaptive Grid Middleware at PACT, 2003
@inproceedings{parrot-agm2003,author={Thain, Douglas and Livny, Miron},title={{Parrot: Transparent User-Level Middleware for Data Intensive Computing}},booktitle={{Workshop on Adaptive Grid Middleware at PACT}},year={2003},cclpaperid={67},}
XtremWeb and Condor: Sharing Resources Between Internet Connected Condor Pools
Oleg Lodygensky, Gilles Fedak, Vincent Neri, Franck Cappello, Miron Livny, and Douglas Thain
In Workshop on Global and Peer-to-Peer Computing on Large Scale Distributed Systems at CCGRID, 2003
@inproceedings{xw-condor,author={Lodygensky, Oleg and Fedak, Gilles and Neri, Vincent and Cappello, Franck and Livny, Miron and Thain, Douglas},title={{XtremWeb and Condor: Sharing Resources Between Internet Connected Condor Pools}},booktitle={{Workshop on Global and Peer-to-Peer Computing on Large Scale Distributed Systems at CCGRID}},pages={382-390},year={2003},note={{doi: 10.1109/CCGRID.2003.1199391 }},cclpaperid={70},}
2002
Error Scope on a Computational Grid
Douglas Thain and Miron Livny
In IEEE High Performance Distributed Computing, 2002
@inproceedings{error-scope,author={Thain, Douglas and Livny, Miron},title={{Error Scope on a Computational Grid}},booktitle={{IEEE High Performance Distributed Computing}},pages={199-208},year={2002},note={{doi: 10.1109/HPDC.2002.1029919}},cclpaperid={71},}
2001
Gathering at the Well: Creating Communities for Grid I/O
Douglas Thain, John Bent, Andrea Arpaci-Dusseau, Remzi Arpaci-Dusseau, and Miron Livny
@inproceedings{gridio-sc01,author={Thain, Douglas and Bent, John and Arpaci-Dusseau, Andrea and Arpaci-Dusseau, Remzi and Livny, Miron},title={{Gathering at the Well: Creating Communities for Grid I/O}},booktitle={{IEEE/ACM Supercomputing}},pages={21-31},year={2001},note={{doi: 10.1109/SC.2001.10023}},cclpaperid={72},}
The Kangaroo Approach to Data Movement on the Grid
Douglas Thain, Jim Basney, Se-Chang Son, and Miron Livny
In IEEE High Performance Distributed Computing, 2001
@inproceedings{kangaroo-hpdc01,author={Thain, Douglas and Basney, Jim and Son, Se-Chang and Livny, Miron},title={{The Kangaroo Approach to Data Movement on the Grid}},booktitle={{IEEE High Performance Distributed Computing}},pages={325-333},year={2001},note={{doi: 10.1109/HPDC.2001.945200}},cclpaperid={73},}
Multiple Bypass: Interposition Agents for Distributed Computing
@article{bypass-jcc,author={Thain, Douglas and Livny, Miron},title={{Multiple Bypass: Interposition Agents for Distributed Computing}},journal={{Journal of Cluster Computing}},volume={4},number={1},pages={39-47},year={2001},note={{doi: 10.1023/A:1011412209850}},cclpaperid={20},}
2000
Bypass: A Tool for Building Split Execution Systems
Douglas Thain and Miron Livny
In IEEE High Performance Distributed Computing, 2000
@inproceedings{bypass-hpdc00,author={Thain, Douglas and Livny, Miron},title={{Bypass: A Tool for Building Split Execution Systems}},booktitle={{IEEE High Performance Distributed Computing}},pages={79-85},year={2000},note={{doi: 10.1109/HPDC.2000.868637}},cclpaperid={74},}
