Research Scientist, Distributed Workflows

About the role

Oak Ridge National Laboratory (ORNL) is seeking a Research Scientist to advance distributed workflow orchestration and cross-facility integration in support of the Genesis Mission, the American Science Cloud, and the Oak Ridge Leadership Computing Facility (OLCF).

Responsibilities

• Distributed Workflow Orchestration: Architect, implement, and operate cross-facility workflow orchestration systems that integrate job schedulers (e.g., SLURM), data movement services (e.g., Globus), and computing endpoints across multiple DOE facilities.
• Design execution patterns that support autonomous experimentation, multi-facility data pipelines, and event-driven automation.
• API and Service Mesh Development: Design and implement RESTful APIs and service meshes that expose OLCF computing resources to diverse scientific workflows and external facilities. Define API specifications, manage versioning, and guide systems from prototype through production deployment.
• Genesis Mission / American Science Cloud Integration: Lead ORNL’s technical engagement with the Genesis Mission, architecting cross-facility workflow and data infrastructure that connects leadership computing, experimental facilities, and cloud platforms in support of DOE’s vision for integrated science. Drive contributions to the American Science Cloud (AmSC) by designing and deploying interoperable APIs, data movement services, and orchestration layers that enable seamless scientific workflows across the DOE complex. Coordinate with partner laboratories and facilities (e.g., ANL, NERSC) to align interfaces and standards, and establish ORNL/OLCF as a foundational compute and data hub within the emerging national science infrastructure.
• Agentic Workflow Design and Autonomous Control Systems: Design and implement agentic workflow architectures that enable autonomous, closed-loop scientific experimentation across HPC and instrument facilities. Develop LLM-integrated orchestration layers, tool-use pipelines, and event-driven control systems that allow AI agents to plan, execute, monitor, and adapt multi-step scientific workflows with minimal human intervention. Apply agentic patterns to use cases such as autonomous materials discovery, self-steering simulations, and adaptive experimental campaigns, in coordination with domain scientists and facility operators.
• Scientific Domain Support: Collaborate with domain scientists in areas such as climate science, materials science, and autonomous experimentation to translate research objectives into scalable, automated workflow solutions. Provide documentation, training, and direct user support.
• Scripting and Tooling: Develop command-line tools and automation in Python, Bash, and/or C/C++ to encapsulate workflow steps, manage configuration files (e.g., YAML/JSON), and implement robust logging, error handling, and checkpoint/retry strategies.
• Operational Reliability and Optimization: Diagnose job failures, mitigate bottlenecks, and improve throughput, latency, and resource utilization. Use scheduler and Linux tools (e.g., sacct, squeue, coreutils, ssh, tmux, top, iostat) to monitor, analyze, and tune workflows.
• Research and Publication: Conduct original research on workflow systems, orchestration, and data management. Publish findings in high-impact, peer-reviewed venues and present work at major conferences (e.g., SC, HPDC, PEARC, eScience).

Qualifications

• D. in Computer Science, Computer Engineering, or a closely related field.
• At least 2 years of experience designing or operating distributed systems, APIs, or workflow orchestration frameworks.
• At least 1 year of programming experience in Python, with working familiarity in Bash and/or a compiled language (C/C++ or similar).
• Experience with Linux-based HPC environments and familiarity with job scheduling systems (e.g., SLURM or PBS).
• Strong verbal and written communication skills, with the ability to collaborate across technical and scientific teams.

Preferred Qualifications

• Demonstrated experience publishing research in high-impact venues such as SC, IPDPS, HPDC, PEARC, or IEEE eScience.
• Familiarity with RESTful API design, containerization (Docker/Singularity/Apptainer), and microservice architectures.
• Experience working in or coordinating across DOE national laboratory environments, particularly in the context of integrated or cross-facility research infrastructure.
• Background in scientific data management, provenance tracking, or metadata systems.
• Demonstrated ability to lead technical projects, manage milestones, and coordinate multidisciplinary teams.
• Familiarity with basic cyber-security principles (e.g., SSH key hygiene, least privilege, network segmentation) as applied to workflow and API design.