Scientific Systems Software Developer
About the role
The Application and User Services (AUS) group at SLAC National Accelerator Laboratory is seeking an energetic, forward-thinking software engineer to develop tools and workflows for science projects such as the Vera C. Rubin Observatory US Data Facility, hosted at SLAC. You will work with a team that supports multiple global, open-science collaborations, leveraging cutting-edge, best-in-class computing, platform, and data services.
Responsibilities
Design, implement, and maintain applications and APIs for S3DF science programs, enabling scientists worldwide to access, process, and analyze large-scale experimental and observational datasets.
Containerize applications and deploy them on Kubernetes clusters, following modern cloud-native best practices.
Contribute to software tools for scientific data management and data processing, with assignments varying according to experiment priorities and lifecycles.
Write clean, well-tested, well-documented code; contribute to and maintain shared software repositories across S3DF science programs.
Contribute to the integration and reconciliation of the Rubin data access layer with distributed data management systems (including Rucio), ensuring consistent views of datasets across systems.
Develop and support workflows for exporting managed data subsets of data for downstream science, collaborators, and community data releases.
Help rationalize and consolidate the landscape of backing databases, improving consistency and reducing operational complexity.
Collaborate with Rubin data management teams across institutions to align on standards and tooling.
Support scientists and users in their day-to-day use of Data Facility services, diagnosing issues, debugging problems, and providing timely, clear resolutions.
Monitor the health and performance of deployed services and supporting infrastructure; gather metrics and produce reports.
Identify and resolve bottlenecks in data-intensive workflows, from ingestion and processing through serving and analysis.
Participate in on-call rotation and incident response for production services.
Develop and maintain runbooks, operational documentation, and user-facing guides.
Use the incident management system to track problems to resolution in a timely manner.
Collaborate with platform engineers to deploy and tune services on Kubernetes and HPC systems.
Contribute to CI/CD pipelines, GitOps workflows, and infrastructure-as-code tooling for Rubin services.
Collaborate with the broader scientific software community (including teams at partner institutions) to align on interfaces, standards, and shared components.
Evaluate and test emerging technologies and technical developments.
Provide feedback and concrete recommendations for service improvements to the AUS team and scientific stakeholders.
Qualifications
Bachelor's degree in physics, computer science, or a related field, and 5 years of relevant experience in software development, systems administration, or scientific/high-performance computing or an equivalent combination of education and experience.
Proficiency in one or more programming languages; Python strongly preferred, with C/C++ or JavaScript a plus.
Experience with modern software development practices: version control (Git), CI/CD pipelines, code review, and agile/scrum methodologies.
Hands-on experience with PostgreSQL (schema design, query optimization, and operational management).
Experience with Apache Kafka or similar event streaming platforms for high-throughput, real-time data pipelines.
Practical experience deploying and operating applications on Kubernetes in a production environments at scale.
Familiarity with distributed compute and storage systems, high-performance computing, and networking concepts.
Understanding of system monitoring, benchmarking, and performance analysis.
Strong organizational and communication skills; ability to work effectively in a collaborative, distributed team environment.
Ability and genuine willingness to learn, adopt best practices, and grow technical skills on the job.