Data Engineer

Key Responsibilities

• Design, implement, and maintain relational and time-series databases for lab instrument data, environmental measurements, and operational records.
• Develop and manage ETL/ELT pipelines to ingest, transform, and store data from IoT sensors, measurement hardware, and remote sensing platforms.
• Build and deploy internal data access tools and applications using modern frameworks (e.g., Streamlit, FastAPI, React, or similar) to enable lab staff to query, visualize, and export lab data.
• Apply AI-assisted development tools (e.g., GitHub Copilot, Cursor, Claude) to accelerate software delivery while maintaining code quality and auditability appropriate for a laboratory environment.
• Develop and enforce QA/QC protocols to validate incoming data from lab instruments and field sensors in accordance with applicable regulatory and accreditation standards (e.g., ISO 17025 or similar).
• Implement automated checks, flagging routines, statistical validation, and audit trails to detect anomalies, missing data, and calibration drift.
• Maintain defensible data records that satisfy chain-of-custody and traceability requirements.
• Ensure data integrity from acquisition through delivery to downstream consumers.
• Architect and optimize database schemas for performance, scalability, and ease of access.
• Evaluate and recommend appropriate database technologies (SQL, NoSQL, time-series) based on data volume, query patterns, and the lab’s analysis and reporting requirements.
• Partner with lab engineers, metrology staff, and operations to understand data access requirements and translate them into technical solutions.
• Serve as the primary point of contact for the lab’s internal data availability and reporting needs.
• Design and build internal data access tools, dashboards, and reporting interfaces using modern full-stack frameworks (e.g., React, FastAPI, Streamlit, Plotly Dash).
• Leverage AI-assisted development environments (e.g., GitHub Copilot, Cursor, Claude Code) to accelerate development cycles while ensuring maintainability, security, and compliance with lab data governance requirements.
• Enable non-technical lab staff to explore, filter, and export lab datasets through intuitive interfaces without requiring direct database access.
• Maintain comprehensive data dictionaries, schema documentation, and data lineage records consistent with laboratory quality management systems.
• Contribute to laboratory SOPs and data management plans.
• Stay current with emerging data engineering technologies, AI tooling, and laboratory informatics practices to continuously improve the lab’s data infrastructure.

Qualifications and Experience

• Minimum of 3 years of experience in data engineering, database engineering, ETL/ELT pipeline development, or a related technical discipline, preferably in a laboratory, engineering, or renewable energy context.
• Experience designing and operating production data pipelines and infrastructure is required.
• Experience in photovoltaic (PV) testing, solar energy measurement, or a physical laboratory environment is highly preferred.
• Proficiency in SQL and experience with relational databases (PostgreSQL, MySQL, or similar); familiarity with time-series or NoSQL databases a plus.
• Proficiency in Python (pandas, SQLAlchemy, FastAPI, or similar) for data engineering, scripting, and backend service development.
• Hands-on experience designing and operating ETL/ELT data pipelines and workflow orchestration tools (e.g., Apache Airflow, Dagster, Prefect, or similar), including scheduling, dependency management, and pipeline monitoring.
• Experience building web applications or data dashboards using tools such as Streamlit, Dash, FastAPI, React, or modern AI-assisted development environments (e.g., GitHub Copilot, Cursor, Claude Code); ability to deliver functional, user-facing tools rapidly using AI pair-programming workflows.
• Experience implementing QA/QC workflows for instrument or sensor data, including anomaly detection, validation rules, statistical flagging, and audit logging; familiarity with laboratory quality management standards (e.g., ISO 17025, GLP, or similar regulatory frameworks) is a strong plus.
• Excellent communication skills; ability to translate complex technical data concepts for non-technical stakeholders including lab engineers and business analysts.
• Familiarity with version control (Git), CI/CD practices, and cloud data platforms (AWS, Azure, or GCP); experience with containerization (Docker) is a plus.
• Demonstrated experience using AI-assisted development tools (e.g., GitHub Copilot, Cursor, Claude Code, or similar) to write, debug, and refactor code; comfort evaluating AI-generated outputs for correctness, security, and suitability in a regulated laboratory data environment.
• Understanding of laboratory informatics concepts and data management in accredited or regulated settings; experience with LIMS (Laboratory Information Management Systems) or similar platforms is a plus.