Agentic AI Data Engineer - CMC Data Integration

Key Responsibilities

• Agentic Pipeline Components: Implement individual agent components (e.g., document extraction agent, schema mapping agent, validation agent) within the established orchestration framework (LangGraph, LlamaIndex, or equivalent)
• Write tool-calling logic, handle failure modes, and ensure each agent component is testable and observable with instrumented logging of inputs, outputs, and intermediate decisions
• Iterate on agent behavior based on real data performance; work with the senior engineer to identify and resolve failure patterns
• Participate in validation and qualification activities for AI-assisted workflows, supporting documentation that demonstrates computational tools reflect scientific intent
• Human-in-the-Loop (HITL) Workflow Implementation: Build review queues and flagging logic that surface low-confidence or out-of-specification extractions to scientific reviewers for approval before data is loaded
• Implement routing logic that captures reviewer decisions, logs outcomes with full audit trail, and reintegrates approved data into the pipeline per 21 CFR Part 11 electronic records requirements
• Tune flagging thresholds based on feedback from scientific owners; maintain and improve HITL logic as new data sources are onboarded
• Data Ingestion & Pipeline Engineering: Design and build AI-assisted ingestion pipelines that extract and structure the data from unstructured CDMO/CRO data sources: PDFs (Certificates of Analysis, batch records), Excel files, and vendor portal exports
• Implement validation, reconciliation, and exception-handling logic to ensure data completeness and integrity before loading
• Build monitoring and alerting for pipeline health, data quality, and ingestion failures
• Design a data quality framework with automated checks, rejection handling, and audit trail logging
• Develop reusable pipeline templates and schema documentation that reduce onboarding time for new CDMO partners

Required Qualifications

• MS in Computer Science, Computer Engineering, Data Engineering, or related technical field with 1–2 years of relevant experience; OR BS in Computer Science or Computer Engineering with 3–5 years of hands-on data engineering experience
• Proficiency in Python and SQL; ability to write, review, and own production-quality code
• Demonstrated experience building ETL/ELT pipelines from unstructured or semi-structured sources (PDFs, Excel, JSON, XML)
• Hands-on experience building LLM-powered applications: retrieval-augmented generation, tool-calling, multi-step orchestration, or equivalent agentic patterns
• Hands-on experience with cloud data platforms: Azure (Data Factory, Databricks, Fabric) or AWS (S3, Glue, Lambda, Redshift)
• Solid understanding of relational data modeling, schema design, and data normalization principles
• Familiarity with data orchestration tools (Airflow, Azure Data Factory, Prefect, or similar)

Additional Preferences

• Working knowledge of 21 CFR Part 11, ALCOA+, and GxP data integrity principles, or clear demonstrated ability to apply similar audit/compliance frameworks
• Experience integrating data from LIMS, ELN, SDMS, or CDS systems (Benchling, LabVantage, OpenLABS, or equivalent)
• Familiarity with pharmaceutical CMC data types: analytical results, batch records, stability studies, specifications
• Experience with data mesh architecture or data product ownership models
• Knowledge of MLOps practices and preparing data for AI/ML model training in regulated environments
• Exposure to regulatory submission data formats (eCTD, CTD, CDISC SEND/SDTM)
• Experience with CI/CD pipelines (GitHub Actions, Azure DevOps) applied to data engineering workloads