Agentic AI Data Engineer - CMC Data Integration

Key Responsibilities

• Agentic Pipeline Components: Implement individual agent components (e.g., document extraction agent, schema mapping agent, validation agent) within the established orchestration framework (LangGraph, LlamaIndex, or equivalent).
  • Write tool-calling logic, handle failure modes, and ensure each agent component is testable and observable with instrumented logging of inputs, outputs, and intermediate decisions.
  • Iterate on agent behavior based on real data performance; work with the senior engineer to identify and resolve failure patterns.
  • Participate in validation and qualification activities for AI-assisted workflows, supporting documentation that demonstrates computational tools reflect scientific intent.
• Human-in-the-Loop (HITL) Workflow Implementation:
  • Build review queues and flagging logic that surface low-confidence or out-of-specification extractions to scientific reviewers for approval before data is loaded.
  • Implement routing logic that captures reviewer decisions, logs outcomes with full audit trail, and reintegrates approved data into the pipeline per 21 CFR Part 11 electronic records requirements.
  • Tune flagging thresholds based on feedback from scientific owners; maintain and improve HITL logic as new data sources are onboarded.
• Data Ingestion & Pipeline Engineering:
  • Design and build AI-assisted ingestion pipelines that extract and structure the data from unstructured CDMO/CRO data sources: PDFs (Certificates of Analysis, batch records), Excel files, and vendor portal exports.
  • Implement validation, reconciliation, and exception-handling logic to ensure data completeness and integrity before loading.
  • Build monitoring and alerting for pipeline health, data quality, and ingestion failures.
  • Design a data quality framework with automated checks, rejection handling, and audit trail logging.
  • Develop reusable pipeline templates and schema documentation that reduce onboarding time for new CDMO partners.

Required Qualifications

• MS or PhD in Computer Science, Computer Engineering, Data Engineering, or related technical field with 1–2 years of relevant experience; ORBS in Computer Science or Computer Engineering with 3–5 years of hands-on data engineering experience.
• Proficiency in Python and SQL; ability to write, review, and own production-quality code.
• Demonstrated experience building ETL/ELT pipelines from unstructured or semi-structured sources (PDFs, Excel, JSON, XML).
• Hands-on experience building LLM-powered applications: retrieval-augmented generation, tool-calling, multi-step orchestration, or equivalent agentic patterns.
• Hands-on experience with cloud data platforms: Azure (Data Factory, Databricks, Fabric) or AWS (S3, Glue, Lambda, Redshift).
• Solid understanding of relational data modeling, schema design, and data normalization principles.
• Familiarity with data orchestration tools (Airflow, Azure Data Factory, Prefect, or similar).

Additional Preferences

• Working knowledge of 21 CFR Part 11, ALCOA+, and GxP data integrity principles, or clear demonstrated ability to apply similar audit/compliance frameworks.
• Experience integrating data from LIMS, ELN, SDMS, or CDS systems (Benchling, LabVantage, OpenLABS, or equivalent).
• Familiarity with pharmaceutical CMC data types: analytical results, batch records, stability studies, specifications.
• Experience with data mesh architecture or data product ownership models.
• Knowledge of MLOps practices and preparing data for AI/ML model training in regulated environments.
• Exposure to regulatory submission data formats (eCTD, CTD, CDISC SEND/SDTM).
• Experience with CI/CD pipelines (GitHub Actions, Azure DevOps) applied to data engineering workloads.