US_East | Data Engineer_L4

Key Responsibilities

• Design and own Azure-based data platform architecture, including: Azure Data Lake Storage (ADLS Gen2), Azure Databricks / Synapse Analytics, Azure Data Factory and event-driven services.
• Align architecture to enterprise cloud, security, and governance standards.
• Architect integration of lab systems including: LIMS (Laboratory Information Management Systems), SDMS (Scientific Data Management Systems), ELN and instrument data sources.
• Define ingestion patterns for: Structured laboratory data (samples, results, metadata), Unstructured scientific data (instrument files, reports, raw datasets).
• Address challenges such as: Instrument data variability and formats, Vendor system constraints, Data synchronization across lab workflows.
• Define and govern data models for lab entities, including: Methods, samples, experiments, results, instruments, and documents.
• Align LIMS/SDMS data structures to canonical enterprise data models.
• Enable end-to-end traceability and lineage (digital thread).
• Define: Data classification and sensitivity, Ownership and stewardship for lab datasets.
• Ensure compliance with: GxP / FDA regulations, Auditability and traceability requirements.
• Define integration patterns for LIMS/SDMS: APIs, batch ingestion, file-based transfers, and streaming.
• Enable digital thread across lab processes, linking: Samples → experiments → results → reports → downstream analytics.
• Ensure consistent identifiers and cross-system linkage.
• Enable analytics via: Power BI semantic models, Data science pipelines on Databricks / Azure ML.
• Support use cases such as: Quality reporting, Regulatory submissions, Advanced analytics (predictive insights, AI).
• Define Azure architecture for: Scalability and performance for large instrument datasets, High availability and disaster recovery, Monitoring (Azure Monitor, Log Analytics), Optimize cost and performance for data-intensive workloads.
• Define CI/CD pipelines using: Azure DevOps / GitHub.
• Enable automated deployment for: Data pipelines, Data models, Infrastructure.

Mandatory Skills

• Hands-on experience working with: LIMS platforms (LabWare, STARLIMS, Thermo Fisher, etc.), SDMS solutions (BIOVIA, Waters NuGenesis, LabVantage, etc.).
• Strong understanding of: Laboratory workflows (samples, methods, results), Instrument data capture and file formats, Scientific data lifecycle management.
• Experience handling: Integration of LIMS/SDMS with enterprise data platforms, Challenges of structured vs unstructured lab data, Regulatory and audit requirements for lab environments.
• Data ingestion from lab systems and instruments, File-based workflows (CSV, XML, proprietary instrument formats), Metadata extraction from SDMS platforms, Handling high-volume, high-granularity scientific datasets.
• Governance & Data Management: Microsoft Purview, lineage, cataloging, Data quality, stewardship, and SoR frameworks, Regulatory compliance (GxP, audit, traceability).
• Integration & Digital Thread: API, batch, event-driven architectures, Cross-system identifier strategy.