Senior Software Engineer- Big Data & MCP, Data Foundations

Job Summary

Essential Functions:

• Collaborate and Innovate: Partner with product managers, data engineers, and business leaders to translate complex product and data requirements into scalable, reliable data pipelines and the search experiences they power.
• Architect Data Pipelines: Design, build, and optimize large-scale distributed batch and streaming pipelines (using Apache Airflow, Apache Beam/Dataflow, and BigQuery) to ingest, model, and transform high-volume healthcare data into clean, well-tested, query-ready datasets and search indices.
• Build Data Models & Backend Services: Develop resilient Python services and DBT models that power data delivery and self-service analytics, including Model Context Protocol (MCP) servers that expose curated data and tooling to downstream and AI consumers, and integrate with external REST/SOAP APIs and third-party data sources.
• Optimize Data & Search Performance: Deeply tune pipeline throughput, data warehouse performance, and search indexing — optimizing BigQuery cost and query performance and Elasticsearch index design to ensure data freshness, relevance, and scalability across high-volume datasets.
• Drive Engineering Excellence: Write clean, maintainable, well-tested code and lead by example through rigorous code reviews, architectural and data-modeling design discussions, and mentoring, driving a culture of high-quality software and trustworthy data.
• Pioneer New Technologies: Stay at the forefront of modern data engineering, the analytics-engineering ecosystem (e.g., DBT, BigQuery), and information retrieval, proactively applying these advancements to strengthen our data platform and the products it powers.

Minimum Requirements

• Data Engineering: Proven experience designing and orchestrating large-scale ETL/ELT pipelines using Apache Beam/Google Cloud Dataflow (or similar), and DBT, built on modern cloud data warehouses. BigQuery experience is a plus.
• Databases & SQL: 4+ years of experience working with relational databases and analytical data warehouses, with deep, advanced SQL skills and solid data-modeling fundamentals (e.g., dimensional and normalized modeling).
• Search & Indexing: Working experience with search indexing and Elasticsearch, including index management, mappings, and building and maintaining search indices from pipeline output. Familiarity with hybrid (BM25 + semantic/vector) search is a plus.
• Backend & Data Services: Experience building scalable Python services and high-performance data APIs, including developing Model Context Protocol (MCP) servers that expose data and tooling to downstream and AI consumers.
• Infrastructure & DevOps: Strong understanding of containerization (Docker), CI/CD methodologies (e.g., GitHub Actions), Git, Infrastructure as Code (e.g., Terraform/Pulumi), and managing services within cloud platforms (3+ years of GCP experience preferred).
• Familiarity with healthcare data standards (e.g., NPPES/NPI registries, NUCC Provider Taxonomy, machine-readable files (MRFs) for cost transparency, and FHIR).
• Experience with data quality and pipeline testing frameworks (e.g., dbt tests, Great Expectations) and streaming/event ingestion (e.g., Pub/Sub, Kafka).
• Experience integrating graph-based data and healthcare taxonomy ontologies to enrich datasets and search query context.
• Experience with observability and logging platforms (e.g., DataDog) for monitoring pipeline health and data freshness.

Education

• Bachelor’s Degree

Experience

• 5+ years of professional experience with Python, with strong software-engineering fundamentals (testing, code review, design).
• 3+ years experience with Java or another JVM language is also highly desired, particularly for Beam/Dataflow.

Supervision

• N/A

Certifications

• N/A

Language Skills

• Ability to read, analyze and interpret general business periodicals, professional journals, technical procedures or governmental regulations.
• Ability to write reports, business correspondence and procedure manuals.
• Ability to effectively present information and respond to questions from a variety of both internal and external sources.

Physical Capabilities

• Sit.
• Use hands to finger, handle, or feel.
• Reach with hands and arms.
• Talk or hear.
• Occasionally lift and/or move up to 10 pounds.