Principal Data Engineer

About the Role

The Principal Data Engineer (IC) is a senior individual contributor and the accountable technical leader for assigned cross-domain initiatives and enterprise data engineering capabilities. This role owns integrated technical direction and outcomes for work spanning multiple data domains, defines and stewards enterprise engineering standards and reference architectures, and drives convergence where duplicated or inconsistent solutions create enterprise cost, risk, or operational burden. The position advises on scope, sequencing, capacity, dependencies, and technical debt but does not independently commit domain resources or business delivery dates. This role is based in Madison, WI, and has no people-management responsibility.

Responsibilities

• Own the technical outcome of assigned cross-domain initiatives from initial ambiguity through production and lifecycle support.
• Convert enterprise needs into integrated architecture and executable technical plans, decomposing complex work into deliverable increments across domains.
• Coordinate technical execution across domain Staff Engineers and partner teams, identifying dependencies and architectural risks, and escalating decisions requiring business or delivery authority.
• Remain hands-on through prototypes, reference implementations, critical-path development, technical validation, and production problem solving.
• Define, steward, and evolve enterprise data engineering standards, patterns, and reference architectures.
• Lead architecture for assigned enterprise capabilities, including semantic and metrics layers, canonical data models, batch, API, event-driven, and streaming patterns, and governed data products.
• Establish enterprise data-contract standards covering schemas, service expectations, compatibility, breaking-change policy, and producer-consumer responsibilities.
• Lead enterprise design and code reviews for high-complexity or cross-domain work, and coach and mentor Staff and Senior Engineers.
• Communicate architecture and technical tradeoffs clearly to engineering, business, and executive stakeholders.
• Evaluate architecture tradeoffs across reliability, scalability, performance, security, privacy, operability, adoption, and technical cost.
• Serve as the enterprise technical escalation for incidents involving multiple domains or shared architecture patterns.
• Design and review architectures handling protected health information to meet applicable security, privacy, and regulatory requirements.
• Provide technical direction and due diligence for vendor and external-partner solutions.
• Ability to work nights and/or weekends, as needed.

Qualifications

• Bachelor's Degree in Data Science, Computer Science, Information Systems, Mathematics, or Engineering.
• Expert-level experience with software development design and relevant domain-specific skills:
• Spark on Databricks or comparable platforms; Python, Scala, SQL, and Snowflake experience.
• ETL and ELT data pipelines, including batch and event-driven patterns.
• Designing and implementing data modeling solutions using relational, dimensional, and/or NoSQL databases.
• Database architecture testing methodology, including execution of test plans, debugging, and testing scripts and tools.
• Open data file and table formats (Parquet, Avro, Delta Lake); cloud infrastructure and delivery services (AWS, S3, SQS, and GitLab CI/CD).
• REST API development; familiarity with BI concepts and Tableau performance considerations.
• Agile development tools, including JIRA and Confluence.
• Demonstrated ability to lead through influence across multiple teams and communicate complex technical decisions to senior stakeholders.

Preferred Qualifications

• Experience with Databricks, Apache Spark, Delta Lake, and Unity Catalog at enterprise scale.
• Kafka, change data capture, and production event-streaming architectures.
• Semantic or metrics layer design, canonical data models, and governed data products.
• Cloud data architecture in AWS, Azure, or Google Cloud Platform.
• Experience in life sciences, diagnostics, or clinical laboratory environments involving protected health information and regulatory requirements (HIPAA, CLIA, FDA, etc.).
• Technical assessment and architecture direction for vendor and external-partner platforms.

Pay

The base pay for this position is $129,300.00 – $258,700.00. In specific locations, the pay range may vary from the range posted.