Senior Data Engineer
General Motors · Austin, TX · 4 wk ago
HybridInformation Technology$133k–$189k/yrFull-time
Responsibilities
- Design and develop batch and streaming ETL/ELT pipelines in Azure Databricks to support ingestion, transformation, curation, and delivery of manufacturing and enterprise data.
- Build reusable Bronze, Silver, and Gold data pipelines, data quality frameworks, and governed data products in Unity Catalog.
- Build and operationalize ML workflows for data preparation, feature engineering, model training, evaluation, deployment, batch scoring, retraining, and inference.
- Establish and maintain MLflow‑based model lifecycle controls, including experiment tracking, model registration, versioning, lineage, traceability, and controlled promotion across environments.
- Implement CI/CD pipelines for data products, ML code, workflows, and deployment bundles using Git‑based development practices, automated testing, and release automation.
- Build and support Databricks jobs and workflows for orchestration, scheduled execution, retraining, scoring, and downstream operational consumption.
- Improve reliability, observability, and production readiness through testing, monitoring, validation, alerting, documentation, and automation that reduce deployment risk and manual intervention.
- Partner with data scientists, data engineers, platform teams, architecture, manufacturing, and business stakeholders to productionize scalable, supportable solutions aligned to production constraints.
- Support modernization of legacy data and ML workflows into standardized cloud‑native Databricks patterns and help bridge proof‑of‑concept work into repeatable production solutions.
Required Qualifications
- 5+ years of relevant full‑time experience or equivalent knowledge.
- Bachelor’s degree in Computer Science, Data Science, Engineering, or a related technical field, or equivalent practical experience.
- Strong proficiency in Scala, Python, SQL, Spark, and Databricks, with production‑quality software engineering practices.
- Experience building and supporting cloud‑based ETL/ELT pipelines, data platforms, and pipeline orchestration patterns for both batch and streaming operations.
- Experience deploying machine learning models into production environments and supporting model development through operational release.
- Strong experience with Databricks jobs, workflows, repos, Unity Catalog, and MLflow‑based experimentation and model lifecycle management.
- Experience building CI/CD pipelines with Git‑based workflows, automated testing, and deployment of data or ML products.
- Strong troubleshooting, communication, and cross‑functional collaboration skills.
Preferred Qualifications
- Experience with Delta Live Tables (DLT), structured streaming, Event Hub, Kafka, Scala, or Java in enterprise data pipeline environments.
- Experience with model governance, lineage, reproducibility, monitoring, validation checks, and operational observability.
- Experience designing resilient pipelines that can handle changing upstream data conditions, retraining needs, and production operations.
- Experience with manufacturing, industrial IoT, plant‑floor data, quality, or manufacturing analytics use cases.
- Experience helping transition proof‑of‑concept or R&D solutions into scalable production‑ready patterns.
- Eligibility for relocation benefits.
Compensation
The expected base compensation for this role is $133,000 – $188,600. Actual base compensation within this range will vary based on factors relevant to the position. An incentive pay program offers bonus payouts based on company performance, job level, and individual performance.
Benefits
- Medical, dental, and vision coverage
- Health Savings Account (HSA) and Flexible Spending Accounts (FSA)
- Retirement savings plan
- Sickness and accident benefits
- Life insurance
- Paid vacation and holidays
- Tuition assistance programs
- Employee assistance program
- General Motors vehicle discounts
- Relocation assistance (where applicable)
Schedule
This is a hybrid role. The successful candidate is expected to report to the Austin Technical Center three times per week, or at a frequency dictated by business needs if more than three days are required.
General Motors does not provide immigration‑related sponsorship for this role. Candidates requiring sponsorship now or in the future should not apply.