Data Infrastructure & ML Engineer (Hybrid Role)

Data Pipeline Engineering & Data Flow

Design and build end-to-end data pipelines (ETL/ELT) for ingesting, processing, and transforming data.

Handle multiple data sources including: Tool-generated logs (e.g., AT log files), JSON and semi-structured data.

Ensure full data traceability, enabling backward tracking of all data points.

Implement validation, monitoring, and error handling to ensure data quality and reliability.

Python & Machine Learning Data Processing

Develop data processing workflows using Python.

Work extensively with dataframes for transformation and analysis.

• Utilize libraries such as: Pandas, NumPy for data manipulation.
• Plotly (or similar) for visualization and exploratory analysis.
• Automate data workflows and integrate them into pipelines.

Database Design & Data Architecture

Design and manage scalable database schemas.

Support both single-node and distributed database environments.

• Implement tablespaces, partitioning, and sharding strategies to ensure performance and scalability.
• Optimize queries and maintain high performance for large-scale datasets.

Machine Learning Data Enablement

Prepare and transform datasets for machine learning models.

Collaborate with data scientists and engineers to support model training and deployment workflows.

Enable scalable data foundations for AI/ML integration into production systems.

Required Qualifications

• Bachelor’s or Master’s degree in Computer Science, Engineering, or related field with 5+ years of experience.
• Strong experience in database design and SQL-based systems.
• Hands-on experience with distributed systems, partitioning, and sharding.
• Proven experience building data pipelines (ETL/ELT).
• Strong proficiency in Python for data processing.
• Experience working with log-based and semi-structured data (e.g., JSON).
• Understanding of data traceability, validation, and governance.

Preferred Qualifications

• Experience with time-series or log analytics systems.
• Exposure to real-time/streaming architectures (e.g., Kafka).
• Experience with cloud platforms (Azure, AWS, or GCP).
• Familiarity with machine learning workflows and lifecycle.
• Domain experience in semiconductor or high-throughput systems (nice to have).

Key Competencies

• Strong problem-solving and analytical skills.
• Ability to design production-grade, scalable systems.
• Focus on data integrity, performance, and reliability.
• Effective collaboration across engineering and data teams.
• Clear communication and documentation.

PAY

$122,133.07 - $183,199.61