AI DATA ENGINEER
SIDRAM TECHNOLOGIES · New York, NY · 1 mo ago
On-siteInformation TechnologyFull-time
Client: Altimetrik / S&P Global • Location: Princeton, NJ & NYC, NY • Employment: Full‑Time Hybrid • Experience: 15+ years • Work Authorization: Independent Visa required
Responsibilities
- Design and implement data pipelines for AI model metadata, training data lineage, and model performance tracking.
- Build large-scale data infrastructure using Databricks and Apache Spark.
- Develop MCP servers and enable AI data distribution through MCP.
- Develop feature engineering and data preprocessing pipelines for AI model training and inference.
- Implement model versioning, experiment tracking, and model registry using MLflow or similar platforms.
- Build automated workflows for AI agent discovery, classification, and inventory management.
- Design knowledge graph structures representing AI model relationships, dependencies, and data lineage.
- Build real-time pipelines for AI model monitoring, drift detection, and performance tracking.
- Develop data quality frameworks for AI training and validation datasets.
- Collaborate with data scientists and ML engineers to optimize data access patterns and feature stores.
- Implement security and compliance controls for sensitive AI training data and model artifacts.
- Create documentation for AI data architectures, schemas, and integration patterns.
Qualifications
- 15+ years of overall professional experience.
- Bachelor s or Master s degree in Computer Science, Data Science, Machine Learning, or related field.
- Strong hands-on experience in data engineering and AI/ML workloads.
- Expert proficiency in Python.
- Strong experience with Databricks, Apache Spark, and distributed computing.
- Experience with ML frameworks such as PyTorch, TensorFlow, or scikit-learn.
- Strong understanding of the machine learning lifecycle including training, deployment, and monitoring.
- Experience with feature engineering, data preprocessing, and ML data pipelines.
- Knowledge of vector databases, embeddings, and similarity search.
- Strong SQL skills.
- Understanding of data governance, model governance, and AI security.