AI Data Engineer
Shrive Technologies · Tennessee, United States · 4 days ago
Information TechnologyFull-time
Key Responsibilities
- Design, develop, and maintain scalable ETL/ELT pipelines for structured and unstructured data.
- Build and optimize data lakes, data warehouses, and AI-ready data platforms.
- Develop ingestion, transformation, and orchestration frameworks using cloud-native technologies.
- Prepare, cleanse, and engineer datasets for AI/ML and Generative AI workloads.
- Integrate Large Language Models (LLMs), vector databases, embeddings, and RAG (Retrieval-Augmented Generation) pipelines into enterprise solutions.
- Implement data governance, security, lineage, and quality controls.
- Collaborate with Data Scientists, AI Engineers, Business Analysts, and Solution Architects.
- Monitor, troubleshoot, and optimize data pipelines and platform performance.
- Create technical documentation and data dictionaries for enterprise data assets.
Required Skills
- Technical Skills:
- Python, SQL, PySpark
- Data Modeling (Star Schema, Snowflake Schema)
- Apache Spark, Databricks
- Airflow, Dataflow, Informatica, ADF, Synapse, or equivalent tools
- Relational & NoSQL Databases
- Data Warehousing concepts
- REST APIs and Microservices
- Git, CI/CD, DevOps practices
- AI & GenAI Skills:
- Machine Learning fundamentals
- Data preparation for AI models
- Vector Databases (Pinecone, ChromaDB, FAISS)
- LLM Integration (OpenAI, Azure OpenAI, Gemini, Claude, etc.)
- RAG Architecture
- Embeddings and Semantic Search
- Prompt Engineering fundamentals