Software Engineer - 6 month+ On-Site (ML)

Overview

We are partnering with a leading autonomous vehicle company seeking a Software Engineer with strong data processing and pipeline engineering experience to help build and scale ScenarioScout, a platform that enables teams to discover and analyze driving scenarios across the AV development lifecycle.

Key Responsibilities

• Data Pipelines (Primary Focus)
• Design and maintain Airflow workflows for end-to-end data pipelines
• Build distributed processing jobs (Ray/Spark) for embedding generation, clustering, FAISS indexing, and dataset analytics
• Optimize performance and reliability across large datasets in AWS (S3)
• Backend & Data Systems
• Develop Python/FastAPI services for search and dataset management
• Own dataset lifecycle (creation, refresh, clustering, registry)
• Improve processing of large-scale Parquet datasets
• Frontend & UX
• Support React-based tools for dataset management, pipeline monitoring, and search
• Enhance user experience (search performance, deduplication, result grouping)
• Collaboration
• Work closely with ML, data, and product teams to deliver scalable solutions

Required Qualifications

• 3+ years of software engineering experience (data pipelines preferred)
• Strong Python + experience with backend frameworks (FastAPI or similar)
• Hands-on experience with distributed data processing (Ray, Spark, Databricks, etc.)
• Experience with Airflow or similar orchestration tools
• AWS experience (S3, EC2)
• SQL proficiency and strong data fundamentals
• Familiarity with vector search tools (FAISS, Milvus, Annoy)
• Exposure to React or modern frontend frameworks

Nice to Have

• ML knowledge (embeddings, clustering, similarity search)
• LLM or RAG systems experience
• ScenarioScout experience
• Observability tools (Datadog, Prometheus, Grafana)
• Autonomous vehicle, robotics, or telemetry data experience
• Full-stack or CI/CD experience

On-Site Requirements

5 days per week

Pay Range

$90-$95.79/hour (W2)

Benefits

• Creative Circle's Freelance Employee benefits package includes eligibility for Minimum Essential Coverage (MEC) medical plan, dental/vision/term life package, discount prescription program, critical illness, accident, tele-behavioral health, and 401(k) plan.
• Sick leave is provided to Candidates whose assignment work location is in a state or city subject to sick leave laws.
• A Minimum Value (MV) PPO medical plan, Employee Stock Purchase Plan, and paid holiday eligibility are based on length and dates of service.