Data Platform Engineer

Position Summary

We are seeking a skilled Data Platform Engineer with a proven track record of innovation to design, build, and maintain robust data systems that power person and business search and verification services.

What You’ll Be Doing

• Build, optimize, and maintain data ingestion and transformation pipelines from multiple sources (internal systems, vendor data, web data, APIs).
• Design and implement data models using the most suitable tool for the task — SQL, NoSQL, GraphDBs, or VectorDBs.
• Integrate machine learning models into pipelines for entity resolution, de-duplication, semantic enrichment, and embedding generation.
• Work with Vector Databases (e.g., AWS S3 Vector, PostgresVectorDb, OpenSearch) to support similarity and semantic search applications.
• Collaborate with data scientists, software engineers, and analysts to deliver reliable, high-performance data infrastructure.
• Ensure data quality, consistency, and performance monitoring across all pipelines and systems.

What You’ll Bring

• 5+ years of professional software development or data engineering experience.
• Strong programming skills in Python.
• Experience with data modeling and schema design in SQL and NoSQL systems.
• Experience designing and maintaining data pipelines (Airflow, Dagster, Prefect, or similar).
• Proficiency with cloud-based data services (AWS, GCP, Azure).
• Proficiency in multiple programming languages.
• Experience with entity resolution or record linkage algorithms.
• Experience incorporating ML workflows into ETL pipelines.
• Hands-on experience with Vector Databases and embedding-based search pipelines.
• Familiarity with graph databases (Neo4j, Neptune, or Gremlin) for ETL, modeling, and querying.
• Experience with OpenSearch / Elasticsearch, including index creation, tuning, and advanced queries.
• Experience with streaming data systems (Kafka, Kinesis) or distributed processing frameworks (Spark, Flink).
• Knowledge of semantic search, RAG pipelines, or LLM-enhanced retrieval.
• Experience with containerization and orchestration (Docker, Kubernetes) and CI/CD pipelines.
• Background in information retrieval, knowledge graphs, or data platform architecture.
• Experience using data catalog/lineage tools (OpenMetadata, DataHub, etc.).
• Strong experience with modern ETL tools for both large and small data processing (PySpark, Dask, DuckDB, etc.).

Nice To Have

• Strong analytical and problem-solving abilities.
• Excellent communication and collaboration across technical and non-technical teams.
• Curious, proactive, and adaptable to emerging data technologies.
• Self-starter with ownership and accountability for delivering high-quality solutions.

Interview Process

• Recruiter Interview: 30 minutes
• Hiring Manager Interview: 45 minutes
• Panel Interview: 60 minutes