Data Platform Engineer (Member of Technical Staff)

About the role

We are looking for Founding- and Staff-level Engineers to design and implement the foundational pillars of Cogent’s data platform and integration pipeline.

Responsibilities

• Build the new standard for ingesting and extracting enterprise data
• Design highly performant data pipelines and indexing strategies that enable Applied AI use cases like semantic search and retrieval augmented generation
• Work with Applied AI teams to taxonomize, correlate and deduplicate data across various enterprise sources, and architect indexing, storage and query strategies for data in Cogent’s data lake
• Arcitect the modern data platform for Gen AI
• Implement generalizable design patterns and system components that allow us to scale from 1 integration to thousands of integrations

Requirements

• 5+ years of professional experience as a hands-on engineer and technical leader leading multiple projects
• Experience building data pipelines for Information Retrieval Systems (such as knowledge graphs, search engines, or similar) for enterprise use cases
• Experience in ETL orchestration and workflow management tools (such as Temporal, Airflow) as well as building batch processing and streaming systems with tools like Apache Kafka/Flink
• Expert in database fundamentals, SQL, data reliability practices and distributed computing
• Experience working fluently with standard orchestration & deployment technologies like Kubernetes, Terraform, Docker, Databricks, etc in multiple clouds

Qualifications

• Strong understanding of data engineering principles and practices
• Experience with cloud platforms (AWS, Azure, GCP)
• Knowledge of data modeling and normalization techniques
• Experience with data warehousing and analytics tools

Skills

• Proficiency in Python, Java, or similar programming languages
• Experience with big data technologies (Hadoop, Spark, etc.)
• Understanding of security and privacy best practices
• Experience with CI/CD pipelines and automation tools

Benefits

• Competitive compensation package including base salary and potential for variable compensation, equity, and benefits
• Flexible work arrangements
• Comprehensive health and wellness programs
• Professional development opportunities

Pay

The standard base salary range for this position is $100,000 - $300,000 annually. Compensation offered will be determined by factors such as location, job level, job-related knowledge, skills, and experience. Certain roles may be eligible for variable compensation, equity, and benefits.

Schedule

Full-time position