Backend/Data Engineer

Things you should know about us

• We work extensively with industry experts to build software that is useful for our clients. This means frequent customer calls and a constant level of iteration.
• Talent density is far more important than team size. We want to hire the absolute best, and are willing to wait to find the right person.
• If you’re early in your career and/or don’t come from a “prestigious” background, don’t be afraid to apply - we all went to a state school and don’t place a large emphasis on prestige.
• Our office is in Manhattan.

What you’ll do

• Design, develop, and maintain scalable and efficient data pipelines to process and transform large volumes of data from disparate sources across the internet
• Implement and optimize data pipelines using Python/SQLAlchemy and Dagster, ensuring high performance, reliability, and scalability
• Work with PostgreSQL databases, including design, optimization, and troubleshooting
• Collaborate with teammates to build and maintain software that can query hundreds of millions of vectors in a single request
• Participate in code reviews, testing, and deployment processes to ensure code quality and maintainability
• Maintain and troubleshoot data infrastructure to identify and resolve issues proactively

Requirements

• Bachelor's degree in Computer Science, Engineering, or a related field
• Strong proficiency in Python programming language
• Solid experience with PostgreSQL databases, including writing efficient queries, performance tuning, and database administration
• Familiarity with data pipeline frameworks such as Dagster, Airflow, Luigi, or similar
• Knowledge of data processing and transformation techniques, including ETL processes
• Experience with Git
• Experience with Docker
• Excellent problem-solving skills and ability to debug complex systems
• Strong communication and collaboration skills, with the ability to work effectively in a team environment
• US citizen

Preferred qualifications

• Experience with cloud platforms such as AWS, GCP, or Azure
• Experience with data engineering best practices
• Experience with FastAPI and asynchronous programming
• Experience with SQLAlchemy
• Experience with data warehousing and business intelligence tools

Pay

$130K - $200K

Schedule

Full-time

Benefits

Competitive salary, comprehensive benefits package, and opportunities for growth and development within a fast-paced and innovative company.