Data Engineer
Alt · United States · Yesterday
RemoteRemoteEngineering$155k–$165k/yrFull-time
The Role
We're looking for a data engineer to build robust pipelines that power our pricing model and marketplace insights. Your responsibilities include:
- Design, optimize, and own data pipelines that ingest transaction and listing data from major auction houses and marketplaces.
- Build comprehensive monitoring and alerting systems to track latency, uptime, and coverage metrics.
- Continuously improve data infrastructure by modernizing storage and processing technologies, reducing manual interventions, and optimizing for cost, performance, and reliability.
- Partner with internal teams to understand data usage patterns and ensure pipelines deliver clean, standardized data that meets product requirements.
What you'll do here
Here’s what you’ll be doing:
- Design, optimize, and own data pipelines that scrape, process, and ingest transaction and listing data from major auction houses and marketplaces.
- Build comprehensive monitoring and alerting systems to track latency, uptime, and coverage metrics across all data sources.
- Continuously improve our data infrastructure by modernizing storage and processing technologies, reducing manual interventions, and optimizing for cost, performance, and reliability.
- Partner with internal teams to understand data usage patterns and ensure pipelines deliver clean, standardized data that meets product requirements.
Must-haves
- 3-4 years of experience in data engineering or related fields
- Strong Python proficiency with at least 3 years of hands-on experience
- Proven experience with large-scale data processing using dataframe technologies (Pandas, Polars, PySpark, or similar)
- Hands-on experience with pipeline orchestration tools (Airflow, Dagster, or similar DAG-based systems)
- Solid SQL skills for data analysis and transformation
- Track record of owning at least one data pipeline end-to-end within the past 2 years
- Solid understanding of data quality and the importance of delivering clean and usable data
- A pragmatic mindset focused on delivering value incrementally rather than pursuing perfection
Nice-to-haves
- Experience with web scraping technologies (Selenium, Puppeteer, Beautiful Soup)
- Familiarity with data infrastructure and cloud services (AWS preferred)
- Interest in or knowledge of trading cards, collectibles, or alternative asset markets
- Experience with LLM-based automation tools for data extraction and processing