Data & Software Engineer
Avalore.ai · Chantilly, VA · 2 mo ago
On-siteInformation TechnologyFull-time
Responsibilities
- Work with stakeholders to understand data requirements, assess feasibility, and design appropriate solutions with minimal oversight
- Leverage strong problem-solving and debugging skills for data quality issues, pipeline failures, and performance bottlenecks
- Leverage a background in large-scale data migration or platform modernization efforts
- Contribute to data engineering documentation, best practices, and design patterns
Requirements
- Minimum of 5 years' experience with: Apache Spark & PySpark
- Advanced Python skills (including Pandas & NumPy)
- Docker, Podman
- AWS S3, Lambda & Step functions
- Airflow, etc
- SQL (with Trino)
- NoSQL, DynamoDB
- Unity Catalog OSS, Apache Polaris
- Airflow, etc
- Apache Superset
- Terraform or CloudFormation
- OpenLineage
- H3, PostGIS