Manager, Data Engineer
Burtch Works · Queens, NY · 1 mo ago
Information TechnologyFull-time
About the role
We are a leading entertainment and hospitality organization operating in a highly data-intensive environment spanning gaming, finance, workforce operations, and enterprise planning. As the organization continues to modernize its data ecosystem, the Data & Analytics team is building scalable pipelines and integrations that improve how critical business data moves across enterprise systems and becomes available for planning, reporting, and decision-making. This is an opportunity to join a growing data organization and contribute directly to the continued modernization, reliability, and scalability of its enterprise data platform.
Responsibilities
- Build and maintain ETL/ELT pipelines that ingest financial, HR, gaming, and operational data into a SQL Server data warehouse.
- Develop pipelines within a newly established Medallion Architecture, implementing appropriate data validation and quality checks throughout each layer.
- Write production-quality Python using pandas and database connectivity frameworks such as pyodbc, SQLAlchemy, or similar technologies.
- Develop and maintain T-SQL stored procedures, SQL Agent jobs, and BULK INSERT/MERGE-based data loading processes.
- Build integrations that deliver validated enterprise data into downstream planning and reporting platforms through REST APIs.
- Develop and schedule pipeline workflows using established orchestration patterns, including assets, sensors, schedules, concurrency controls, and pools.
- Build comparison and reconciliation scripts that validate data across environments and incorporate automated data quality checks directly into event-driven pipelines.
- Monitor pipeline health through existing observability tools and dashboards, including OpenTelemetry, Loki, and Grafana, while adding appropriate logging to newly developed pipelines.
- Troubleshoot pipeline failures, data discrepancies, transaction issues, concurrency problems, and other complex production incidents through root-cause analysis.
- Work across both Linux and Windows environments to support enterprise data integrations and production workloads.
- Partner with Data, Analytics, Finance, HR, Gaming, and other business stakeholders to understand data requirements and deliver reliable solutions.
- Contribute to the continued improvement of engineering standards, documentation, data quality, observability, and overall platform reliability.
Requirements
- Education: Bachelor's degree in Computer Science, Engineering, Information Systems, Data Analytics, or a related technical discipline preferred.
- Experience:
- 3–5+ years of professional experience building production data pipelines, integrations, or backend data systems.
- Demonstrated hands-on experience developing ETL/ELT pipelines using Python and SQL within enterprise data environments.
- Strong experience with SQL Server and T-SQL, including writing, maintaining, and debugging stored procedures and working with SQL Agent jobs.
- Experience with workflow orchestration platforms such as Dagster, Airflow, Prefect, or similar technologies.
- Experience integrating applications and data platforms through REST APIs, including authentication, request/response handling, and error management.
- Experience working across Linux and Windows environments.
- Proven ability to independently troubleshoot complex data and production issues and drive problems through root-cause resolution.
Skills
- Strong hands-on Python development skills, including experience using pandas for data transformation and pyodbc, SQLAlchemy, or similar libraries for database connectivity.
- Strong SQL Server and T-SQL capabilities with the ability to develop and troubleshoot stored procedures, data loads, transactions, SQL Agent jobs, and BULK INSERT/MERGE patterns.
- Solid understanding of modern ETL/ELT architecture and data engineering principles, including Medallion Architecture, data quality, reconciliation, and scalable pipeline development.
- Experience working with workflow orchestration technologies and the ability to develop within established scheduling, event-driven, concurrency, and dependency-management patterns.
- Strong API integration capabilities with experience moving data between enterprise platforms through REST APIs and managing authentication, file transfers, job status polling, and error handling.
- Strong troubleshooting and complex problem-solving skills with the ability to investigate subtle production issues involving data integrity, pipeline performance, transactions, or concurrency.
- Understanding of observability and production monitoring concepts, including structured logging, metrics, dashboards, and tools such as OpenTelemetry, Loki, and Grafana.
- Strong communication and collaboration skills with the ability to work effectively with both technical teams and business stakeholders while independently owning engineering deliverables.
Preferred Qualifications
- Experience integrating enterprise planning, financial planning, or FP&A platforms through REST APIs, including chunked file uploads and polling for long-running processes.
- Experience extracting and integrating data from HR, payroll, or workforce management platforms such as Workday, UKG, or similar enterprise systems.
- Experience building observability capabilities, including structured logging, metrics pipelines, monitoring queries, and production dashboards.
- Experience owning infrastructure or data migrations across platforms and environments while maintaining data integrity and minimizing downtime.
- Demonstrated experience diagnosing complex production incidents through root-cause analysis rather than simply resolving immediate symptoms.
- Familiarity with Agentic Coding workflows and Model Context Protocol (MCP) integrations is a plus.