Data Engineer

About the role

The Senior Data Engineer will be responsible for designing, developing, and maintaining robust and scalable SQL Server databases in Azure. The position requires expertise in T-SQL, Azure SQL Database, performance tuning, and data integration.

Responsibilities

• Design, develop, and optimize complex T-SQL queries, stored procedures, views, and functions.
• Design, build, and maintain scalable data pipelines and infrastructure to support mission-critical runtime applications, including data ingest, consumption, and ETL.
• Troubleshoot database performance and availability issues.
• Ensure data security, integrity, and compliance with organizational and regulatory standards.
• Partner with software engineers, project managers, and other stakeholders to understand data needs and deliver solutions that enable low-latency, high availability, eventually consistent consumption patterns.
• Implement data validation, monitoring, and governance frameworks to ensure data accuracy, consistency, and compliance with privacy and security standards.
• Leverage modern tools and frameworks (e.g., Apache Spark, Kafka, Airflow, or equivalent) to automate data workflows and improve operational efficiency.
• Provide technical guidance and mentorship to junior data engineers, fostering a culture of innovation and continuous improvement.

Requirements

• 5+ years of experience in data engineering or a related field, with a proven track record of building large-scale data systems.
• Bachelor’s in Computer Science, Information Systems, or related field.
• Strong proficiency in T-SQL and query optimization.
• Experience with Azure Data Factory, Logic Apps, and Azure Functions.
• Familiarity with DevOps practices and tools (e.g., Azure DevOps, Git).
• Expertise in SQL and experience with relational and NoSQL databases (e.g., MSSQL Server, PostgreSQL, MongoDB, Cassandra).
• Strong experience with big data technologies (e.g., Hadoop, Spark, Flink) and cloud platforms (e.g., AWS, GCP, Azure).
• Familiarity with data orchestration tools (e.g., Airflow, Dagster) and data modeling techniques.
• Knowledge of data streaming technologies (e.g., Kafka, Azure Event Hubs) and real-time processing.
• Strong analytical and problem-solving skills, with the ability to tackle complex data challenges and deliver efficient solutions.
• Excellent communication and teamwork skills, with experience working in cross-functional environments.

Preferred Qualifications & Skills

• Microsoft Certified: Azure Data Engineer Associate or similar certification.
• Experience with tableau, Power BI, Synapse Analytics, or Databricks.
• Knowledge of Python or C# for data-related scripting.
• Experience in Agile/Scrum environments.
• Familiarity with data governance, compliance, and privacy regulations (e.g., SOC, NIST, GDPR, CCPA).
• Contributions to open-source data projects or relevant certifications.