Data Engineer

What You Will Do

Aid in constructing a world-class Big Data platform capable of processing streams of data and enabling machine learning and advanced analytics. Operate entirely in the cloud for enhanced scalability and rapid market entry. Manage and integrate extensive data volumes across various internal and external systems. Stay updated on emerging technologies and their applicability to data management. Collaborate with an agile team comprising of business analysts, testers, architects, and project managers. Concentrate on developing intricate logic integrations. Ensure high-quality documentation, code, and business logic. Prioritize non-functional requirements by maintaining code, supporting, restoring, monitoring, and optimizing performance for all deliveries.

What You Need

• Design, implement, and extend core data systems that facilitate reporting and data visualization.
• Manage data integrations within the company’s technology stack.
• Provide runtime and automation solutions to empower developers in migrating and running workloads in the public cloud.
• Maintain and support all data workflows.
• Design, enhance, and support CI/CD frameworks, container solutions, runtime environments, and public cloud infrastructure.
• Create and maintain complex data workflows adhering to data management policies.
• Design and document database architectures.
• Responsible for ingestion and extraction of data using MDM tools like Informatica, Amperity, etc.
• Expertise in data warehousing and familiarity with cloud-based warehouse offerings.
• Create and maintain diagnostic, alerting, and monitoring code.
• Develop database schemas, tables, procedures, and permissions.
• Develop database utilities and automated reporting.
• Prepare written materials for documentation, reference, and information conveyance.
• Experience building data flows for data acquisition, aggregation, and modeling, utilizing both batch and streaming paradigms.
• Experience working with public cloud providers (AWS, GCP, Azure).
• Experience managing CI/CD pipelines.
• Familiarity with Kubernetes clusters in diverse environments.
• Experience with access controls, secrets management, monitoring, and service discovery in Kubernetes clusters.
• Experience with containerized workflows, applications, and promoting container adoption among developers and teams.
• Experience building ingestion, ETL data pipelines, particularly through code-oriented systems like Spark, Airflow, Luigi, or similar, and handling varied data formats.
• Experience operating in secure networking environments (e.g., behind a corporate proxy).
• Expertise in data engineering languages such as Python, Java, Scala, SQL.
• Familiarity with data visualization tools like Power BI, Tableau, etc.
• Experience in creating business requirements documents and other application system-related documents.

Pay

TBD

Schedule

Remote

Benefits

N/A