DataOS Data Engineer

About the role

DataOS is seeking a skilled Data Engineer to join our innovative team. You will collaborate on advanced data solutions, applying your expertise to build and optimize data pipelines, repositories, and models. Your role will involve designing and implementing secure, scalable data architectures, enhancing existing systems, and leading cross-functional teams to deliver high-quality data-driven products.

Responsibilities

• Design and establish secure and performant data architectures for data product pipelines, repositories, or models.
• Analyze design requirements and determine necessary coding, programming, and integration activities based on general objectives and overall architecture.
• Write and execute complete testing plans, protocols, and documentation for assigned portions of data systems or components; identify and resolve issues with code and integration.
• Lead a project team of other data engineers to develop reliable, cost-effective, and high-quality solutions for assigned data systems, models, or components.
• Collaborate and communicate with project teams regarding project progress and issue resolution.
• Support co-development processes and tools to promote common approaches to solving complex problems.
• Represent the data engineering team during all phases of larger and more-complex development projects.
• Provide guidance and mentorship to less experienced staff members.

Requirements

Must have a Bachelor's or Master's degree in Computer Science, Data Engineering, Information Systems, Engineering, or a related field. Typically 4-6 years of relevant experience.

Skills

• Proficiency in data engineering tools, languages (Python is mandatory, DBT is preferred), and frameworks for large data set mining, cleansing, and exploration.
• Familiarity with SQL and cloud-based data systems, relational data modeling, and complex, distributed, and massively parallel cloud systems (AWS, GCP, Azure).
• Strong analytical and problem-solving skills, able to represent complex algorithms in software.
• Experience with designing scalable and performant data systems/solutions.
• Ability to performance tune Spark code.
• Strong understanding of database technologies and management systems.
• Strong understanding of cloud-based systems/services, differentiating between data lake houses and warehouses.
• Experience with workflow orchestration tools (Airflow, Jenkins).
• Strong Notebooks environment experience (Jupyter, DataBricks).
• Experience collecting requirements from partners and selecting appropriate technologies for end-to-end data flow requirements.
• Experience with database architecture testing methodology, including execution of test plans, debugging, and testing scripts and tools.
• Excellent written and verbal communication skills in English and a local language.

Benefits

The company offers a competitive salary range of $105,550 to $161,800 USD annually, with opportunities for bonuses and equity. Additional benefits include:

• Health insurance
• Dental insurance
• Vision insurance
• Long-term/short-term disability insurance
• Employee assistance program
• Flexible spending account
• Life insurance
• Generous time off policies, including 4-12 weeks fully paid parental leave based on tenure, 11 paid holidays, and additional flexible paid vacation and sick leave.