Senior Data Engineer
About the role
Omni Platform Group (OPG) is currently seeking a senior data engineer to join our technology team. This role involves building Omni products that sit atop cloud-based data infrastructure, integrating data into applications, and developing scalable and reusable systems that handle large amounts of data.
Responsibilities
- Steward data and compute environments to facilitate usage of data assets.
- Design, build, test and deploy scalable and reusable systems that handle large amounts of data.
- Lead and mentor a small team of developers.
- Perform code reviews and provide leadership and guidance to junior developers.
- Learn and teach new technologies to the team.
Requirements
- Experience designing and managing data flows.
- Experience designing systems and APIs to integrate data into applications.
- 8+ years of Linux, Bash, Python, and SQL experience.
- 4+ years using Spark and other Hadoop ecosystem software.
- 4+ years using AWS cloud services, esp. EMR, Glue, Athena, and Redshift.
Qualifications
- Passion for Technology: Excitement for new technology, bleeding edge applications, and a positive attitude towards solving real world challenges.
- BS, MS or PhD in Computer Science, Engineering, or equivalent real-world experience.
- Significant experience with Python, C++, or other popular language.
- Experience with big data and/or infrastructure.
- Bonus for having experience in setting up Petabytes of data so they can be easily accessed.
- Understanding of data organization, i.e., partitioning, clustering, file sizes, file formats.
- Data cataloging with Hive/Hive metastore or Glue or something similar.
- Experience working with relational databases.
- Experience with Hadoop, Hive, Spark, or other data processing tools.
- Experience in any of the modern data stack technologies like dbt, airbyte is desired.
- Experience building scalable data pipelines (Airflow experience a plus).
- Significant experience working with AWS and/or GCP.
- Proven ability to independently execute projects from concept to implementation to launch and to maintain a live product.
Skills
- Experience designing and managing data flows.
- Experience designing systems and APIs to integrate data into applications.
- Significant experience with Python, C++, or other popular language.
- Experience with big data and/or infrastructure.
- Experience with Hadoop, Hive, Spark, or other data processing tools.
- Experience in any of the modern data stack technologies like dbt, airbyte is desired.
- Experience building scalable data pipelines (Airflow experience a plus).
- Significant experience working with AWS and/or GCP.
- Proven ability to independently execute projects from concept to implementation to launch and to maintain a live product.
Benefits
Flywheel Commerce Network is committed to fair and equitable compensation practices. Actual compensation packages are based on several factors that are unique to each candidate, including but not limited to skill set, depth of experience, certifications, and specific work location. Salary Range: $120,000 USD - $140,000 USD
Pay
$120,000 USD - $140,000 USD
Schedule
Based out of Jersey City Office
Benefits
- Competitive paid time off, including annual leave plus paid public holidays
- Great learning and development opportunities
- Benefits that help you live your best life
- Parental leave and benefits
- Volunteering opportunities
- So you know: The hired candidate will be required to complete a background check
Equal Opportunity Employer
Flywheel Commerce Network is an Equal Opportunity Employer and participates in E-Verify. All applicants will receive fair consideration for employment. We do not discriminate based upon race, color, religion, sex, sexual orientation, age, marital status, gender identity, national origin, disability, or any other applicable legally protected characteristics in the location in which the candidate is applying.