Senior Big Data Engineer

About the role

We are seeking a Senior Big Data Engineer to join our Infrastructure Engineering team. This role is responsible for designing, building, maintaining, and modernizing our enterprise big data platforms that support large-scale data processing and analytics across the organization.

Responsibilities

• Design, deploy, administer, and optimize enterprise Big Data platforms.
• Manage and support distributed technologies including: Apache Kafka, Hadoop / MapR, Aerospike.
• Build highly available, scalable, and resilient data infrastructure across multiple data centers.
• Perform capacity planning, performance tuning, troubleshooting, and root cause analysis for production environments.
• Automate infrastructure deployment, configuration management, and operational workflows using Infrastructure as Code and scripting.
• Collaborate with Software Engineering, Data Engineering, SRE, and Database teams to support new applications and platform requirements.
• Implement monitoring, alerting, and observability solutions to ensure platform health and availability.
• Participate in production support and on-call rotations for critical infrastructure.

Requirements

• 4+ years of experience supporting large-scale Linux infrastructure.
• 4+ years of experience administering distributed Big Data platforms.
• Strong experience with: Apache Kafka, Hadoop (MapR, Cloudera, Apache Hadoop, or similar), Aerospike.
• Strong Linux system administration experience (RHEL, Rocky Linux, CentOS).
• Experience with relational databases such as: PostgreSQL, MySQL.
• Experience troubleshooting distributed systems in production environments.
• Strong knowledge of networking fundamentals including DNS, TCP/IP, load balancing, and security.
• Experience with shell scripting (Bash, Python, or similar).
• Experience with monitoring platforms such as Grafana, Prometheus, Graphite, or similar.
• Experience working in highly available, 24x7 production environments.
• Ability to work independently in a fast-paced production environment.
• Passion for automation, operational excellence, and continuous improvement.

Preferred Qualifications

• Experience with Kubernetes and containerized workloads.
• Experience with Ceph or other distributed storage platforms.
• Experience with Infrastructure as Code tools such as Terraform or Spacelift.
• Experience with configuration management tools such as Puppet or Ansible.
• Experience with cloud platforms (AWS).
• Familiarity with CI/CD pipelines and DevOps practices.

Qualifications

• Experience with Apache Kafka, Hadoop (MapR, Cloudera, Apache Hadoop, or similar), and Aerospike.
• Strong Linux system administration experience (RHEL, Rocky Linux, CentOS).
• Experience with relational databases such as PostgreSQL and MySQL.
• Experience with shell scripting (Bash, Python, or similar).
• Experience with monitoring platforms such as Grafana, Prometheus, Graphite.
• Experience with Kubernetes and containerized workloads.
• Experience with Ceph or other distributed storage platforms.
• Experience with Infrastructure as Code tools such as Terraform or Spacelift.
• Experience with configuration management tools such as Puppet or Ansible.
• Experience with cloud platforms (AWS).
• Familiarity with CI/CD pipelines and DevOps practices.

Benefits

• Comprehensive Healthcare Coverage from Day One
• Generous Time Off
• Holiday Breaks and Quarterly Wellness Days
• Equity and Employee Stock Purchase Plan
• Family-Focused Benefits and Parental Leave
• 401k Retirement Savings Plan with Employer Match
• Disability and Life Insurance
• Cell Phone Subsidy
• Monthly Fitness and Reimbursement
• Career Growth

Company Culture

• Hybrid Schedule: Two remote workdays (Monday and Friday) and three in-office days (Tuesday through Thursday).
• Catered lunches, beverages, snacks, training opportunities, and engaging office events.
• Community service and volunteer opportunities.
• Company-matched charitable contributions.
• Mental health resources.
• Structured career development initiatives and growth frameworks.
• Culture and inclusion programs.
• Bonusly peer-to-peer recognition platform.

About Magnite

Magnite (NASDAQ: MGNI) is the world’s largest independent sell-side advertising platform. Publishers use our technology to monetize their content across all screens and formats including CTV / streaming, online video, display, and audio. Our tech fuels billions of transactions per day!

Our Current Tech Stack

• Big Data: Apache Kafka, Hadoop / MapR, Aerospike, Spark
• Operating Systems: Rocky Linux
• Databases: PostgreSQL, MySQL, ScyllaDB
• Automation: Bash, Python, Terraform, Ansible, Puppet
• Monitoring: Grafana, Prometheus, Graphite
• Infrastructure: Kubernetes, Ceph
• Networking: TCP/IP, DNS, Load Balancing

Location and Compensation

New York, NY Hybrid Schedule (M/F remote, T/W/TH in-office) U.S. Remote
At Magnite, we cultivate an environment of continuous growth and collaboration. Our work impacts what millions of people read, watch, and buy, and we’re looking for people to help us tackle that responsibility with creativity and focus.
In NY, the base salary range for this role is $170,000 – $180,000. Actual compensation will be determined based on location, experience, skills, and internal equity.