Java Engineer

About the Role

Long Finch Technologies is seeking a Java Spark Engineer to architect, build, and optimize scalable, fault‑tolerant data systems. The role focuses on Apache Spark, Java, batch and streaming ETL/ELT, distributed systems, data architecture, performance optimization, and production reliability. The position involves technical leadership, mentoring engineers, driving architecture decisions, collaborating across product and analytics teams, and owning the reliability of large‑scale data pipelines.

Key Responsibilities

• Architect and build scalable, fault‑tolerant data pipelines using Apache Spark and Java.
• Lead the design of batch and streaming ETL/ELT systems handling large data volumes.
• Perform performance tuning involving partitioning, memory management, shuffle and data‑skew optimization, and job cost reduction.
• Establish coding standards and lead code and design reviews.
• Drive technical decisions related to data architecture, storage formats, and pipeline orchestration.
• Mentor mid‑level and junior engineers and serve as a technical escalation point.
• Partner with product, analytics, and platform teams to translate requirements into scalable data systems.
• Own production reliability, including on‑call responsibilities, incident response, and root‑cause analysis for pipeline failures.
• Evaluate and introduce tools and frameworks that improve system capabilities.
• Contribute to capacity planning and infrastructure cost optimization.
• Communicate technical tradeoffs and solutions to technical and non‑technical stakeholders.
• Drive cross‑team technical initiatives in changing and sometimes ambiguous environments.

Required Qualifications

• Bachelor's or Master's degree in Computer Science, Engineering, or a related field.
• 7+ years of professional Java development experience.
• 5+ years of hands‑on Apache Spark experience in production environments.
• Expert‑level understanding of distributed systems, including fault tolerance, data locality, shuffle mechanics, and resource management.
• Proven experience designing systems that process terabyte‑scale or larger data volumes.
• Strong SQL skills.
• Deep familiarity with columnar and modern data storage formats, including Parquet, ORC, Avro, Delta Lake, or Iceberg.
• Experience with cluster managers such as YARN and Kubernetes.
• Experience with cloud‑managed Spark.
• Proficiency with Kafka.
• Strong understanding of CI/CD, containerization, and infrastructure‑as‑code practices.
• Strong problem‑solving and analytical skills.
• Good communication and collaboration skills.
• Ability to work independently and as part of a team.

Preferred Qualifications

• Experience with Apache Flink or other stream‑processing frameworks.
• Familiarity with data governance, data lineage, and data quality frameworks.
• Experience with workflow orchestration at scale.
• Experience designing multi‑tenant or multi‑region data platforms.
• Prior experience leading a team or serving as a technical lead.
• Strong mentorship and coaching experience.

Skills & Competencies

• Java
• Apache Spark
• Distributed Systems
• Data Engineering
• Batch Processing
• Stream Processing
• ETL / ELT
• SQL
• Data Pipelines
• Data Architecture
• Data Modeling and Storage
• Parquet, ORC, Avro, Delta Lake, Iceberg
• Kafka
• YARN
• Kubernetes
• Cloud‑Managed Spark
• CI/CD
• Containerization
• Infrastructure as Code
• Performance Tuning
• Partitioning and Shuffle Optimization
• Memory Management
• Data Skew Optimization
• Workflow Orchestration
• Production Reliability
• Incident Response
• Root‑Cause Analysis
• Capacity Planning
• Cost Optimization
• Technical Leadership
• Mentoring and Coaching
• Cross‑Functional Collaboration

Education & Experience

• Education