Databricks Data Engineer (1157930)

Location: Arlington, VA (hybrid) • Pay: $55–$65/hr (W‑2) • Employment type: Full‑time contract

About the role

The Judge Group is seeking a Databricks Data Engineer to support a newly awarded contract with a large federal agency. This is a hybrid, W‑2‑only opportunity (no third parties).

Responsibilities

• Build and maintain Databricks ingestion, transformation, orchestration, migration, and data‑processing pipelines.
• Develop scalable ETL/ELT solutions using Databricks, Apache Spark, PySpark, Python, SQL, and Delta Lake.
• Implement source‑to‑target mappings, transformation rules, business logic, data contracts, metadata, lineage, and data‑quality requirements.
• Develop migrated data feeds, APIs, curated datasets, reporting layers, and reusable data products.
• Create reconciliation routines and support source‑to‑target validation, exception analysis, and defect resolution.
• Build unit, integration, regression, performance, and data‑quality tests and document technical results.
• Support Git‑based development, CI/CD, release management, deployment, monitoring, troubleshooting, and stabilization.
• Analyze and resolve job failures, cluster‑performance issues, pipeline bottlenecks, schema changes, and data‑quality problems.
• Collaborate with data architects, engineering leads, integration engineers, profiling analysts, testers, security specialists, governance leads, and source‑system teams.
• Document reusable code, technical patterns, runbooks, troubleshooting procedures, and lessons learned for later modernization waves.
• Participate in Agile ceremonies, backlog refinement, sprint planning, demonstrations, and technical reviews.

Requirements

• Bachelor’s degree in computer science, data engineering, information systems, engineering, analytics, or a related field.
• Minimum 4 years of experience in data engineering, software development, ETL/ELT, cloud analytics, or a related technical discipline.
• Hands‑on experience with Databricks, Apache Spark, PySpark, Python, SQL, Delta Lake, and workflow orchestration.
• Experience building, testing, and maintaining batch data pipelines, transformations, and curated data layers.
• Experience with enterprise data migration, source‑to‑target mapping, reconciliation, validation, and data‑quality controls.
• Familiarity with Git, CI/CD, automated testing, deployment, monitoring, logging, and troubleshooting.
• Strong analytical, problem‑solving, documentation, collaboration, and communication skills.
• Ability to obtain and maintain a Federal Public Trust clearance.

Qualifications

• Databricks Certified Data Engineer Associate or Professional certification.
• Experience with Unity Catalog, Delta Live Tables, Databricks Workflows, or Auto Loader.
• Experience developing APIs, integration pipelines, or data‑consumption services.
• Experience with Azure DevOps, GitHub Actions, Jenkins, Terraform, or related engineering tools.
• Experience with streaming technologies such as Kafka or Azure Event Hubs.
• Experience migrating HR, ERP, workforce, financial, or sensitive enterprise data.
• Experience supporting federal agencies, regulated organizations, or public‑sector modernization.
• Experience with Microsoft Azure or Amazon Web Services.

Benefits

• Medical, dental, and vision insurance available to qualified candidates who meet eligibility requirements.