Data Engineer (junior to mid) - Secret Clearance

Direct-hire, full-time role based on-site at Scott Air Force Base in Shiloh, IL. General schedule is 9:00 AM–5:00 PM, with flexibility based on project needs.

About the role

Our partner is seeking a skilled and highly motivated Data Engineer to support a growing portfolio of data and technology initiatives within the Defense sector. The Data Engineer will design, develop, and maintain cloud-based data infrastructure, scalable data pipelines, and analytics solutions. This role focuses on building and optimizing end-to-end ETL processes, integrating large and complex datasets, improving data quality, and supporting business intelligence and reporting initiatives.

Responsibilities

• Design, develop, and maintain end-to-end data pipelines using ETL processes and modern data engineering technologies.
• Build and optimize scalable data pipelines using tools and technologies such as Databricks, Python, Spark, Scala, JavaScript/JSON, SQL, and Jupyter Notebooks.
• Develop data processing workflows that support reliable, efficient, and high-performance analysis.
• Perform data cleansing, transformation, integration, and quality assurance activities to maintain the accuracy and integrity of large datasets.
• Leverage big data technologies to process and analyze large datasets within federal and Department of Defense environments.
• Troubleshoot data-related issues and develop effective solutions to complex data challenges.
• Implement and support data governance policies, standards, and procedures in alignment with regulatory requirements and industry best practices.
• Collaborate with technical and business stakeholders to understand data requirements and develop effective data models and architectures.
• Partner with data scientists, analysts, and other stakeholders to provide timely and accurate data for reporting, analytics, and decision-making.
• Maintain technical documentation for data pipelines, applications, workflows, and related processes.
• Evaluate emerging technologies and recommend tools and solutions that improve automation, scalability, performance, and data quality.

Requirements

• 3+ years of experience in data engineering, data analytics, software development, or a related technical field.
• U.S. citizenship and an active Secret clearance. Clearance sponsorship may be available for qualified candidates.
• Strong SQL skills with the ability to write, troubleshoot, and modify complex queries and procedures across a variety of datasets.
• Experience with Extract, Transform, Load (ETL) processes, including establishing and maintaining data ingestion and transformation workflows.
• Experience manipulating, integrating, and combining data from multiple sources.
• Hands-on programming experience with Python and related data engineering technologies.
• Experience with technologies such as Spark, Scala, JavaScript/JSON, and SQL.
• Business intelligence experience, including creating and updating applications, reports, dashboards, and analytical solutions.
• Familiarity with business intelligence platforms such as Cognos Analytics, Qlik Sense, and Power BI.
• Experience working with large datasets and using big data technologies to support efficient processing and analysis.
• Strong analytical and problem-solving skills.
• Strong communication and collaboration skills with the ability to work effectively across technical and business teams.
• Detail-oriented approach with a commitment to high-quality and accurate results.

Preferred Qualifications

• Experience with Databricks and modern cloud-based data engineering environments.
• Familiarity with Amazon Web Services, including AWS and Amazon S3.
• Experience working with Jupyter Notebooks.
• Understanding of data modeling, data visualization, database design principles, and data governance practices.
• Experience with Teradata, including Multiload, BTEQ, and related Teradata utilities or procedures.
• Experience supporting Department of Defense or other federal agency environments.
• Familiarity with Department of Defense accounting systems such as DEAMS, TFMS, NERP, or similar platforms.
• Knowledge of Department of Defense accounting processes, financial systems, or transportation policies.
• Experience supporting cloud modernization, data transformation, reporting, or analytics initiatives.

Pay

$65,000–$90,000/year

Benefits

• Health, vision, and dental plans for employees and their families.
• Competitive 401(k) plan with 100% company matching on employee contributions up to the first 6%.
• Generous paid time off based on length of service.
• Bonus opportunities based on individual performance and achievement of goals.
• Professional development support through training programs and certifications.
• Company-sponsored team-building activities, retreats, and professional conferences.