Data Engineer

We need to output HTML fragment with sections: About the role, Responsibilities, Requirements, Qualifications, Skills, Benefits, Pay, Schedule. Only include sections if posting has real specific content. We need to parse the posting. First, identify content: - About the role: maybe a paragraph describing the role: "Peak Performers has partnered with a Texas State Agency to assist in their search for a hybrid Data Engineer (Big Data Engineer) in Austin, TX. In this position, you will perform advanced data engineering duties, supporting data engineering, machine learning, and analytics initiatives using the Databricks Unified Analytics Platform." That's about the role. - Responsibilities: list of major duties (the bullet list after "Major duties"). - Requirements: maybe the "What are we looking for?" list includes many years of experience. Could be considered requirements/qualifications. - Qualifications: maybe certifications preferred? Could be separate. - Skills: maybe list of technical skills: Python, R, SQL, Azure, Spark, Databricks, etc. - Benefits: "Exciting Opportunity Details Rate: $64.50/hour W2 Contract" is pay. Benefits: "Excellent medical health, dental & vision insurance at 60 days Our employees may choose to participate in a 403(b) retirement plan Qualified applicants with chronic medical conditions and/or disabilities receive priority placement We pay weekly through direct deposit In business since 1994 and thousands of job placements All employment offers are contingent on the successful completion of a pre-employment criminal background check, in compliance with all applicable federal and state laws." Also "Peak Performers is an equal opportunity employer..." but that's boilerplate EEO, we should remove. Also "Peak Performers does not accept unsolicited resumes from headhunters..." remove. - Pay: Rate: $64.50/hour W2 Contract. Also maybe mention "Rate: $64.50/hour". So we can have Pay section. - Schedule: maybe hybrid? Not explicit schedule. Could be hybrid (remote/onsite). The posting says hybrid Data Engineer. Could infer schedule hybrid but not explicit. We should not create Schedule unless posting actually states them. It says hybrid but not schedule details like full-time, part-time. Might be considered schedule? The instruction: "never create Pay/Schedule/Benefits sections unless the posting actually states them." It does state hybrid (work arrangement) but not schedule like hours. Might be okay to include Schedule? Probably not, because they didn't give explicit schedule like "Full-time, 40 hrs/week". Safer to omit Schedule. - Benefits: we have medical, dental, vision, 403(b), priority placement, weekly direct deposit. Those are benefits. - Qualifications: maybe the certifications preferred: "Databricks Certified Associate Developer for Apache Spark or Azure Data Engineer Associate preferred." Also maybe "Knowledge of ML libraries (MLflow, Scikit-learn, TensorFlow) preferred." Could be qualifications. - Skills: maybe list of technical skills: Python, R, SQL, Azure, Spark, Databricks, etc. We need to avoid repeating job title as heading. Also no Job ID/requisition numbers. We need to preserve all substantive information; do not invent or summarize away detail. So we need to include all details from the posting, but we can group them. We must not include boilerplate: cookie/navigation/legal/EEO/equal-opportunity/scam-warning boilerplate, duplicated lines, and application instructions. So we need to remove EEO statement, the line about unsolicited resumes, maybe also "All employment offers are contingent on the successful completion of a pre-employment criminal background check, in compliance with all applicable federal and state laws." That's legal boilerplate; should remove. Also "In business since 1994 and thousands of job placements" maybe not boilerplate but could be considered fluff; but it's substantive? It's about the company; maybe we can keep as part of Benefits? It's not a benefit per se. Might be considered company info; but we could drop as not required. The instruction: remove cookie/navigation/legal/EEO/equal-opportunity/scam-warning boilerplate, duplicated lines, and application instructions. The line about background check is legal; remove. The line about equal opportunity employer is EEO; remove. The line about unsolicited resumes is recruitment agency policy; maybe not boilerplate but could be considered application instruction? It's about not accepting unsolicited resumes; could be considered application instruction; remove. We need to keep all other substantive info. Let's extract sections: About the role: paragraph. Responsibilities: list of duties (the bullet list after "Major duties"). Requirements: maybe the "What are we looking for?" list (the many 4+ years etc). Could be considered requirements. Qualifications: maybe the preferred certifications and ML libraries. Skills: maybe the specific technical skills: Python, R, SQL, Azure, Spark, Databricks, etc. But those are already in requirements. Might be overlapping. We could combine Requirements and Skills? But we need to follow headings only if content exists. Could have Requirements (including years of experience) and Skills (specific technologies). However the "What are we looking for?" list includes both years and technologies. Could split: Requirements = years of experience; Skills = specific technologies (Python, R, SQL, Azure, Spark, Databricks, etc). But the list also includes "4+ years of proficiency in Python and R programming languages." etc. So it's overlapping. Better to have one section "Requirements" that includes all the bullet points from "What are we looking for?" plus maybe the certifications preferred as "Preferred Qualifications". But we need to avoid thin sections. Could have Requirements (including years and tech) and Qualifications (certifications). Or we could have Requirements and Skills separate. Let's decide: - About the role: paragraph. - Responsibilities: bullet list of major duties. - Requirements: bullet list of the "What are we looking for?" items (the 4+ years etc). This includes years of experience with various technologies. - Qualifications: bullet list of preferred certifications and ML libraries knowledge. - Benefits: bullet list of benefits (medical, dental, vision, 403(b), priority placement, weekly direct deposit). Also maybe mention "Exciting Opportunity Details Rate: $64.50/hour W2 Contract" is Pay, not benefit. - Pay: bullet or paragraph with rate and contract type. - Schedule: not present; omit. We must not repeat job title as heading. We need to output only HTML fragment. Let's craft:

About the role

Peak Performers has partnered with a Texas State Agency to assist in their search for a hybrid Data Engineer (Big Data Engineer) in Austin, TX. In this position, you will perform advanced data engineering duties, supporting data engineering, machine learning, and analytics initiatives using the Databricks Unified Analytics Platform.

Responsibilities

• Designs and develops scalable data pipelines using Apache Spark on Databricks.
• Implements ETL/ELT workflows for both structured and unstructured data.
• Optimizes Spark jobs for performance and cost efficiency.
• Integrates Databricks solutions with cloud services, such as Azure Data Factory.
• Automates deployments using CI/CD tools and version control systems.
• Collaborates with cross-functional teams including data scientists, analysts, and stakeholders.
• Ensures data quality, governance, and security using Unity Catalog or Delta Lake.
• Designs and maintains data models, schemas, and database structures to support analytical and operational use cases.
• Evaluates and implements appropriate data storage solutions, including data lakes (Azure Data Lake Storage) and data warehouses.
• Implements data validation and quality checks to ensure accuracy and consistency.
• Contributes to data governance initiatives, including metadata management, data lineage, and data cataloging.
• Implements data security measures, including encryption, access controls, and auditing, while ensuring compliance with regulations and best practices.

Requirements

• 4+ years of experience implementing ETL/ELT workflows for both structured and unstructured data.
• 4+ years of experience automating deployments using CI/CD tools, DevOps, and version control systems.
• 4+ years of experience collaborating with cross-functional teams (data scientists, analysts, stakeholders) in agile, multicultural environments.
• 4+ years of experience designing and maintaining data models, schemas, and database structures.
• 4+ years of experience evaluating and implementing data storage solutions, including Azure Data Lake Storage and data warehouses.
• 4+ years of experience with data validation, quality checks, metadata management, data lineage, and data cataloging.
• 4+ years of experience implementing data security measures (encryption, access controls, auditing) and ensuring compliance.
• 4+ years of proficiency in Python and R programming languages.
• 4+ years of strong SQL querying and data manipulation skills.
• 4+ years of experience with the Azure cloud platform.
• 4+ years of strong troubleshooting and debugging capabilities.
• 3+ years of experience designing and developing scalable data pipelines using Apache Spark on Databricks.
• 3+ years of experience optimizing Spark jobs for performance and cost efficiency.
• 3+ years of experience integrating Databricks solutions with Azure Data Factory.
• 3+ years of experience ensuring data quality, governance, and security using Unity Catalog or Delta Lake.
• 3+ years of deep understanding of Apache Spark architecture, RDDs, DataFrames, and Spark SQL.
• 3+ years of hands-on experience with Databricks notebooks, clusters, jobs, and Delta Lake.

Qualifications

• Knowledge of ML libraries (MLflow, Scikit-learn, TensorFlow) preferred.
• Databricks Certified Associate Developer for Apache Spark or Azure Data Engineer Associate preferred.

Benefits

• Excellent medical, dental & vision insurance effective after 60 days.
• Option to participate in a 403(b) retirement plan.
• Priority placement for qualified applicants with chronic medical conditions and/or disabilities.
• Weekly pay via direct deposit.

Pay

Rate: $64.50/hour, W2 contract.

We should also include the estimated start date and location? Those are details about the role; maybe they belong in About the role or a separate section like "Details". But we have no heading for "Details". Could add to About the role paragraph: include location, start date, contract duration. The posting includes: "Exciting Opportunity Details Rate: $64.50/hour W2 Contract Worksite Location: Austin, TX 78751 Estimated Start Date: 10/28/2026 through 08/31/2027 (opportunity for extension)". We already captured pay. We could add location and dates to About the role paragraph or a separate section like "Details". But we only have predefined headings; we could add location and dates to About the role. Thus modify About the role paragraph to include: "Peak Performers has partnered with a Texas State Agency to assist in their search for a hybrid Data Engineer (Big Data Engineer) in Austin, TX. In this position, you will perform advanced data engineering duties, supporting data engineering, machine learning, and analytics initiatives using the Databricks Unified Analytics Platform. Worksite location: Austin, TX 78751. Estimated start date: 10/28/2026 through 08/31/20