DATA ENGINEER-Python, AWS, Spark (Hybrid)

Responsibilities

• Utilizes industry-adopted languages and frameworks in coding, testing, security, DevOps, DataOps and data engineering practices
• Develops and maintains reusable, scalable, and compliant data solutions across multiple platforms and compute environments
• Responsible for the identification, acquisition, cleansing, profiling, and ETL (extracting, transformation, and loading) of data used in analytic discovery and production solution deployment across multiple platforms
• Establishes business domain knowledge for existing State Farm data sources and investigates, recommends, and initiates acquisition of data resources, both internal and external
• Identifies and consults on emerging technologies and critical core systems, including techniques, tools, data sources, and platforms in the data engineering field
• Familiar with handling datasets containing mixes of structured and unstructured data
• Exhibits DataOps mindset where team is accountable for ensuring data aligns to enterprise needs and leveraging automation to deliver quality data solutions
• Collaborates with cross-functional teams to analyze, design, deploy, support, and secure technology to ensure efficient management of technology and data-related assets in accordance with market best-practices and external regulations
• Applies complex principles, theories, and concepts in computer science for data engineering solutions

Qualifications

• Minimum of 2-4 years of professional experience as a Data Engineer
• Proficiency in programming languages such as Python, Spark SQL (or PySpark), R, Java, Bash, etc.
• Hands-on experience with AWS services including ETL tools (Glue, EMR Serverless), Lambda, Step Functions, EventBridge, S3, DynamoDB, Kinesis Firehose, Redshift, Iceberg, and SageMaker
• Experience with distributed data processing frameworks such as Apache Spark, Databricks
• Experience with infrastructure as code tools such as OpenTofu (formerly Terraform) for managing cloud resources and deployments
• Familiarity with CI/CD pipelines including automated testing, security scans, and tools like Airflow
• Additional Experience or ability to rapidly gain P&C data domain knowledge, including rating, underwriting, and/or claims
• Experience with relational databases such as DB2, Postgres, Redshift, etc.
• Experience with version control systems such as GitHub or GitLab
• Data access skills using SQL, and Athena
• Experience in designing, building, and maintaining data pipelines for automated data processing
• Knowledge of data modeling techniques such as star schema and snowflake schema, with an understanding of data architecture