Senior Data Engineer

Responsibilities

• Develop Python data pipelines to build business data objects supporting applications.
• Perform all programming, project management, and development assignments without close supervision.
• Work directly on complex application/technical problem identification and resolution.
• Interpret design requirements for engineering implementation.
• Build and deploy CI/CD pipelines.
• Implement source to target mapping as pipeline code.
• Perform unit and integration testing.
• Maintain high standards of software quality within the team by establishing good practices and habits.
• Identify and encourage areas for growth and improvement within the team.
• Perform other job duties as assigned by Caterpillar management from time to time.

Requirements

• Extensive software development experience and solid working knowledge of OOP principles.
• Knowledge and experience with embedding models and concepts for effectively chunking data.
• Extensive experience in designing and developing software applications in Python.
• Extensive experience working with Git version control.
• Extensive experience deploying software using CI/CD tools such as Jenkins, Github Actions, Azure Devops etc.
• Experience with AWS components such as Bedrock, Lambda, Dynamo, Fargate, S3, Sagemaker, IAM and RDS.
• Experience with relational and/or NoSQL databases such as PostgreSQL, DynamoDB, MongoDB, Oracle, etc.
• Demonstrated strong learning ability and a proactive approach to staying current with the latest technologies and industry trends.

Qualifications

Considerations for top candidates:

• Knowledge and experience with embedding models and concepts for effectively chunking data.
• Extensive experience in designing and developing software applications in Python.
• Extensive experience working with Git version control.
• Extensive experience deploying software using CI/CD tools such as Jenkins, Github Actions, Azure Devops etc.
• Experience with AWS components such as Bedrock, Lambda, Dynamo, Fargate, S3, Sagemaker, IAM and RDS.
• Experience with relational and/or NoSQL databases such as PostgreSQL, DynamoDB, MongoDB, Oracle, etc.
• Demonstrated strong learning ability and a proactive approach to staying current with the latest technologies and industry trends.

Skills

Software Development: Extensive knowledge of software development tools and activities; ability to produce software products or systems in line with product requirements.

Software Development Life Cycle: Knowledge of software development life cycle; ability to use a structured methodology for delivering and managing new or enhanced software products to the marketplace.

Software Product Design/Architecture: Knowledge of software product design; ability to convert market requirements into the software product design.

Software Product Technical Knowledge: Knowledge of technical aspects of a software products; ability to design, configure and integrate technical aspects of software products.

Software Product Testing: Extensive knowledge of software product testing; ability to design, plan, and execute testing strategies and tactics to ensure software product quality and adherence to stated requirements.

Benefits

• Medical, dental, and vision benefits
• Paid time off plan (Vacation, Holidays, Volunteer, etc.)
• 401(k) savings plans
• Health Savings Account (HSA)
• Flexible Spending Accounts (FSAs)
• Health Lifestyle Programs
• Employee Assistance Program
• Voluntary Benefits and Employee Discounts
• Career Development
• Incentive bonus
• Disability benefits
• Life Insurance
• Parental leave
• Adoption benefits
• Tuition Reimbursement