Associate Data Engineer 2027

About the role

As an Associate Data Engineer at IBM, you will harness the power of data to unveil captivating stories and intricate patterns. You'll contribute to data gathering, storage, and both batch and real-time processing. Collaborating closely with diverse teams, you'll play an important role in deciding the most suitable data management systems and identifying the crucial data required for insightful analysis. As a Data Engineer, you'll tackle obstacles related to database integration and untangle complex, unstructured data sets. These positions are anticipated to start in 2027.

Responsibilities

• Assist in designing and implementing scalable data architecture and management systems tailored for modern cloud environments
• Work on optimizing existing data pipelines and processes for improved performance
• Work with ETL/ELT ingestion pipelines
• Collect and analyze data to identify trends, providing clients with actionable insights to enhance marketing, operational, and business practices
• Participate in troubleshooting data-related issues, working to solve challenges and data inconsistencies
• Create visually compelling and user-friendly data visualizations, dashboards, and reports to effectively communicate findings to both technical and non-technical stakeholders
• Ensure the integrity, accuracy, and reliability of data through rigorous data cleaning, validation, and preprocessing procedures
• Work with the project team to prioritize and translate Client requirements and define current and future operational scenarios (processes, models, use cases, plans and solutions)
• Work collaboratively with Client and the Architect to ensure proper translation of business requirements to solution requirements
• Present analytical findings and recommendations clearly and concisely, demonstrating the value of data-driven decision-making to clients

Consulting Skills

Advocates business process transformation by analyzing application portfolios across the ecosystem to identify process optimization and automation opportunities, leveraging planning, project management, and Agile methodologies to drive effective solutions. Demonstrates strong communication, problem-solving, adaptability, and teamwork skills, with a collaborative mindset, curiosity to learn, and the ability to clearly articulate ideas, ask insightful questions, and recommend solutions.

Required Technical and Professional Expertise

• Ability to incorporate a variety of statistical and machine learning techniques
• Basic understanding of Cloud (AWS, Azure, etc.)
• Ability to use programming languages like Java, Python, Scala, etc., to build pipelines to extract and transform data from a repository to a data consumer
• Ability to use Extract, Transform, and Load (ETL) tools and/or data integration, or federation tools to prepare and transform data as needed
• Ability to use leading edge tools such as Linux, SQL, Python, Spark, Hadoop and Java
• Exposure to ETL/ELT projects, data warehouse design, analytics pipelines, capstone data engineering
• Willingness to travel up to 100%, based on project requirements

Preferred Qualifications

• Bachelor's degree in a related field (Computer Science, Data Science, Statistics, Math, MIS, Engineering) — preferred
• Preferred Skills/Tech: SQL, Python, dbt, Snowflake, BigQuery, Airflow, data modeling, CI/CD basics
• Preferred Certifications: SnowPro Core, Databricks Data Engineer Associate, Google Associate Cloud Engineer or Data Engineer coursework
• General skills: GenAI literacy - prompt engineering, RAG, fine-tuning, and evaluation of generative models