Palantir Foundry Data Engineer

About the role

Design, build, and maintain data pipelines and workflows using Palantir Foundry. Support advanced analytics and operational insights for the utility industry. Integrate diverse data sources into Foundry. Ensure data quality, governance, security, and reliability. Enable real-time decision-making for:

• Grid reliability
• Asset management
• Regulatory compliance
• Predictive analytics

Technical Expertise

Hands-on experience with Palantir Foundry development and data integration. Strong proficiency in:

• Python
• PySpark
• SQL
• Palantir Foundry-specific frameworks

Knowledge of utility data systems:

• SCADA
• GIS
• OMS
• ERP

Experience with cloud platforms:

• AWS
• Azure

Familiarity with containerization:

• Docker
• Kubernetes

Experience Required

• 3+ years of experience in data engineering or related data engineering roles
• Utility or energy industry experience preferred
• Experience working with cloud platforms and modern data engineering technologies

Data Pipeline Development

• Build and optimize workflows in Palantir Foundry
• Ingest data from:
  • SCADA
  • GIS
  • OMS
  • ERP systems
• Develop scalable pipelines for:
  • Real-time sensor data
  • IoT data streams
• Implement:
  • Data transformations
  • Data enrichment
• Support analytics and reporting requirements

Integration & Automation

• Integrate structured and unstructured data sources into Palantir Foundry
• Automate:
  • Data ingestion
  • Data validation
  • Data processing
• Ensure data accuracy and timeliness
• Collaborate with IT and OT teams to integrate operational and enterprise data systems

Data Quality & Governance

• Monitor and maintain data integrity across pipelines
• Apply enterprise data governance standards
• Ensure compliance with:
  • NERC CIP
  • FERC regulations
  • State regulations
• Implement robust:
  • Security controls
  • Access controls
• Maintain data quality and governance within Foundry

Collaboration & Support

• Work closely with:
  • Data scientists
  • Data analysts
  • Business teams
  • IT teams
  • OT teams
• Deliver reliable datasets for:
  • Predictive analytics
  • Reporting
  • Operational insights
• Provide technical support for Foundry workflows
• Create and maintain technical documentation

Performance Optimization

• Optimize workflows for:
  • High-volume data
  • High-velocity data
  • Real-time processing
• Troubleshoot:
  • Pipeline failures
  • Performance bottlenecks
  • Data processing issues
• Improve scalability and reliability of Foundry pipelines

Real-Time & Streaming Technologies

• Experience with real-time data streaming technologies preferred:
  • Kafka
  • Spark
• Experience developing real-time sensor and IoT data pipelines

Utility / Energy Domain Knowledge

• Knowledge of:
  • Grid modernization
  • Distributed Energy Resources (DER) integration
  • Renewable energy forecasting
  • Utility operational systems
• Familiarity with utility data from:
  • SCADA
  • GIS
  • OMS
  • ERP

Operations & Application Support

• Willingness to work on application maintenance
• Understanding of application support models
• Experience with:
  • L1 application support
  • L2 application support
  • L3 application support
• Ability to troubleshoot and resolve production issues
• Support application reliability and operational continuity

Soft Skills

• Strong problem-solving skills
• Strong analytical skills
• Excellent communication skills
• Ability to work effectively in cross-functional teams
• Ability to collaborate with technical and business stakeholders

Preferred Qualifications

• Real-time data streaming experience
• Kafka experience
• Spark experience
• Knowledge of grid modernization
• DER integration experience
• Renewable forecasting knowledge
• Palantir Foundry certification
• Data engineering certification
• AWS/Azure experience
• Docker/Kubernetes experience