Data Engineer
Vulcan Elements · Durham, NC · Yesterday
On-siteEngineeringFull-time
Responsibilities
- Evaluate and select platforms for the data Lakehouse, ETL tooling, and operational databases, weighing scalability, compliance requirements, operational burden, and cost.
- Review, refine, and implement data architecture design documents, ensuring designs are technically sound and account for CUI and ITAR data handling requirements.
- Make and document key platform and design decisions with enough clarity that future team members can understand the reasoning and build on it.
- Ensure the architecture scales from pilot plant to full-scale facility without fundamental redesign.
- Apply sound engineering practices to everything you build: version control, testing, observability, and documentation, and hold those standards as the data team grows.
- Design and build ETL pipelines that move data from operational data stores into the data Lakehouse with full contextual enrichment, making it ready for analytics and AI workloads.
- Collaborate across engineering, operations, and IT to understand data flows, dependencies, and integration requirements, and translate them into pipeline and architecture decisions.
- Identify and eliminate manual data workflows, replacing them with monitored, reliable pipelines.
- Diagnose and resolve data quality issues across the stack, and build monitoring into pipelines so problems surface early.
- Define data models that support operational queries, analytical workloads, and future AI and ML applications.
- Own data contextualization standards ensuring every data point carries the metadata needed to make it meaningful.
- Contribute to schema design and payload definitions for operational data stores, working toward consistency and legibility across the organization.
- Support the development of reporting and visibility tools that give operations and leadership clear insight into process and quality data.
- Write clear technical documentation for architecture decisions, data models, pipeline designs, and operational runbooks.
Qualifications
- 8+ years of experience in data engineering, data infrastructure, or a closely related technical role with a track record of owning and delivering production systems.
- Demonstrated experience designing and building data lakes, Lakehouses, or analytical data stores; understands the tradeoffs between platforms and can make and defend platform selection decisions.
- Strong experience designing and building ETL/ELT pipelines that enrich and contextualize data.
- Deep fluency with data modeling for both operational and analytical workloads; can design schemas that serve present needs without foreclosing future ones.
- Experience with relational databases (PostgreSQL, SQL Server, or similar); writes and debugs SQL confidently.
- Comfortable working in a fast-moving environment with a small team, making decisions with incomplete information and documenting them clearly for future colleagues.
- Strong communicator who can work across technical and non-technical stakeholders and translate between operational requirements and data architecture decisions.
- Must be a U.S. Person due to required access to U.S. export-controlled information or facilities.