Senior AI Data Engineer

About the role

We're looking for a Senior AI Data Engineer to lead the AI engineering workstreams on the Data Platform team. This role has three distinct dimensions:

• Building the data infrastructure that enables AI use cases across the company (AI in the data layer)
• Serving platform stakeholders in building and shipping data products faster with AI (AI in the data platform)
• Helping our own team accelerate development through AI tooling (AI in the development lifecycle)

Responsibilities

Own the deployment path for Databricks Apps — creating the infrastructure and guardrails that let non-technical users bring their AI applications to production safely and consistently

Build the AI layer on top of Scribd's Medallion Architecture and Semantic Layer — connecting AI agents and agentic workflows to governed data, and enabling non-technical users to get self-service answers on top of it

Build AI skills and agents on top of our existing declarative tooling — giving platform stakeholders the interfaces they need to ship pipelines faster without deep expertise in every layer of the stack

Partner with teams across the organization to identify the right AI tools, frameworks, and agentic patterns that accelerate data product development and broader AI adoption

Identify and embed AI tools into how the Data Platform team writes, tests, and ships code — making AI-assisted development a standard part of our engineering workflow

Establish guardrails to ensure AI-generated code, queries, and pipelines are correct, consistent, and production-ready

Identify and embed AI tools into how the Data Platform team writes, tests, and ships code — making AI-assisted development a standard part of our engineering workflow

Help define and evolve data modeling and metadata patterns required to support AI use cases (e.g., context, documentation, discoverability)

Mentor other engineers and help define what great AI data engineering looks like at Scribd

Requirements

5+ years of data engineering experience, with at least 1 year focused on AI/ML infrastructure or LLM-powered applications

Strong proficiency in Python and SQL; comfort working across the full data stack from ingestion to serving

Hands-on experience with Databricks and cloud data platforms (Unity Catalog experience a strong plus)

Experience building or integrating NLP/LLM-based systems — RAG pipelines, semantic search, agent frameworks, or natural language interfaces

Working knowledge of how modern LLMs are trained, aligned and evaluated (RLHF, fine-tuning, prompt engineering, retrieval patterns) and the judgment to know when each approach is the right tool

A solid understanding of data governance, access control, and what it means to build on top of trusted data

A security-first mindset when building AI surfaces, including secret management, encryption at rest and in transit, and responsible handling of sensitive and PII data across AI pipelines and workflows

The ability to work autonomously on ambiguous problems and drive them to production

Strong communication skills — you can explain complex systems to technical and non-technical audiences alike

Qualifications

None provided

Skills

Python, SQL, Databricks (Unity Catalog, Databricks Apps, Databricks SQL, DABs, Genie Spaces), Apache Airflow, Spark, Terraform/Spacelift, AWS

Benefits

Comprehensive health, dental, and vision coverage
Mental health support and disability coverage
Generous paid time off, including vacation, sick time, holidays, winter break, volunteer time, and sabbaticals
Paid parental leave and family support benefits
Retirement matching and employee equity
Learning and development programs and professional growth opportunities
Wellness and home office stipends
Complimentary access to the Scribd, Inc. suite of products
Enterprise access to leading AI tools

Pay

Base pay is determined within a range. In the state of California, the reasonably expected salary range is between $155,000 to $235,000. In the United States, outside of California, the reasonably expected salary range is between $127,500 to $224,000. In Canada, the reasonably expected salary range is between $162,000 CAD to $205,000 CAD.

Schedule

Flexible work model (Scribd Flex)