Senior Engineer

Role Overview

We are seeking a Principal Splunk Engineer to lead the design, operation, and evolution of our large-scale Splunk Enterprise / Splunk Cloud deployment.

Key Responsibilities

• Platform Architecture & Operations:
  • Architect, operate, and optimize a distributed, large-scale Splunk environment (indexer clusters, search head clusters, cluster masters, deployment servers, IDM, ADFS/SAML integrations)
  • Lead capacity planning, index design, data retention strategies, and SmartStore lifecycle management
  • Maintain high availability, scaling, and resilience across multi-site deployments (including DR strategy)
  • Drive Splunk version upgrades, app updates, cluster maintenance, and platform hardening
• Security Logging & SOC Enablement:
  • Collaborate with SOC, Incident Response, and Threat Hunting teams to ensure high-quality security log ingestion
  • Onboard and normalize logs from firewalls, EDR, identity platforms, cloud providers, network telemetry, and custom applications
  • Develop and optimize detection content: correlation searches, risk-based alerting, data models, macros, lookups, summaries
  • Ensure compliance with logging standards (MITRE ATT&CK mapping, CIS/SOC2/ISO27001 logging requirements)
• Data Engineering & Observability:
  • Build and manage ingestion pipelines, parsing, field extractions, CIM compliance, HEC configurations, and forwarder architecture
  • Implement data lifecycle tiers, filtering strategies, routing, and ingestion controls to reduce cost and improve efficiency
  • Optimize search performance, knowledge objects, summary indexing, and acceleration strategies
• Governance & Best Practices:
  • Establish Splunk development standards, dashboards, and naming conventions
  • Mentor junior engineers and act as a technical escalation point for the team
  • Maintain documentation, operational runbooks, and logging onboarding guidelines
  • Partner with Engineering, Cloud, SecOps, and App teams to drive company-wide observability maturity

Required Qualifications

• 5+ years experience administering large Splunk Enterprise or Splunk Cloud environments
• Strong hands-on knowledge of:
• Indexer clustering, search head clustering
• SmartStore / S3-compatible object store design
• Universal/heavy forwarder architecture
• Ingest actions, parsing, props/transforms
• KVStore, RBAC, SAML, encryption
• Deep experience with security log ingestion and SIEM use cases
• Strong SPL expertise, including:
• Search optimization
• Summary indexing / data model acceleration
• CIM mapping and field normalization
• Experience with Linux systems engineering, scripting (Python/Bash), and automation frameworks (Ansible, Terraform, GitOps preferred)

Preferred Qualifications

• Splunk certifications (Core Consultant, Enterprise Admin, Enterprise Architect, ES Analyst/ES Admin, or equivalent)
• Experience with:
• Enterprise Security (ES)
• SOAR (Phantom or comparable)
• AWS/Azure/GCP cloud logging architectures
• Familiarity with high-throughput message brokers (Kafka/FluentD/Cribl)
• Background in cybersecurity engineering or threat detection

Skills

• Automation
• Influence
• Result Orientation
• Stakeholder Management
• Technical Strategy Development
• Application Development
• Architecture
• Business Acumen
• Risk Management
• Solution Design
• Agile Practices
• Analytical Thinking
• Collaboration
• Data Management
• Solution Delivery Process

Shift

1st shift (United States of America)

Hours Per Week

40