Sr. Staff Performance Tuning Engineer - CPU PMU & Virtualization
About NIO
NIO Inc. is a pioneer and leading company in the global smart electric vehicle market. Founded in November 2014, NIO aims to shape a sustainable and brighter future with the mission of “Blue Sky Coming.” NIO envisions itself as a user enterprise where innovative technology meets experience excellence. NIO designs, develops, manufactures, and sells smart electric vehicles, driving innovations in next-generation core technologies. NIO distinguishes itself through continuous technological breakthroughs and innovations, exceptional products and services, and a community for shared growth. NIO provides premium smart electric vehicles under the NIO brand, premium smart electric vehicles for families through the ONVO brand, and small smart high-end electric cars with the FIREFLY brand.
Responsibilities
- Identify and analyze performance bottlenecks across CPU, memory hierarchy, cache, interconnect, and interrupt subsystems.
- Leverage hardware PMU counters to measure and interpret cycles, instructions, cache misses, branch mispredictions, TLB misses, and stall cycles.
- Compute and analyze performance metrics such as IPC, CPI, MPKI, memory bandwidth utilization, and stall breakdowns.
- Optimize system software, applications, and drivers for throughput, latency, and performance determinism.
- Profile and reduce hypervisor overhead, including VM exits, interrupt injection, and stage-2 MMU translation costs.
- Optimize vCPU placement strategies, huge page usage, and IRQ affinity for performance-sensitive workloads.
- Analyze and tune virtio and IOMMU performance paths.
- Investigate cross-core and cross-cluster latency issues, including IPI overhead and interrupt routing on ARM-based systems.
- Configure and interpret ARM PMUv3 and related vendor-specific performance monitoring extensions.
- Apply Top-Down Microarchitecture Analysis (TMA) to identify frontend/backend bottlenecks.
- Correlate PMU-derived metrics with kernel traces, source code paths, and system behavior.
- Design and implement custom event sets tailored to workload-specific performance characterization.
Required Qualifications
- Bachelor’s, Master’s, or Ph.D. in Computer Science, Electrical Engineering, or a related field.
- 8+ years of experience in system performance engineering or related roles.
- Strong understanding of computer architecture, including: CPU pipelines, execution engines, and memory hierarchy; Interrupt architectures (e.g., GICv3/GICv4, APIC concepts); MMU, virtual memory, and page table structures.
- Deep expertise with Linux performance tooling: perfeBPF / BCC / bpftrace, ftrace / trace-cmd.
- Proficiency in C/C++, Python, and shell scripting for performance tooling and automation.
Preferred Qualifications
- Hands-on experience with ARM PMUv3 on Cortex-A55/A76/A78 or Neoverse platforms.
- Experience with one or more virtualization stacks: KVM/QEMU, XenseL4, or other RTOS/hypervisor environments.
Benefits
- Anthem Blue Cross, HSA, and Kaiser HMO medical plans with $0 for Employee Only Coverage.
- Dental (including orthodontic coverage) and vision plan. Both provide options with a $0 paycheck contribution covering you and your eligible dependents.
- Company Paid HSA (Health Savings Account) Contribution when enrolled in the High Deductible Anthem Blue Cross medical plan.
- Healthcare and Dependent Care Flexible Spending Accounts (FSA).
- 401(k) with Brokerage Link option.
- Basic Life, AD&D, short-term and long-term disability insurance.
- Employee Assistance Program.
- Sick and Vacation time.
- 13 Paid Holidays a year.
- Paid Parental Leave for first 8 weeks at full pay (eligible after 90 days of employment with NIO).
- Paid Disability Leave for first 6 weeks at full pay (eligible after 90 days of employment with NIO).
- Voluntary benefits including: Voluntary Life and AD&D options for you, your spouse/domestic partner and dependent child(ren), pet insurance.
- Commuter benefits.
- Mobile Cell Phone Credit.
- Free lunch and snacks.
- Onsite gym.
- Employee discounts and perks program.