Software Engineer, Inference Runtime

EngRadar / LM Studio Software Engineer, Inference Runtime LM Studio LM Studio builds on-device AI software allowing users to run large language models locally on their own hardware. Remote New York City Full-time Posted today aillmdeveloper-tools $150k–$350k Apply directly → LM Studio is used by millions of people around the world to run AI on their own computers, and now with Bionic - also in the cloud. Our values prioritize putting the human in the center, and creating tools that we want to use ourselves, and recommend to our friends and family. As a team, we work with high technical intensity and personal responsibility. We are looking for curious, self-motivated, creative, and technically excellent teammates to join us and build the future of human-AI interactions in software. The Role We are looking for an Inference Runtime Software Engineer to push forward LM Studio's inference stack on-device and in the cloud. You will integrate new inference engines and runtime capabilities, bring up new open-weight models and modalities, and optimize model execution for a wide range of CPU and GPU targets. You will also contribute improvements to the open-source projects we build on. Qualifications Significant experience building production ML systems, inference runtimes, or performance-sensitive infrastructureStrong programming ability in Python and C++Deep understanding of transformer architectures and the mechanics of model inferenceExperience profiling CPU or GPU workloads and reasoning about compute, memory, synchronization, and data movementExperience with PyTorch and inference systems such as llama.cpp, MLX, ExecuTorch, vLLM, SGLang, or TensorRT-LLMStrong debugging instincts across model code, runtime internals, operating systems, and CPU or GPU executionTakes personal responsibility for the correctness and performance of their work Bonus Qualifications Past contributions to open-source inference runtime projects such as llama.cpp, MLX, ExecuTorch, vLLM, SGLang, or TensorRT-LLM Responsibilities Maintain and push forward our inference stack on-device and in the cloudBring up new model architectures and multimodal modelsImprove latency, throughput, memory use, and reliability across CPU, CUDA, Metal, Vulkan, and ROCm runtimesBuild runtime capabilities for model loading, batching, scheduling, caching, and distributed executionBenchmark and diagnose correctness and performance problems across the inference stackContribute upstream to open-source projects such as llama.cpp and MLX Benefits Competitive salary and equity grantsGreat medical, vision, dental healthcare plansCatered team lunch / expensed dinners in the officeFlexible PTOFlexible WFHSun-drenched office in SoHo in NYC Posted by LM Studio on their own careers page — you apply directly, no recruiter in between. View original / apply → More at LM Studio L Systems Engineer (C++) LM Studio LM Studio builds on-device AI software allowing users to run lar… New York City aillm $150k–$350k 2mo ago L Software Engineer, Application LM Studio LM Studio builds on-device AI software allowing users to run lar… New York City aillm $175k–$275k 2d ago L GTM, Enterprise LM Studio LM Studio builds on-device AI software allowing users to run lar… New York City aillm $170k–$270k 1d ago L Software Engineer, Agent Harness LM Studio LM Studio builds on-device AI software allowing users to run lar… New York City aillm $150k–$350k today $150k–$350kdisclosed by employer Apply directly → Browse Full-Stack Jobs