Disiarkan 1 minggu lalu
Inference Engineer
- Hibrid
- Sepenuh masa
- Python
Huraian kerja
About Us
Most AI is frozen in place - it doesn’t adapt to the world. We think that’s backwards. Our mandate is to build efficient intelligence that evolves in real-time. We bring together exceptional people with the drive and creativity to push the boundaries of continual adaptation. We’re looking for builders and creative thinkers ready to shape the next era of intelligence.
Our team is small, talent-dense, and moving quickly, with a culture rooted in accountability, initiative and shared success. We look for people who care deeply, ship fast, and are eager to make a dent in the world. We operate with low egos, high velocity, and extreme ownership to build technology that expands access and benefits the many, not the few.
How We Work
- Impact over activity: We are building a generational company. We value people who execute on ideas rather than comment from the sidelines. We believe our competitive edge lies in caring about excellence across the entire lifecycle - from research to product. No task is beneath anyone: no one is too senior to test the build, clean the data, or unblock a teammate.
- Vulnerable, low-ego collaboration: We believe exceptional work comes from people who challenge, support and push one another to think bigger. Rapid iteration requires stepping into discomfort. We encourage bold, half-baked, and unorthodox ideas. When we disagree, we share feedback directly, never through side channels.
- Celebrate decisively: Extraordinary effort should be deeply memorable. We celebrate milestones generously and publicly - via spot bonuses, team dinners, spontaneous gifts, and company-wide recognition - to fuel each other for the next sprint.
The Role
Our vision is AI systems that are flexible, adaptable, and accessible to everyone. We believe efficiency is what makes this possible. It’s how we expand access and ensure innovation benefits the many, not the few.
You'll own the cost and performance of our inference stack. Your work will determine how efficiently we serve models as workloads, traffic, and hardware change.
You'll work closely with the engineers operating the serving fleet while owning the core performance levers: caching, batching, quantization, decoding, and kernel-level optimization. Success means improving throughput and latency without compromising reliability or model quality.
What You'll Do
- Improve throughput, cost, and tail latency through KV-cache management, continuous batching, speculative decoding, and quantization.
- Optimize long-context prefill and decode workloads based on real production traffic.
- Tune routing between our infrastructure and external providers based on cost, capacity, and performance.
- Work within serving engines such as vLLM, SGLang, and TensorRT-LLM, going below the framework when needed.
- Build profiling and measurement systems that show where time, memory, and compute are being spent.
Skills You'll Bring
We look beyond traditional markers of experience. What matters most to us is what you’ve built and the impact you’ve had. We value demonstrable output, curiosity and alignment with our core values:
- 5+ years in ML systems, inference infrastructure, or performance engineering, with measurable improvements in cost or latency.
- Deep understanding of model serving, including prefill and decode, memory bandwidth, batching, and concurrency.
- Production experience with serving engines such as vLLM, SGLang, or TensorRT-LLM.
- Strong Python skills and proficiency in C++, Rust, or another systems language.
- Experience with GPU performance, including CUDA, NCCL, mixed precision, memory layout, kernels, or quantization.
Above all, we're looking for great teammates who make work feel lighter and aren't afraid to go out on a limb with bold ideas. You don't need to be perfect, but you do need to be adaptable. We encourage you to apply, even if you don't check every box.
What We Offer
- Innovative culture: You’ll be part of a generational opportunity to define the trajectory of AI, surrounded by a team pushing the boundaries of what’s possible.
- Growth paths: A dynamic environment with opportunities to take on new challenges, expand your responsibilities, and make an impact beyond your immediate role.
- Adaption Passport: An annual travel allowance to explore somewhere new, experience different cultures and keep expanding your horizons. It can be used to travel to any country you haven’t previously been to.
- Lunch on us: A weekly allowance for takeout or grocery delivery, wherever you are in the world. Whether it's stocking your fridge or having dinner delivered, it’s one less thing to think about.
- Comprehensive benefits: High-quality medical coverage and benefits designed to support your health and well-being.
- Team offsites and visits to SF HQ: We're a global team, but believe there's real value in spending time together in person. We regularly bring teammates together at our SF HQ and for team offsites in locations around the world, including London, Mexico City, and New York.
Kerja lain di Amerika Syarikat
Senior Lending Assistant (Full Time) - SB South Belt
Prosperity Bank Houston, Texas, Amerika Syarikat🇺🇸
Personal Banker I (Full Time) - Converse
Prosperity Bank Converse, Texas, Amerika Syarikat🇺🇸
Account Executive (MA) — Residential, C&I Energy Storage
EV.Careers Boston, Massachusetts, Amerika Syarikat🇺🇸
Junior Sales Manager
EV.Careers Dallas, Texas, Amerika Syarikat🇺🇸
Patient Services
FamilyCare Health Centers Eleanor, Virginia Barat, Amerika Syarikat🇺🇸
Project Manager
ServiceMaster DSI Eagan, Minnesota, Amerika Syarikat🇺🇸
Restoration Technician
ServiceMaster DSI Indianapolis, Indiana, Amerika Syarikat🇺🇸
Direct Support Professional (Full-time Tues-Sat 7am-3pm)
Lehigh Valley Companion Care Homes Allentown, Pennsylvania, Amerika Syarikat🇺🇸
Direct Support Professional (Full-time Tues-Sat 3-11pm)
Lehigh Valley Companion Care Homes Allentown, Pennsylvania, Amerika Syarikat🇺🇸
Career Coach- Marion South, Indianapolis, IN
RADcube - A NLogix Company Indianapolis, Indiana, Amerika Syarikat🇺🇸
Senior Maintenance Technician
RHI Magnesita GmbH Moulton, Alabama, Amerika Syarikat🇺🇸
Kiln Tech Nights
RHI Magnesita GmbH York, Pennsylvania, Amerika Syarikat🇺🇸
Consultant Federal Tax Specialty Test Fall 2029 | Multiple Locations
Forvis Mazars Raleigh, Carolina Utara, Amerika Syarikat🇺🇸
SITE DIRECTOR (IN120_02)
Mersen Boonton, New Jersey, Amerika Syarikat🇺🇸