Open to new opportunities

I make AI run
fast on real hardware.

Senior AI Systems Engineer. I optimize LLM inference at the metal — CUDA kernels, ARM NEON intrinsics, Vulkan compute shaders, quantization pipelines. From 300% speedups on NVIDIA Jetson to building a Vulkan LLM runtime from scratch.

Resume ↓ GitHub ↗
3.5+
Years in AI Systems
300%
Best Perf Improvement
9
Quant Formats Shipped
151
tok/s (Q4_HQQ, llama.cpp)
Ramaseshan Subramanian
Experience

Where I've worked

Senior Software Engineer (AI) — MulticoreWare Inc.
Jan 2023 – Dec 2025
  • Engineered 300% performance gains and single-digit ms inference latency on NVIDIA Jetson by optimizing CUDA kernels and C++ runtimes — directly enabled a Flash LIDAR partner to secure a production contract with Rivian.
  • Architected high-performance LLM inference pipelines on AMD EPYC servers using NUMA-aware scheduling and custom SIMD vectorization (AVX2/AVX-512), contributing to platform competitiveness and commercial adoption for AMD.
  • Integrated BEVDet and BEVFormer perception models into Autoware Universe for real-time sensor fusion, delivering deterministic latency and ~3x pipeline speed increase for autonomous vehicle stacks.
  • Built automated accuracy and performance benchmarking frameworks to validate bf16/fp32 regressions, preventing latency and quality degradation in production inference releases.
  • Led a research team porting x86 deep learning pipelines to ARM and DSP-accelerated edge devices, redesigning memory execution models for constrained hardware — efforts garnered strategic recognition from a premier Indian automotive manufacturer.
IoT Product Manager & AI Engineer — Indicold
Apr 2021 – Oct 2021
  • Led end-to-end development of telepresence robotics and warehouse automation IoT solutions, owning edge perception, networking, and control systems.
  • Designed and deployed reliable edge architectures integrating real-time sensor processing and decision-making for autonomous operations with severe latency constraints.
  • Built a secure, WebRTC-based video conferencing platform on Raspberry Pi hardware for remote operations during the COVID-19 pandemic.
Projects

What I've built

🚘

ECU-Insight — Autonomous ECU Diagnostic Agent

A framework-free LLM agent that diagnoses automotive ECU faults from live OBD-II signals. 62 tools across 9 namespaces, data-driven tool registry with scored search, explicit context compaction with scratchpad persistence, and structurally-isolated typed subagents. Graded 5-dimension eval rubric with deterministic CI replay.

PythonAnthropic API OBD-IIPydantic asyncioELM327SQLite
62tools / 9 namespaces
1.73efficiency score
5/5rubric dims pass
Read the deep-dive →
⚡

Viking — Vulkan LLM Runtime for Android

Built a from-scratch Vulkan 1.1 compute inference engine for on-device LLM inference. 9 GGUF quantization shaders (Q4_0 through IQ4_XS), thermal governor, paged KV cache, and Kotlin SDK. Discovered Qualcomm's 6ms Vulkan fence overhead, pivoted to NEON SDOT achieving 11.2 tok/s.

C++20VulkanGLSL ARM NEONGGUFKotlinAndroid NDK
0.50→11.2tok/s journey
9quant shaders
37security fixes
Read the deep-dive →
🔥

llama.cpp Q4_HQQ Quantization Engine

Integrated the Q4_HQQ quantization format deep into llama.cpp with custom AVX2 and NEON SIMD kernels. Built a fast Python GGUF converter and enabled KV cache quantization. 3.3x model footprint reduction with near-lossless quality.

C++CUDAAVX2 NEONllama.cppPython
151tok/s
3.3xcompression
-1.06%perplexity vs Q4_0
👁

ORB-SLAM3 Edge Deployment on DSP

Redesigned ORB-SLAM3's memory architecture and execution pipeline for TI TDA4VM DSP hardware. Achieved ~3x rendering and localization speedup, unlocking real-time visual SLAM on constrained embedded devices.

C++ROS2Computer Vision TI TDA4VMDSP
~3xspeedup
Real-timeSLAM on edge
🔒

EnterpriseGradeRAG — RBAC-Enforced RAG Backend

Production RAG system with physically-isolated role-based access control, hybrid dense + sparse retrieval via Reciprocal Rank Fusion, grounded answer generation with citation tracking, and a full evaluation pipeline (RAGAs + retrieval benchmarking). Five roles, four access levels, 167 tests.

PythonFastAPIChromaDB BM25SQLiteStreamlit DockerRAGAs
0.634NDCG@10
167unit tests
5 roles4 access levels
Read the deep-dive →
🚗

BEV Perception for Autonomous Vehicles

Integrated BEVDet and BEVFormer perception models into Autoware Universe. Real-time multi-sensor fusion with deterministic latency on NVIDIA Jetson Orin. ~3x pipeline throughput increase for production AV stacks.

PyTorchCUDAROS2 AutowareJetson Orin
~3xthroughput
<10mslatency
Skills

What I work with

Languages & Low-Level

C++20CUDA GLSL / SPIR-VPython KotlinBash ARM Assembly

AI / ML Inference

llama.cppvLLM ONNX RuntimePyTorch TensorFlow LiteZenDNN Quantization (GGUF/GPTQ/AWQ)

Hardware & Optimization

Vulkan ComputeOpenCL AVX2 / AVX-512ARM NEON / SDOT NUMAKV Cache Kernel Fusion

Platforms

NVIDIA Jetson (Orin/Xavier) AMD EPYCQualcomm Adreno TI TDA4VMRaspberry Pi Android NDK

Tools & Infrastructure

CMakeGit CI/CDADB PerfettoGradle Docker

Domain Expertise

On-Device LLM Inference Autonomous Vehicles Edge AIIoT / Robotics Product Management
Writing

Things I've learned

2026-06-06
ECU-Insight: Autonomous Vehicle Diagnostics with a Framework-Free Agent Loop →
I built ECU-Insight — an autonomous agent that diagnoses ECU faults from live OBD-II signals with no stored DTCs to rely on. 62 tools, 9 namespaces, typed subagent isolation, explicit context compaction, and a graded 5-dimension eval rubric. Flagship run: 23 tool calls, 4 context compactions, 1.73 efficiency score — all 5 dimensions pass.
2026-06-01
Building an Enterprise RAG System That Actually Enforces Access Control →
A production-grade RAG backend with physically-isolated RBAC, hybrid dense + sparse retrieval with Reciprocal Rank Fusion, grounded generation with citation tracking, and an offline evaluation pipeline. Nine deep dives: ingestion, storage, RBAC, retrieval, generation, API, observability, evaluation, and security. NDCG@10 = 0.634.
2026-04-10
I Built an AI That Remembers Its Own Mistakes. Here Is What Broke Anyway. →
I built Veda — a memory system for AI-assisted porting — and used it to port 28 FFmpeg modules to ARM64 NEON. The corpus caught a real segfault this week. It also silently lost nine sessions of failure data. Here is the honest account.
2026-03-27
How I Found a Bug in TurboQuant by Asking a Language Model What 2+2 Is →
Three strategies to fix gibberish output all failed because the theory was wrong. A four-run isolation test found a vec_dot bug in TQ3_0's Key cache path in ten minutes.
2026-03-27
Cutting LLM Quantization Error in Half: What Happens When You Rotate Weights Before Compressing Them →
I adapted two techniques from Google's TurboQuant — Hadamard rotation and QJL residual correction — to improve HQQ weight quantization. 32-45% perplexity reduction at 3-bit across four models. Open source, GPU-verified, and portable to any quantization framework.
2026-03-27
Running LLMs on Phones: What Happens When You Compress the Memory That Models Think With →
I integrated HQQ and Google's TurboQuant into llama.cpp, cross-compiled for Android, and measured what happens when you compress the KV cache from 16 bits to 3.5 bits on a phone. TurboQuant was 3x faster. Here's the full story.
2026-03-25
What I Learned Building a Vulkan LLM Runtime for Android From Scratch →
I spent weeks building a Vulkan compute inference engine, wrote 9 quantization shaders, and ended up 10x slower than llama.cpp on CPU. Here's what I discovered about mobile GPU dispatch overhead, the 6ms Qualcomm fence, and why NEON SDOT won.
2025-12-16
FMA-Net++: Video Super-Resolution and Deblurring →
Technical summary of the FMA-Net++ architecture for joint video super-resolution and deblurring using flow-guided dynamic modulation.
Contact

Let's talk

I'm looking for my next role in AI systems engineering, inference optimization, or on-device ML. If you're building something that needs to run fast on real hardware, I'd love to hear about it.

✉ m.s.ramaseshan@gmail.com 💻 GitHub 💼 LinkedIn