Currently
- Building
- Michi v2.0 just shipped. Next: SciNLI domain adaptation for Meridian's verifier and the 200K-abstract corpus.
- Researching
- Faithfulness verification, calibrated abstention, and efficient inference for high-stakes domains.
- Reading
- Hoffmann et al. on compute-optimal scaling; Leviathan et al. on speculative decoding.
Featured Projects
View all →Meridian
Stable · v1.0A from-scratch grounded RAG engine over biomedical literature (PubMed). Custom BPE tokenizer, dense retriever, HNSW/IVF ANN, cross-encoder reranker, and NLI faithfulness verifier — no third-party pre-trained models. Every answer is cited, verified, or refused.
Polaris
Stable · v1.1A production-inspired NLP engineering platform for understanding text. The complete encoder-side stack — data, tokenization, collation, models, training, evaluation, deployment — from scratch on PyTorch tensors.
Zenith
Stable · v1.0A from-scratch generative NLP library — decoder-only language models and text generation. Llama-style internals (RoPE, RMSNorm, SwiGLU), KV-cached generation, speculative decoding, LoRA, instruction tuning. Matches the nanoGPT baseline.
Michi
Stable · v2.0道 — a local-first ML workbench. Independent CLI tools that profile datasets, evaluate models, and benchmark them with real statistical rigor. Automates the implementation, never the judgement.
Research Interests
Research →Transformer Architectures
Attention variants, positional encodings, normalization schemes, and the design space of encoder and decoder stacks.
Efficient Inference
KV caching, speculative decoding, quantization, and the systems work that makes large models cheap to serve.
Retrieval & Grounding
Dense retrieval, retrieval-augmented generation, and grounding generative models in verifiable sources.
Representation Learning
What embeddings encode, how pretraining objectives shape them, and how to evaluate them honestly.
Generative AI
Decoder-only language modeling, sampling strategies, and evaluation of open-ended generation.
NLP
Tokenization, pretraining pipelines, and task-specific finetuning across the modern NLP stack.
Latest Engineering Notes
View all →The Cross-Validation Bug Worth 26 Points of Fake Accuracy
Michi warned users about group leakage while its own benchmark command ignored groups entirely. How entity leakage inflates scores, and why it had to be fixed before the API freeze.
Michi · 2 min read
When Your Reranker Is Worse Than Random
Meridian's cross-encoder pushed Recall@5 to 0.029 — below random ordering. Diagnosing the collapse, and why fallible pipeline stages must degrade gracefully.
Meridian · 2 min read
Greedy-Exact Speculative Decoding
How Zenith uses a small draft model to cut target-model forward passes by 3×+ while producing output identical to greedy decoding.
Zenith · 1 min read
Weight Tying in Language Models
Sharing the input embedding and output projection — why it works, when it stops working, and how it interacts with logit scale.
Zenith · 2 min read
Contact
Open to ML engineering roles, research collaborations, and conversations about open-source NLP infrastructure. Reach me at satyasainischal@gmail.com or on GitHub.