Polaris

pip install "polaris-nlp[torch]"

Overview

Polaris is a reference implementation first: an end-to-end NLP stack built from scratch on PyTorch tensor primitives, clean enough to read as a teaching text and engineered well enough to train a real model on a real dataset and serve it. The primary product is the codebase itself. It does not compete with Hugging Face on features — the goal is to be the clearest, best-engineered from-scratch NLP system you can learn from (this identity is captured in an ADR).

Problem

Modern NLP tooling optimizes for using models, not understanding them. The lifecycle — data, tokenization, collation, model, training, evaluation, deployment — is fragmented across libraries, and every stage hides its mechanics behind abstractions. When you want to know how the system actually works, there is nowhere readable to look.

Implemented (v1.1)

• Tokenizers — whitespace and from-scratch byte-pair encoding (BPE).
• Data & collation — Polaris-native dataset interfaces (IMDB backend via an optional extra) and a collation layer that turns tokenizer output into padded, model-ready batches.
• Models — a mean-pooling classifier baseline and a from-scratch transformer encoder; optional pretrained GloVe word embeddings.
• Self-supervised pretraining — masked language modeling from scratch: pretrain the transformer trunk on unlabeled text, then transfer it into the classifier.
• Training engine — configuration-driven training with recorded, reproducible runs (config + metrics + environment + report per run).
• Evaluation — accuracy, per-class precision/recall/F1, confusion matrices.
• Deployment — trained models save as self-describing bundles (weights + architecture + vocabulary + labels) that reload with no training code; a polaris CLI (predict, console, serve), a FastAPI HTTP service, and a Docker image.

Benchmarks: a four-lever study

The headline is not a number — it's a controlled investigation on IMDB sentiment (25k train / 25k test, seed 0, all components from scratch). Four classic levers, each measured against the same pipeline:

| Lever | Result | | --- | --- | | Transformer vs mean-pooling baseline | Tie (~85.5%) — at ~14× the cost | | Subword tokenization (BPE) | Slightly hurts (~84%) — dilutes whole-word sentiment signal | | Pretrained GloVe embeddings | No movement — 25k labels already suffice | | MLM pretraining (controlled ablation) | Big head start (epoch-1 val 0.81 vs 0.74), same ~86% ceiling |

All four levers land at ~85–86%: the ceiling is the task and the data/compute regime, not any single component. Measuring and explaining that — with reproducible, controlled experiments — is the point of building the stack from scratch.

Engineering discipline

• Vertical slices — every release leaves the system runnable end to end; never a model you can't train or a trainer with nothing to train.
• Evidence-driven abstraction — an abstraction is extracted only when two or more concrete implementations demand it (ADR-0004). A registry system built early sits deliberately dormant until a real consumer exists (ADR-0005).
• Own the interface — PyTorch and Hugging Face datasets are implementation details, never public API.
• Strict gates — Black, Ruff, MyPy (strict), and offline-only Pytest gate every merge; major decisions become ADRs.

Lessons learned

The benchmark study is the lesson: four classic levers — a bigger model, subword tokenization, pretrained embeddings, and self-supervised pretraining — all landed at the same ~86% on IMDB. The ceiling belonged to the task and the data/compute regime, not to any component. Getting a result you can explain required owning every layer of the stack, which is the strongest argument for building it from scratch.

Roadmap

Post-1.0: experiment tracking, expanded inference and deployment utilities, and visualization — each added as a vertical slice when a real use case demands it.