SemIf is a tool for making semantic decisions from open language models running locally on consumer hardware.
The problem it solves is that agent systems often need to make small runtime decisions—routing, retries, evidence validation—by asking a language model, only to immediately parse the generated text back into a boolean or categorical choice. This wastes the model's generation capacity on text that software will discard. SemIf reproduces the interface pattern of Jev, a closed TypeSafe service for semantic decisions, but using open models. Instead of generating and parsing text, it reads typed option probabilities directly from the model's logits, skipping the answer sentence and decoding overhead entirely.
The tool suits projects where you control the hardware and want to avoid cloud dependencies or waitlists for decision-making services. It runs on a single GPU—the README demonstrates it on a 3090—and also supports Apple Silicon via MLX and CPU-only inference via llama.cpp. The baseline approach works with frozen 4B models and includes temperature calibration per workload. If you need semantic decisions embedded in agent loops and prefer local execution to external APIs, this is a direct alternative to closed services. The README does not compare it to other open-source decision frameworks.
The project accepts contributions across multiple backends and model variants. Recent work added PyTorch and MPS support for Apple Silicon, integrated a Qwen 27B variant with committed evidence, and implemented per-workload temperature calibration. The tool maintains deterministic prompt construction so that scores remain consistent across backends, with explicit documentation of where quantized weights or different evaluation paths may introduce small numerical differences. Each decision result includes typed scores, timing, model revision, and a prompt hash for reproducibility.
