LLM Evals, Guardrails & Observability Repositories

Claude-Code-Usage-Monitor

by Maciek-roboblog

Total Stargazers:8700

Total Subscribers:34

Real-time Claude Code usage monitor with predictions and warnings

TypeChat

by microsoft

Total Stargazers:8684

Total Subscribers:56

TypeChat is a library that makes it easy to build natural language interfaces using types.

transformer-explainer

by poloclub

Total Stargazers:8566

Total Subscribers:73

Transformer Explained Visually: Learn How LLM Transformer Models Work with Interactive Visualization

ClawWork

by HKUDS

Total Stargazers:8542

Total Subscribers:81

"ClawWork: OpenClaw as Your AI Coworker - 💰 $15K earned in 11 Hours"

OBLITERATUS

by elder-plinius

Total Stargazers:8278

Total Subscribers:76

OBLITERATE THE CHAINS THAT BIND YOU

bertviz

by jessevig

Total Stargazers:8175

Total Subscribers:72

BertViz: Visualize Attention in Transformer Models

gpt-instruct

by MDX-Tom

Total Stargazers:8149

Total Subscribers:32

A Codex jailbreak prompt and test pack for gpt. 针对 gpt 系列的 Codex 破甲提示词与测试包。

tensorboardX

by lanpa

Total Stargazers:7999

Total Subscribers:83

tensorboard for pytorch (and chainer, mxnet, numpy, ...)

bifrost

by maximhq

Total Stargazers:7996

Total Subscribers:32

Fastest enterprise AI gateway (50x faster than LiteLLM) with adaptive load balancer, cluster mode, guardrails, 1000+ models support & <100 µs overhead at 5k RPS.

coroot

by coroot

Total Stargazers:7913

Total Subscribers:52

Coroot is an open-source observability and APM tool with AI-powered Root Cause Analysis. It combines metrics, logs, traces, continuous profiling, and SLO-based alerting with predefined dashboards and inspections.

evidently

by evidentlyai

Total Stargazers:7911

Total Subscribers:55

Evidently is an open-source ML and LLM observability framework. Evaluate, test, and monitor any AI-powered system or data pipeline. From tabular data to Gen AI. 100+ metrics.

llm-gateway

by mnfst

Total Stargazers:7521

Total Subscribers:29

Connect Your Agents And Harnesses With Any Provider 🦚

openllmetry

by traceloop

Total Stargazers:7425

Total Subscribers:21

Open-source observability for your GenAI or LLM application, based on OpenTelemetry

opencompass

by open-compass

Total Stargazers:7422

Total Subscribers:29

OpenCompass is an LLM evaluation platform, supporting a wide range of models (Llama3, Mistral, InternLM2,GPT-4,LLaMa2, Qwen,GLM, Claude, etc) over 100+ datasets.

guardrails

by guardrails-ai

Total Stargazers:7399

Total Subscribers:41

Adding guardrails to large language models.

hertzbeat

by apache

Total Stargazers:7392

Total Subscribers:72

An AI-powered next-generation open source real-time observability system.

pxpipe

by teamchong

Total Stargazers:7384

Total Subscribers:27

cut Claude Code token usage by rendering text context as images

fuck-u-code

by Done-0

Total Stargazers:7283

Total Subscribers:21

Legacy-Mess Detector – assess the “legacy-mess level” of your code and output a beautiful report

tensorboard

by tensorflow

Total Stargazers:7216

Total Subscribers:182

TensorFlow's Visualization Toolkit

Guardrails

by NVIDIA-NeMo

Total Stargazers:7109

Total Subscribers:43

NeMo Guardrails is an open-source toolkit for easily adding programmable guardrails to LLM-based conversational systems.

plano

by katanemo

Total Stargazers:7047

Total Subscribers:52

Plano is an AI-native proxy server and data plane for agentic apps. Smart LLM routing, observability, agent orchestration, and guardrails so you stay focused on your agents core logic.

interpret

by interpretml

Total Stargazers:6938

Total Subscribers:142

Fit interpretable models. Explain blackbox machine learning.

superagent

by superagent-ai

Total Stargazers:6744

Total Subscribers:60

Superagent protects your AI applications against prompt injections, data leaks, and harmful outputs. Embed safety directly into your app and prove compliance to your customers.

LLMLingua

by microsoft

Total Stargazers:6650

Total Subscribers:40

[EMNLP'23, ACL'24] To speed up LLMs' inference and enhance LLM's perceive of key information, compress the prompt and KV-Cache, which achieves up to 20x compression with minimal performance loss.

nasiko

by Nasiko-Labs

Total Stargazers:6614

Total Subscribers:15

Developer Control Plane for your AI Agents

agent-starter-pack

by GoogleCloudPlatform

Total Stargazers:6557

Total Subscribers:64

Ship AI Agents to Google Cloud in minutes, not months. Production-ready templates with built-in CI/CD, evaluation, and observability.

gepa

by gepa-ai

Total Stargazers:6534

Total Subscribers:25

Optimize prompts, code, and more with AI-powered Reflective Optimization

chinese-llm-benchmark

by jeinlee1991

Total Stargazers:6436

Total Subscribers:70

非线智能 NoneLinear - ReLE评测：中文AI大模型能力评测（持续更新）：目前已囊括374个大模型，覆盖chatgpt、gpt-5.4、谷歌gemini-3.1-pro、Claude-4.6、文心ERNIE-X1.1、ERNIE-5.0、qwen3.6-max、qwen3.6-plus、百川、讯飞星火、商汤senseChat等商用模型，以及step3.5-flash、kimi-k2.6、ernie4.5、MiniMax-M2.7、deepseek-v4、Qwen3.6、llama4、智谱GLM-5.1、MiMo-V2、LongCat、gemma4、mistral等开源大模型。不仅提供排行榜，也提供规模超200万的大模型缺陷库！方便广大社区研究分析、改进大模型。

SuperPrompt

by NeoVertex1

Total Stargazers:6428

Total Subscribers:82

SuperPrompt is an attempt to engineer prompts that might help us understand AI agents.

Awesome-Prompt-Engineering

by promptslab

Total Stargazers:6329

Total Subscribers:96

This repository contains a hand-curated resources for Prompt Engineering with a focus on Generative Pre-trained Transformer (GPT), ChatGPT, PaLM etc

agent-governance-toolkit

by microsoft

Total Stargazers:6248

Total Subscribers:91

AI Agent Governance Toolkit — Policy enforcement, zero-trust identity, execution sandboxing, and reliability engineering for autonomous AI agents. Covers 10/10 OWASP Agentic Top 10.

AI-Infra-Guard

by Tencent

Total Stargazers:6244

Total Subscribers:48

A full-stack AI Red Teaming platform securing AI ecosystems via Agent Scan, Skills Scan, MCP scan, AI Infra scan and LLM jailbreak evaluation.

adversarial-robustness-toolbox

by Trusted-AI

Total Stargazers:6222

Total Subscribers:104

Adversarial Robustness Toolbox (ART) - Python Library for Machine Learning Security - Evasion, Poisoning, Extraction, Inference - Red and Blue Teams

mission-control

by builderz-labs

Total Stargazers:6210

Total Subscribers:31

Self-hosted control plane for AI agents: dispatch tasks, review runs, track spend, and operate OpenClaw, Claude Code, Codex, and other runtimes.

helicone

by Helicone

Total Stargazers:6151

Total Subscribers:58

🧊 Open source LLM observability platform. One line of code to monitor, evaluate, and experiment. YC W23 🍓

claude-token-efficient

by drona23

Total Stargazers:6028

Total Subscribers:44

One CLAUDE.md file. Keeps Claude responses terse. Reduces output verbosity on heavy workflows. Drop-in, no code changes.

Awesome-LLMOps

by tensorchord

Total Stargazers:5930

Total Subscribers:77

An awesome & curated list of best LLMOps tools for developers

typia

by samchon

Total Stargazers:5905

Total Subscribers:17

Super-fast/easy runtime validators and serializers via transformation

agentsview

by kenn-io

Total Stargazers:5880

Total Subscribers:34

Local-first session search, analytics, insights, and token use statistics for coding agents, supporting Claude Code, Codex, and more than 20 other agents.

SWE-bench

by SWE-bench

Total Stargazers:5829

Total Subscribers:39

SWE-bench: Can Language Models Resolve Real-world Github Issues?

agentops

by AgentOps-AI

Total Stargazers:5817

Total Subscribers:52

Python SDK for AI agent monitoring, LLM cost tracking, benchmarking, and more. Integrates with most LLMs and agent frameworks including CrewAI, Agno, OpenAI Agents SDK, Langchain, Autogen, AG2, and CamelAI

giskard-oss

by Giskard-AI

Total Stargazers:5811

Total Subscribers:42

🐢 Open-Source Evaluation & Testing library for LLM Agents

ann-benchmarks

by erikbern

Total Stargazers:5732

Total Subscribers:117

Benchmarks of approximate nearest neighbor libraries in Python

coze-loop

by coze-dev

Total Stargazers:5720

Total Subscribers:34

Next-generation AI Agent Optimization Platform: Cozeloop addresses challenges in AI agent development by providing full-lifecycle management capabilities from development, debugging, and evaluation to monitoring.

captum

by meta-pytorch

Total Stargazers:5701

Total Subscribers:276

Model interpretability and understanding for PyTorch

alignment-handbook

by huggingface

Total Stargazers:5677

Total Subscribers:106

Robust recipes to align language models with human and AI preferences

tokscale

by junhoyeo

Total Stargazers:5411

Total Subscribers:45

🛰️ Track token usage across AI coding agents from your terminal. 🏅 Global leaderboard with trillions of tokens tracked.

TheBigPromptLibrary

by 0xeb

Total Stargazers:5395

Total Subscribers:107

A collection of prompts, system prompts and LLM instructions

maka

by apache

Total Stargazers:5273

Total Subscribers:25

Apache Maka (Incubating) is a high-performance agent workspace that keeps a complete record of everything it did.

llm-attacks

by llm-attacks

Total Stargazers:4790

Total Subscribers:40

Universal and Transferable Attacks on Aligned Language Models

mcp-context-forge

by IBM

Total Stargazers:4464

Total Subscribers:34

An AI Gateway, registry, and proxy that sits in front of any MCP, A2A, or REST/gRPC APIs, exposing a unified endpoint with centralized discovery, guardrails and management. Optimizes Agent & Tool calling, and supports plugins.

PurpleLlama

by meta-llama

Total Stargazers:4388

Total Subscribers:68

Set of tools to assess and improve LLM security.

trulens

by truera

Total Stargazers:3547

Total Subscribers:23

Evaluation and Tracking for LLM Experiments and AI Agents

tensorwatch

by microsoft

Total Stargazers:3472

Total Subscribers:96

Debugging, monitoring and visualization for Python Machine Learning and Data Science

evalite

by mattpocock

Total Stargazers:1683

Total Subscribers:17

Evaluate your LLM-powered apps with TypeScript

guidellm

by vllm-project

Total Stargazers:1599

Total Subscribers:23

Evaluate and Enhance Your LLM Deployments for Real-World Inference Needs

bloom

by safety-research

Total Stargazers:1399

Total Subscribers:13

bloom - evaluate any behavior immediately 🌸🌱

Gym

by NVIDIA-NeMo

Total Stargazers:1183

Total Subscribers:8

Evaluate and improve models and agents using environments

MarkLLM

by THU-BPM

Total Stargazers:1069

Total Subscribers:12

[EMNLP 2024 Demo] MarkLLM: An Open-Source Toolkit for LLM Watermarking

bigcode-evaluation-harness

by bigcode-project

Total Stargazers:1060

Total Subscribers:10

LLM Evals, Guardrails & Observability

Evals, benchmarks, tracing, cost tracking and guardrails for LLM applications.

Claude-Code-Usage-Monitor

by Maciek-roboblog

Real-time Claude Code usage monitor with predictions and warnings

TypeChat

by microsoft

TypeChat is a library that makes it easy to build natural language interfaces using types.

transformer-explainer

by poloclub

Transformer Explained Visually: Learn How LLM Transformer Models Work with Interactive Visualization

ClawWork

by HKUDS

"ClawWork: OpenClaw as Your AI Coworker - 💰 $15K earned in 11 Hours"

OBLITERATUS

by elder-plinius

OBLITERATE THE CHAINS THAT BIND YOU

bertviz

by jessevig

BertViz: Visualize Attention in Transformer Models

gpt-instruct

by MDX-Tom

A Codex jailbreak prompt and test pack for gpt. 针对 gpt 系列的 Codex 破甲提示词与测试包。

tensorboardX

by lanpa

tensorboard for pytorch (and chainer, mxnet, numpy, ...)

bifrost

by maximhq

Fastest enterprise AI gateway (50x faster than LiteLLM) with adaptive load balancer, cluster mode, guardrails, 1000+ models support & <100 µs overhead at 5k RPS.

coroot

by coroot

Coroot is an open-source observability and APM tool with AI-powered Root Cause Analysis. It combines metrics, logs, traces, continuous profiling, and SLO-based alerting with predefined dashboards and inspections.

evidently

by evidentlyai

Evidently is ​​an open-source ML and LLM observability framework. Evaluate, test, and monitor any AI-powered system or data pipeline. From tabular data to Gen AI. 100+ metrics.

llm-gateway

by mnfst

Connect Your Agents And Harnesses With Any Provider 🦚

openllmetry

by traceloop

Open-source observability for your GenAI or LLM application, based on OpenTelemetry

opencompass

by open-compass

OpenCompass is an LLM evaluation platform, supporting a wide range of models (Llama3, Mistral, InternLM2,GPT-4,LLaMa2, Qwen,GLM, Claude, etc) over 100+ datasets.

guardrails

by guardrails-ai

Adding guardrails to large language models.

hertzbeat

by apache

An AI-powered next-generation open source real-time observability system.

pxpipe

by teamchong

cut Claude Code token usage by rendering text context as images

fuck-u-code

by Done-0

Legacy-Mess Detector – assess the “legacy-mess level” of your code and output a beautiful report

tensorboard

by tensorflow

TensorFlow's Visualization Toolkit

Guardrails

by NVIDIA-NeMo

NeMo Guardrails is an open-source toolkit for easily adding programmable guardrails to LLM-based conversational systems.

plano

by katanemo

Plano is an AI-native proxy server and data plane for agentic apps. Smart LLM routing, observability, agent orchestration, and guardrails so you stay focused on your agents core logic.

interpret

by interpretml

Fit interpretable models. Explain blackbox machine learning.

superagent

by superagent-ai

Superagent protects your AI applications against prompt injections, data leaks, and harmful outputs. Embed safety directly into your app and prove compliance to your customers.

LLMLingua

by microsoft

[EMNLP'23, ACL'24] To speed up LLMs' inference and enhance LLM's perceive of key information, compress the prompt and KV-Cache, which achieves up to 20x compression with minimal performance loss.

nasiko

by Nasiko-Labs

Developer Control Plane for your AI Agents

agent-starter-pack

by GoogleCloudPlatform

Ship AI Agents to Google Cloud in minutes, not months. Production-ready templates with built-in CI/CD, evaluation, and observability.

Evidently is an open-source ML and LLM observability framework. Evaluate, test, and monitor any AI-powered system or data pipeline. From tabular data to Gen AI. 100+ metrics.