Concurrency Limits is a Java library that implements adaptive concurrency control by applying TCP congestion control concepts to automatically detect optimal request limits for services.
The library addresses the problem that traditional rate-limiting based on requests per second becomes outdated in auto-scaling distributed systems and fails to gracefully shed load. Instead of thinking in terms of RPS, the tool applies queuing theory to determine how many concurrent requests a service can handle before queues build up and latency increases. It estimates concurrency limits at each point in the network by measuring latency to detect when queuing occurs and using timeouts or rejected requests to back off aggressively. The approach treats a system's concurrency limit like a TCP congestion window, allowing each node to adjust its local limit as the system scales. The library provides multiple limit algorithms: Vegas, a delay-based algorithm that increases the limit by one per sampling window if estimated queue depth stays below a threshold and decreases it if the queue exceeds a higher threshold, and Gradient2, which tracks divergence between long and short-term exponential latency averages to smooth out outliers and identify queueing trends for aggressive limit reduction.
The tool suits teams operating large distributed systems where RPS limits become stale quickly due to auto-scaling, and where graceful load shedding matters more than absolute throughput. It works best when services have measurable latency characteristics and when operators want to avoid the complexity of manually determining hardware-specific concurrency limits. The library provides enforcement strategies including a simple gauge-based approach that rejects requests once inflight count reaches the limit, and a percentage-based strategy for more nuanced request handling.
The project shows sustained development activity with regular commits addressing both new features and maintenance. The codebase receives updates that refine the limit algorithms and expand enforcement options. Pull requests indicate ongoing refinement of the core congestion control logic and integration patterns. The project maintains documentation of its algorithms and design rationale, suggesting attention to helping operators understand the approach rather than treating it as a black box.
