DataLoader is a generic utility designed to serve as part of an application's data fetching layer, providing a simplified and consistent API over various remote data sources such as databases or web services through batching and caching mechanisms. Originally developed by Facebook in 2010 as the "Loader" API, it became a foundational component of Facebook's Ent framework and ultimately underpinned their GraphQL server implementation. The JavaScript implementation in this repository represents a port of that original concept optimized for Node.js services.
The primary purpose of DataLoader is to reduce the number of requests sent to backend systems by coalescing multiple individual data load requests into single batch operations. When using DataLoader, an application creates loader instances by providing a batch loading function that accepts an array of keys and returns a Promise resolving to an array of corresponding values. DataLoader automatically collects all individual load requests that occur within a single frame of execution (a single tick of the event loop) and presents them together to the batch function. This approach allows developers to decouple unrelated parts of their application without sacrificing the performance benefits of batch loading. For instance, a naive application might issue four separate round-trips to a backend, but with DataLoader the same operation typically requires at most two requests.
The batching mechanism operates with specific constraints to maintain correctness. The batch function must return an array of values with the same length as the input keys array, with each value at a given index corresponding to the key at that same index. DataLoader handles cases where backend services return results in different orders or omit results for certain keys, automatically reordering responses to maintain proper alignment. The framework also supports custom batch scheduling through the batchScheduleFn option, allowing developers to control when batch requests are dispatched rather than relying on the default event loop-based scheduling.
Caching in DataLoader operates at the per-request level through an in-memory memoization cache. After a key is loaded once, subsequent loads with the same key return the cached value without triggering additional batch function calls. However, the resulting Promise still waits for the current batch to complete, ensuring that both cached and uncached requests resolve simultaneously. This optimization prevents cascading requests when dependent loads occur. DataLoader is explicitly not intended to replace distributed caching systems like Redis or Memcache; rather, it serves as a request-scoped cache to prevent redundant data loading within a single request context. Instances are typically created per request in web server environments to prevent cached data from one user appearing in another user's response.
The repository is written in JavaScript and is frequently used when implementing GraphQL services with graphql-js, though its utility extends to other situations requiring efficient data fetching. According to GitGenius activity tracking, the repository has experienced median issue and pull request response latency of approximately 22935.5 hours, with enhancement requests being the most common issue type tracked. Key contributors including saihaj, drodil, and loucadufault have driven development, while the repository maintains connections with major projects including Microsoft's VSCode and TypeScript implementations as well as the Rust language repository through overlapping contributor networks. DataLoader assumes a JavaScript environment with global ES6 Promise and Map classes, available in all supported Node.js versions.
