TensorWatch is a debugging and visualization platform developed by Microsoft Research that enables real-time monitoring and analysis of machine learning training processes directly within Jupyter Notebook environments. The tool is designed to work with Python 3.x and supports PyTorch versions 0.4 through 1.x, with most features also compatible with TensorFlow eager tensors. It requires graphviz for creating network diagrams, which may need manual installation depending on the user's platform.
The core functionality of TensorWatch centers on its stream-based architecture, where nearly every component including files, sockets, consoles, and visualizers operate as streams. This design enables the creation of complex data flow graphs where visualizers can listen to multiple streams simultaneously, and streams can recursively extend to build arbitrary configurations. This decoupling of streams from storage and visualization mechanisms provides significant flexibility in how data is processed and displayed.
TensorWatch supports a diverse range of visualization types including line graphs, histograms, pie charts, scatter charts, bar charts, and three-dimensional versions of these plots. Users can combine multiple visualizations, display a single stream across multiple plots simultaneously, or arrange arbitrary sets of visualizations side-by-side. The platform also allows users to create custom visualization widgets by implementing a Python class with specific methods, enabling extensibility for specialized use cases.
A distinctive feature of TensorWatch is its Lazy Logging Mode, which allows users to query live running processes without requiring data to be logged beforehand. Users can send Python lambda expressions as queries to retrieve results as streams and direct these to preferred visualizations. This capability enables interactive debugging where users can sample and visualize data dynamically during training, such as randomly sampling input and output image pairs from an autoencoder without prior logging.
The tool facilitates experiment comparison by allowing each stream to contain metrics of the user's choice. While TensorWatch saves all streams in a single file by default, users can configure individual stream files or disable file storage entirely, directing streams over sockets or to the console with zero disk overhead. This flexibility enables quick comparison of results from different experimental runs across various visualization types.
TensorWatch is classified across multiple domains including visualization, tensor logging, model training, performance tracking, metrics dashboarding, experiment logging, and model monitoring. It serves as an alternative to TensorBoard with additional capabilities for debugging and interactive analysis. The repository is primarily written in Jupyter Notebook format and is actively maintained as a development tool under heavy development.
The platform includes important security considerations documented in its README. It features Lazy Logging that executes arbitrary Python expressions via eval on network-supplied data, pickle deserialization for ZeroMQ messages and files, and YAML deserialization through bundled hiddenlayer utilities. Mitigations include HMAC-SHA256 message signing, localhost-only binding by default, and a RestrictedUnpickler using allowlists of permitted modules. Users are responsible for keeping HMAC keys secret, avoiding exposure of ports to untrusted networks, and only loading trusted data files.
