Repository Issue Activity (beta)

#1944 Performance snapshot: Ballista vs Spark/Comet for TPC-H @ SF100

#1959 Performance snapshot: Ballista vs Comet vs Spark @ SF100 — join strategy (SMJ vs ShuffledHashJoin)

#1958 Add shuffle-read (fetch) metrics to ShuffleReaderExec and surface per-operator metrics in executor logs

#1943 Shuffle fetch exhausts ephemeral ports at high target_partitions (client connection caching off by default)

#1375 Investigate use of `DynamicFilters` in ballista

#1953 Reuse repeated exchanges/subplans in the distributed planner (ReuseExchange/ReuseSubquery analog)

#1952 Spill large shuffle partition fetches to disk to bound decoded in-flight memory

#1950 Adaptive query planning fails with remote object stores: "No suitable object store found"

#1938 Add a versioned Upgrade Guide to the documentation

#1937 Add suport for `DataFrame.checkpoint()`

#1940 Restore backward-compatible submit_job signature after #1924

#1829 Add support for `DataFrame.cache()` to Ballista

#1923 Implement equivalent of Spark History Server

#1887 Improve python/Cargo.lock drifts - CI deps

#1795 Improve job failure handling

#1679 Extend ColocatedJoinRule and BroadcastSmallSideRule to SortMergeJoinExec

#1922 Broadcast the build side of a SortMergeJoinExec without converting to a hash join

#1920 Track DataFusion git main via a pinned rev and bump it periodically

#1845 Add TPC-DS workflow

#1909 DataFusion 54: uncorrelated scalar subqueries (q11/q15/q22) fail — ScalarSubqueryExpr cannot be deserialized in a split stage plan

#1907 DataFusion 54: DataSourceExec shared work-queue makes each Ballista scan task read the whole table (wrong results + ~Nx slowdown)

#1776 Update to datafusion 54

#1055 Left/full outer join incorrect for CollectLeft / broadcast

#1910 Execute uncorrelated scalar subqueries first and substitute the value, instead of rewriting to a join

#1908 Scheduler marks executor dead on a deterministic task launch/decode failure, hanging the job