Repository Issue Activity (beta)

#1064 [Bug]: JSONL export fails for Python date and datetime values

#1019 Fail-fast validation for single-machine deduplicators when executor_type=ray

#1039 Bound TokenNumFilter tokenizer batches to reduce default-batch memory

#1030 分区续跑时，续跑时的分区切分结果会与第一次不同，导致检查点无法生效

#1031 [Tracking] Reduce peak memory usage across Data-Juicer operators / 降低算子峰值内存

#1034 Bound MinHash permutation workspace to reduce peak memory

#1028 Reduce repetition-filter memory by streaming n-gram counts

#848 支持HDFS或者iceberg数据源

#941 HDFS connector

#1006 I have an idea to implement lineage capabilities for unstructured data—similar to the structured data lineage found in Apache Atlas or OpenLineage—within the current project

#982 data-juicer 有支持arm cpu架构吗？

#1008 Why doesn’t the analyzer support Ray mode?

#971 RayBasicDeduplicator 懒加载策略导致无法实现全局去重

#936 [Bug] JSONStreamDatasource locks first-batch schema and fails on later `null -> concrete type` evolution

#965 [Bug]: KeyError: 'text_formatter'

#964 [Bug]: use_cache is broken — auto-generated job_id in work_dir poisons fingerprints

#920 为什么datajuicer在Ray模式下，不支持groupby算子呢？

#909 ETL过程的咨询

#728 多个算子要操作的数据域不一样如何设置？

#695 data-juicer 有计划支持流式或微批计算么

#915 Multi-Branch Execution (DAG feature enhancment)

#667 如何实现两个算子的逻辑运算关系 How to implement the logical operation between two operators?

#732 什么时候data-juicer支持使用对象存储的数据集？

#476 Update of Jupyter Notebooks

#487 Checkpointer support for Ray-Mode