noctis.data_transformation.preprocessing.utilities._merge_partition_files

noctis.data_transformation.preprocessing.utilities._merge_partition_files(filename: str, tmp_dir: str, output_dir: str, n_partitions: int, prefix: str | None = None)

Merge partition files into a single CSV file.

Args:

filename (str): Name of the file to merge. tmp_dir (str): Temporary directory containing partition files. output_dir (str): Directory to save the merged CSV file. n_partitions (int): Number of partitions to merge. prefix (Optional[str]): Optional prefix for the merged file name.

Note:

Writes headers only once and logs warnings if partition files are missing.