paint.data.dataset

Attributes

log

A logger for the calibration dataset.

Classes

PaintCalibrationDataset

Initialize a PaintCalibrationDataset.

Module Contents

paint.data.dataset.log: A logger for the calibration dataset.

class paint.data.dataset.PaintCalibrationDataset(root_dir: str | pathlib.Path, item_type: str, item_ids: list[int] | None = None)

Bases: torch.utils.data.Dataset

Initialize a PaintCalibrationDataset.

This dataset contains calibration data from the paint dataset.

Parameters

root_dirstr | Path: Directory where the dataset will be stored.
item_typestr: Type of item being loaded, i.e. raw image, cropped image, flux image, flux centered image, or calibration properties.
item_idslist[int], optional: List of item IDs that should be included in the dataset. If no list is provided, all files in that folder of the specified type will be loaded.

mapper

file_identifier

root_dir

item_ids = None

to_tensor

__str__() → str: Generate a user-friendly representation of the dataset.

static _check_accepted_keys(key: str) → None

Check if the considered calibration item type is accepted.

Parameters

keystr: Key of the calibration item type to be checked.

classmethod from_benchmark(benchmark_file: str | pathlib.Path | pandas.DataFrame, root_dir: str | pathlib.Path, item_type: str, download: bool = False, timeout: int = 60, num_parallel_workers: int = 10, results_timeout: int = 300) → tuple[Self, Self, Self]

Initialize calibration dataset from a benchmark file.

This function returns a train, test, and validation dataset given a benchmark.

Parameters

benchmark_filestr | Path | pd.DataFrame: Path to the file containing the benchmark information, or dataframe containing this information.
root_dirstr | Path: Directory where the dataset will be stored.
item_typestr: Type of item being loaded, i.e. raw image, cropped image, flux image, or calibration properties.
downloadbool: Whether to download the data (Default: False).
timeoutint: Timeout for downloading the data (Default: 60 seconds).
num_parallel_workersint: Number of parallel workers for downloading data (Default: 10).
results_timeoutint: Timeout for collecting results from multiple threads (Default: 300 seconds).

Returns

PaintCalibrationDataset: Train dataset.
PaintCalibrationDataset: Test dataset.
PaintCalibrationDataset: Validation dataset.

static _download_benchmark_splits(splits: pandas.DataFrame, root_dir: str | pathlib.Path, item_type: str, timeout: int = 60, num_parallel_workers: int = 10, results_timeout: int = 300) → dict[str, Any]

Download the benchmark splits.

This is a helper function that downloads the data for the benchmark splits. It also returns a dictionary containing information about the item IDs present in each split.

Parameters

splitspd.DataFrame: Information on the splits to be downloaded.
root_dirstr | Path: Directory where the data will be stored.
item_typestr: Type of item being downloaded, i.e. raw image, cropped image, flux image, or calibration properties.
timeoutint: Timeout for downloading individual items in a split.
num_parallel_workersint: Number of parallel workers for downloading data (Default: 10).
results_timeoutint: Timeout for collecting results from multiple threads (Default: 300 seconds).

Returns

dict[str, Any]: Information about the item IDs present in each split.

classmethod from_heliostats(root_dir: str | pathlib.Path, item_type: str, heliostats: list[str] | None = None, download: bool = False) → Self

Initialize calibration data set based on heliostats.

This class method initializes a calibration data set based on specific heliostats from the PAINT database.

Parameters

root_dirstr | Path: Directory where the dataset will be stored.
item_typestr: Type of item being loaded, i.e. raw image, cropped image, flux image, or calibration properties.
heliostatslist[str] | None: List of heliostats for which calibration data should be downloaded. If no list is provided the data for all heliostats will be downloaded (Default: None).
downloadbool: Whether to download the data (Default: False).

Returns

PaintCalibrationDataset: Calibration dataset based on one or more heliostats.

static _download_heliostat_data(root_dir: str | pathlib.Path, item_type: str, heliostats: list[str] | None = None, timeout: int = 60, num_parallel_workers: int = 10, results_timeout: int = 300) → None

Download the heliostat calibration data for the dataset based on heliostats.

Parameters

root_dirstr | Path: Directory where the data will be stored.
item_typestr: Type of item being downloaded, i.e. raw image, cropped image, flux image, or calibration properties.
heliostatslist[str] | None: List of heliostats for which calibration data should be downloaded. If no list is provided the data for all heliostats will be downloaded (Default: None).
timeoutint: Timeout for downloading heliostat data (Default: 60 seconds).
num_parallel_workersint: Number of parallel workers for downloading heliostat data (Default: 10).
results_timeoutint: Timeout for collecting results from multiple threads (Default: 300 seconds).

__len__() → int

Calculate the length of the dataset.

Returns

int: Length of the dataset.

__getitem__(idx: int) → torch.Tensor | dict[str, Any]

Return the item at position idx.

Parameters

idxint: Index of the item to be retrieved.

Returns

torch.Tensor | dict[str, Any]: Either the image as a torch.Tensor or a dict containing the calibration properties, depending on the item type selected.