align

FileSet.align(other, start=None, end=None, matches=None, max_interval=None, return_info=True, compact=False, skip_errors=False)[source]

Collect files from this fileset and a matching other fileset

Warning

The name of this method may change in future.

This generator finds the matches between two filesets (the files that overlap each other in time), reads their content and yields them. The reading is done in parallel threads to enhance the performance.

Parameters

  • other – Another fileset object.

  • start – Start date either as datetime object or as string (“YYYY-MM-DD hh:mm:ss”). Year, month and day are required. Hours, minutes and seconds are optional. If not given, it is datetime.min per default.

  • end – End date. Same format as “start”. If not given, it is datetime.max per default.

  • matches – A list of matches between this fileset and other. Normally, this is just the return value of match(). These matches will be used for aligning. If matches is given, start, end and max_interval will be ignored.

  • max_interval – A time interval (as string, number or timedelta object) that expands the search time period for secondaries.

  • return_info – Additionally, to the file content also its info object will be returned.

  • compact – Not yet implemented. Decides how the collected data will be returned.

  • skip_errors – Normally, when an exception is raised during reading a file, the program stops. But if this parameter is True, the error will be only printed as warning and this and its matching file are skipped.

Yields

If return_info False, it yields two objects – the primary and secondary file content.

fileset = FileSet(
    "old/path/{year}/{month}/{day}/{hour}{minute}{second}.nc",
)

# Delete all files in this fileset:
fileset.delete()