gfw.common.beam.pipeline.hooks#

Pipeline hooks for pre- or post-processing operations.

This module provides helper functions that generate hooks to be executed during a pipeline’s lifecycle. Hooks are callable functions that take a Pipeline object and perform arbitrary operations, such as creating views, deleting data, or any other custom task.

Functions

`create_table_hook`	Returns a hook function to create a BigQuery table.
`create_view_hook`	Returns a hook function to create a view of a BigQuery table.
`delete_events_hook`	Returns a hook function to delete events from a BigQuery table.

create_view_hook(table_config, mock=False)[source]#

Returns a hook function to create a view of a BigQuery table.

Parameters:

table_config (TableConfig) – TableConfig instance containing view details.
mock (bool) – If True, uses a mocked BQ client instead of performing real operations.

Returns:

A callable hook that accepts a Pipeline instance and creates the view.

Return type:

Callable[[Pipeline], None]

delete_events_hook(table_config, start_date, end_date=None, mock=False)[source]#

Returns a hook function to delete events from a BigQuery table.

Parameters:

table_config (TableConfig) – TableConfig object containing table detailsand delete query.
start_date (date) – Date after which events should be deleted.
end_date (date | None) – Date up to which events should be deleted (exclusive).
mock (bool) – If True, uses a mocked BQ client instead of performing real operations.

Returns:

A callable hook that accepts a Pipeline instance and deletes events.

Return type:

Callable[[Pipeline], None]

create_table_hook(table_config, mock=False)[source]#

Returns a hook function to create a BigQuery table.

Parameters:

table_config (TableConfig) – TableConfig instance containing view details.
mock (bool) – If True, uses a mocked BQ client instead of performing real operations.

Returns:

A callable hook that accepts a Pipeline instance and creates the view.

Return type:

Callable[[Pipeline], None]