gfw.common.beam.pipeline.LinearDagFactory

class LinearDagFactory(config)

Abstract base class for factories producing LinearDag objects.

Methods

build_dag

Builds a LinearDag instance from the configured pipeline parts.

Attributes

bigquery_helper_factory	Returns a factory for BigQueryHelper instances.
core	Returns the core PTransform for data processing.
read_from_bigquery_factory	Returns a factory for ReadFromBigQuery clients.
side_inputs	Returns optional side inputs for the core PTransform.
sinks	Returns the sink PTransforms to write data outputs.
sources	Returns the source PTransforms`.
write_to_bigquery_factory	Returns a factory for WriteToPartitionedBigQuery clients.

abstract property sources: Tuple[PTransform, ...]

Returns the source PTransforms`.

Returns:

Tuple of PTransforms serving as data sources.

abstract property core: PTransform

Returns the core PTransform for data processing.

property side_inputs: PTransform | None

Returns optional side inputs for the core PTransform.

abstract property sinks: Tuple[PTransform, ...]

Returns the sink PTransforms to write data outputs.

build_dag()

Builds a LinearDag instance from the configured pipeline parts.

Returns:

A LinearDag composed of sources, core, side inputs, and sinks.

Return type:

LinearDag