cata_log_hub.providers.base module

class cata_log_hub.providers.base.Provider[source]

Bases: ABC

Abstract base class for all catalog providers.

_registry: ClassVar[dict[str, type[Provider]]] = {'action-de': <class 'cata_log_hub.providers.action.Action'>, 'aldi-nord-de': <class 'cata_log_hub.providers.aldi_nord.AldiNord'>, 'aldi-nord-de-preview': <class 'cata_log_hub.providers.aldi_nord.AldiNordPreview'>, 'aldi-sued-de': <class 'cata_log_hub.providers.aldi_sued.AldiSued'>, 'aldi-sued-de-preview': <class 'cata_log_hub.providers.aldi_sued.AldiSuedPreview'>, 'aldi-sued-de-preview-2': <class 'cata_log_hub.providers.aldi_sued.AldiSuedPrepreview'>, 'edeka-markt-de': <class 'cata_log_hub.providers.edeka.EdekaMarkt'>, 'hofer-au': <class 'cata_log_hub.providers.hofer.Hofer'>, 'hofer-au-preview': <class 'cata_log_hub.providers.hofer.HoferPreview'>, 'kaufland-de': <class 'cata_log_hub.providers.kaufland.KauflandWoche'>, 'kaufland-de-preview': <class 'cata_log_hub.providers.kaufland.KauflandWochePreview'>, 'kaufland-sonder-de': <class 'cata_log_hub.providers.kaufland.KauflandSonder'>, 'kik-de': <class 'cata_log_hub.providers.kik.Kik'>, 'lidl-de': <class 'cata_log_hub.providers.lidl.LidlDeutschland'>, 'lidl-de-preview': <class 'cata_log_hub.providers.lidl.LidlDeutschlandPreview'>, 'lidl-de-preview-2': <class 'cata_log_hub.providers.lidl.LidlDeutschlandPrepreview'>, 'lidl-it': <class 'cata_log_hub.providers.lidl.LidlItalia'>, 'metro-de': <class 'cata_log_hub.providers.metro.MetroWochenangebote'>, 'metro-de-preview': <class 'cata_log_hub.providers.metro.MetroWochenangebotePreview'>, 'netto-de': <class 'cata_log_hub.providers.netto.Netto'>, 'netto-de-preview': <class 'cata_log_hub.providers.netto.NettoPreview'>, 'norma-de': <class 'cata_log_hub.providers.norma.Norma'>, 'norma-de-preview': <class 'cata_log_hub.providers.norma.NormaPreview'>, 'norma-de-preview-2': <class 'cata_log_hub.providers.norma.NormaPrepreview'>, 'norma-de-retrospect': <class 'cata_log_hub.providers.norma.NormaRetrospect'>, 'norma-de-retrospect-2': <class 'cata_log_hub.providers.norma.NormaRetrospect2'>, 'penny-de': <class 'cata_log_hub.providers.penny.Penny'>, 'penny-de-preview': <class 'cata_log_hub.providers.penny.PennyPreview'>, 'rewe-de': <class 'cata_log_hub.providers.rewe.Rewe'>, 'rewe-de-preview': <class 'cata_log_hub.providers.rewe.RewePreview'>, 'rossmann-aktion-de': <class 'cata_log_hub.providers.rossmann.RossmannAktion'>, 'rossmann-beilage-de': <class 'cata_log_hub.providers.rossmann.RossmannBeilage'>}

uid: str: A unique identifier for this provider class. Must not be changed after definition.

name: str: The name of this catalog provider.

description: str: The user-facing description for this provider

url: str: The url of this catalog provider

region: type[Region]: The region this catalog is issued in

first_page_number: int = 1: The number of the first page in the providers api

schedule: str = '0 4 * * *': The crontab schedule for fetching this provider

jitter: int = 3600: Jitter for the schedule timing in seconds.

class Configuration[source]

Bases: BaseModel

The configuration for this provider.

model_config: ClassVar[ConfigDict] = {}: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

final __init__(configuration)[source]

Constructor for a provider instance.

Parameters:

configuration (dict[str, str]) – The configuration dictionary for the provider.

Variables:

_configuration – An instance of the cata_log_hub.providers.base.Provider.Configuration loaded from :param:`configuration` .
_client – HTTPX client instance that throws on every error status and follows redirects.
_relevant_datetime – The relevant datetime defining the catalog to cache.

Return type:

None

_cleanup()[source]

Additional cleanup.

Return type:: None

get_relevant_datetime()[source]

Get the datetime that defines the catalog offered by this provider. The current datetime for current catalogs, a future datetime for preview catalogs. Used to set _relevant_datetime.

Return type:: datetime
Returns:: The current datetime in the providers regional timezone.

abstractmethod _get_valid_since()[source]

Get the datetime since which this providers catalog is valid.

Return type:: datetime

final get_valid_since()[source]

_get_valid_since wrapped in error handling.

Raises:: cata_log_hub.exceptions.ProviderBrokenWarning – If any exception is raise in getting the valid_since timestamp.
Return type:: datetime

abstractmethod _get_valid_until()[source]

Get the datetime until which this providers catalog is valid.

Return type:: datetime

final get_valid_until()[source]

_get_valid_until wrapped in error handling.

Raises:: cata_log_hub.exceptions.ProviderBrokenWarning – If any exception is raise in getting the valid_until timestamp.
Return type:: datetime

abstractmethod _get_page(page_number)[source]

Get one page from the provider.

Parameters:: page_number (PageNumber) – The number of the page in the numbering of the provider.
Return type:: bytes
Returns:: The downloaded page in bytes.

get_page(page_number)[source]

_get_page wrapped in error handling.

Raises:

cata_log_hub.exceptions.PagesExhausted – If page_number is not first_page_number and pages are exhausted or a httpx.HTTPStatusError is raised.
cata_log_hub.exceptions.CatalogUnavailableWarning – If page_number is first_page_number and the catalog is unavailable.
cata_log_hub.exceptions.ProviderMisconfiguredOrBrokenWarning – If page_number is first_page_number and pages are exhausted or a httpx.HTTPStatusError is raised or the catalog is unavailable.
cata_log_hub.exceptions.NetworkError – If a httpx.TransportError is raised.
cata_log_hub.exceptions.ProviderBrokenWarning – If any other exception is raised.

Return type:

bytes

Parameters:

page_number (PageNumber)

abstractmethod _get_catalog_data()[source]

Get and store the data for this providers catalog. Can be passed if no data beside the pagenumber is required to fetch pages from the provider.

Return type:: None

get_catalog_data()[source]

_get_catalog_data wrapped in error handling.

Raises:

cata_log_hub.exceptions.CatalogUnavailableWarning – If the catalog is unavailable.
cata_log_hub.exceptions.ProviderMisconfiguredOrBrokenWarning – If a httpx.HTTPStatusError is raised.
cata_log_hub.exceptions.NetworkError – If a httpx.TransportError is raised.
cata_log_hub.exceptions.ProviderBrokenWarning – If any other exception is raised.

Return type:

None

final iter_catalog_pages()[source]

Iterate over pages of this providers catalog.

Return type:: Generator[tuple[int, bytes]]
Returns:: A generator of the pages data as bytes.

final classmethod get_class(class_uid)[source]

Get the provider class to a given class-id.

Parameters:: class_uid (str) – The class-id to get the provider for.
Return type:: type[Provider]
Returns:: The class to the class-id.
Raises:: cata_log_hub.exceptions.ProviderUnknownClassWarning – If there is no class for the given uid.

final classmethod get_class_uids()[source]

Get all registered class-ids.

Return type:: list[str]
Returns:: A set of all registered class identifiers.

final classmethod get_classes()[source]

Get all registered classes.

Return type:: list[type[Provider]]
Returns:: A set of all registered classes.

final classmethod validate_configuration(configuration_dict)[source]

Validate a given configuration and return the validated version.

Parameters:: configuration_dict (dict[str, Any]) – The configuration dictionary to validate.
Return type:: Configuration
Returns:: The validated configuration.
Raises:: cata_log_hub.exceptions.ProviderInvalidConfigurationWarning – If the given configuration is incomplete or otherwise invalid.

class cata_log_hub.providers.base.Preview[source]

Bases: ABC

Preview mixin for a provider class.

abstractmethod _get_preview_timedelta()[source]

Get the dynamic timedelta between the relevant datetime and the relevant datetime of the preview.

Return type:: timedelta
Returns:: The preview timedelta.

final get_preview_timedelta()[source]

Get the preview timedelta with error handling.

Return type:: timedelta
Returns:: The preview timedelta.
Raises:: cata_log_hub.exceptions.ProviderBrokenWarning – If any exception is raised.

get_relevant_datetime()[source]

Return type:: datetime

get_catalog_data()[source]

Return type:: None

get_page(page_number)[source]

Return type:: bytes
Parameters:: page_number (PageNumber)

cata_log_hub.providers.aldi_sued module

cata_log_hub.providers.edeka module