src.page module

The Page class provides a way to fetch HTML and parse it with Beautiful Soup.

class src.page.Page(url: str, session: ClientSession)

Bases: object

async classmethod create(url: str, session: ClientSession) → Page

Factory method. Used to create a Page instance, but avoid creating a new aiohttp session upon each instantiation.

Parameters:

• url (str) – URL whose content to fetch.

• session (aiohttp.ClientSession) – aiohttp.ClientSession to use to get the provided URL.

Returns:

Return a Page instance configured with the provided url and session.

Return type:

Page

async fetch() → ClientResponse | None

Retrieve self.url.

property internal_links: List[str]

Return a list of all internal a.href references in the instance’s HTML.

property links: List[str]

Return a list of all a.href references in the instance’s HTML.

async src.page.main()