Support for Page Pagination via Numeric Ranges or Dynamic Page Numbers
TKMB MN Thái Trần
"Currently, many websites do not utilize traditional 'Next Page' buttons or 'Load More' functionality for pagination. Instead, they often display a series of sequential page numbers (e.g., 7197, 7198, 7199) alongside total page information (e.g., Page 7.198/7199). The page number is often reflected directly in the URL as a parameter (e.g., .../Property?page=X).
To significantly enhance the tool's capability to scrape data from such websites, I propose a feature that allows users to:
Specify a Numeric Page Range: Enable users to input a start page and an end page (e.g., from page 351 to page 7198). The tool should then automatically iterate through and access all URLs within this range by incrementally changing the page parameter in the URL (e.g., .../Property?page=351, .../Property?page=352, ..., .../Property?page=7198).
Automatic Recognition and Clicking of Page Number Elements: As an alternative or complementary method, if direct URL manipulation is not feasible, the tool should be able to automatically identify and "click" on HTML elements representing these numeric page links (such as the numbers 7197, 7198, 7199 shown in the example image) to navigate to the next desired page.
Option to Set Total Pages: Provide an option for the user to manually input the total number of pages (e.g., 7199). This would allow the tool to automatically generate URLs or a sequence of clicks from page 1 (or a user-defined start page) up to the specified total page number.
This feature would be invaluable for automating data collection from websites that employ non-standard pagination structures or dynamic data loading based on sequential page numbers.