- Thursday, Apr 2nd, 2026
| TITLE | Implementation of a Combined Web Crawler and Scraper for General Applications |
|---|---|
| ABSTRACT | The goal is to build a smart internet tool that acts like a highly focused research assistant, automatically finding and organizing specific information from across the web. To do this efficiently, it uses promising links to focus on valuable content. Once it visits a page, it employs adaptable techniques to extract exact data, such as prices or reviews, and organizes it into a structured format. The end result is a powerful and flexible system that supports a wide range of uses, from tracking competitor prices to collecting data for academic research. It is designed to navigate common web challenges, being polite to servers, avoiding duplicate information, and handling dynamic content. Crucially, the system is architected to be smart and respectful, empowering users to make informed decisions based on publicly available web data. |
| AUTHOR | Mohammed Zaki Zaheer Khan, M. Gopi Vardhan, U. Rakesh, Nama Vikram, K. Anand Department of Computer Science and Engineering, Spoorthy Engineering College, Hyderabad, India |
| VOLUME | 13 |
| DOI | DOI:10.15680/IJARETY.2026.1302017 |
| 17_Implementation of a Combined Web Crawler and Scraper for General Applications.pdf | |
| KEYWORDS | |
| References | [1] F. Menczer, G. Pant, and P. Srinivasan, “Web Crawling,†2003. [2] “Playwright Documentation,†https://playwright.dev/. [3] “MySQL Official Documentation,†https://dev.mysql.com/doc/. |
Copyright © IJARETY 2023 All Rights Reserved.