Asynchronous Web Scraping and Data Aggregator

OVERVIEW

The objective of this project is to aggregate data from various websites related to real estate listings and property info. The results should be aggregated into a single CSV which can be easily viewed in Excel.

REQUIREMENTS

1. The program should be configurable to limit results based on price, property type, and city including its surrounding area

2. Individual websites should have ability to be enabled/disabled in the configuration settings

3. Datapoints which are scraped must be normalized and/or converted to the same units of measurement so they can be easily and meaningfully sorted in Excel

4. Furthermore, the program must be able to send asynchronous requests to the websites to speed up execution time

5. All code must be thoroughly commented and easy to understand for future modifications and debugging

6. Code must also be robust to continue working even if minor website layout changes occur

7. Program must also produce meaningful logs to debug and troubleshoot should problems arise in the future

8. Upon completion of the project, an instructional video (screen recording with captions and/or audio narration) and/or detailed instruction manual should be provided to demonstrate the key features and functionality of how the program works as well as the code’s structure

Please see attached document for detailed project description including datapoints and sources to scrape.

Please answer the following questions when submitting a proposal:

1.) Do you have any questions about the job description?

2.) Do you have suggestions to make this project run successfully? How do you plan on improving the execution speed when scrapping thousands of listings from several websites?

3.) What past project or job have you had that is most like this one and why?

4.) Which of the required job skills do you feel you are strongest at?

5.) What programming language, libraries, and frameworks do you plan on using for this project and why do you choose them VS available alternatives?

BeautifulSoup Python Scrapy Selenium Web Scraping

หมายเลขโปรเจค: #25853299

เกี่ยวกับโปรเจกต์