Create an executable scraper in Python that can extract the SKU (product) list from the following Indonesian online grocery apps:
- Astro
- Sayurbox
- Segari
The scraper must be IP-block tolerant, meaning if the scraper gets blocked, execution can be continued from the point where the last row of data was successfully extracted.
Scraper Functions
Function: list_category_tree
Input: competitor_name(astro | sayurbox | segari), existing_csv (optional - CSV file to append to for continuing execution, if the scraper stopped or got IP blocked)
Output: CSV of category tree with the following columns (scrape_date, URL, category name, position index, parent_category (null if top level))
Function: extract_sku
Input: competitor_name, location (autocomplete place from google maps API), category (optional - extract SKUs from all categories if null), sub_category (optional - extract all SKUs from all subcategories if null), existing_csv (optional - CSV file to append to for continuing execution, if the scraper stopped or got IP blocked)
Output: CSV of SKUs with the following columns (category, sub_category, brand, sku_name, description, competitor_name, datetime, location, url, nsp, psp, stock_count (if available), popularity_index (if available), weight (if available). If one SKU is contained in more than one category/sub_category, present the same SKU in different rows.
Unit tests
The scraper must include unit tests that cover main and error test cases, including optional parameters
Submission
The following must be submitted:
- Scraper code and unit tests
- README file for running and deploying the scraper
- Output of all SKUs from the 3 online grocery apps specified above
Project Acceptance Testing
The scraper output will be manually verified within 24 hours of submission and SKU list must be 95% accurate. The SKU list is the most important aspect of the scraper output.
Variations in other data for each SKU like stock count, popularity index and prices may vary and higher levels of discrepancy (~25%) during manual verification is acceptable.
I am an experienced Python developer with expertise in creating executable scrapers. I can build a reliable scraper in Python to extract SKU lists from Indonesian online grocery apps like Astro, Sayurbox, and Segari. The scraper will be IP-block tolerant and will include functions to list category trees and extract SKUs with the required columns. Moreover, I will ensure comprehensive unit tests and fulfill all submission requirements.
Hi David D., How are you doing?
As a professional developer with expertise in PHP, Google App Engine, Software Architecture, Python and Web Scraping, I eagerly anticipate the opportunity to complete this project for you. Please drop me a message to discuss the project detail.
Thank you for considering my services.
As a highly versatile freelancer with extensive experience in Python and PHP, I am confident in my ability to create the dynamic, IP-block tolerant scraper that you require for your Indonesian grocery app extraction project. Not only will my vast coding experience enhance the quality of your executable scraper, but my meticulous attention to detail will ensure its accuracy remains above 95 percent. Additionally, my familiarity with both C# and ASP.NET will enable me to create the necessary unit tests to cover all bases.
My forte lies in delivering efficient tools that maximize outcomes while minimizing headaches. For instance, using Python, I built an automated web application that extracted and organized data from multiple sources for a comprehensive report generation. Comparable to your current venture, this required IP-block tolerance and error resilience paired with high-precision outputs.
In sum, I'm a seasoned professional who excels in executing complex tasks efficiently. The combination of my technical skills, project management prowess and diligent work ethic sets the stage for a successful partnership. Let's commence this venture together to revolutionize your data gathering process!
As a leading member and founder of Leads Lab, I bring to the table not only my excellent Python and Web Scraping skills that are crucial for your project, but also my extensive background in data extraction and manipulation- essential for building an efficient data conductor like this scraper. I am aware of just how touchy online scraping can be, especially when it's important to maintain an unbroken flow of data even in the face of IP blockages.
With my extensive experience in web development, I am confident that I can create the Python scraper you need to extract the SKU (product) lists from multiple online grocery apps. My proficiency in PHP provides a solid foundation for building reliable and functional scripts like the one you've described. Moreover, my ability to design efficient and responsive websites complements the task at hand - ensuring the scraper's seamless execution.
Throughout this project, I will continually update the CSV file with each successful extraction while logging any errors encountered to facilitate troubleshooting processes. As part of your requirements to ensure data accuracy, I pledge to: a) prioritize SKU extraction accuracy by not exceeding a 5% discrepancy rate; and b) maintain high accuracy in other data such as prices even if discrepancies up to 25% are permitted. In me, you'll find a capable hand dedicated to developing a high-quality-and perceptive-result for your urgent requirement.
Dear David D.
This project seems a very fit work for me.
I had done exactly the same work a month ago.
I would like to grab this opportunity and will work till you get 100% satisfied with my work.
I have many years of experience on PHP, Python, Web Scraping, Software Architecture, Google App Engine
Please come over chat and discuss your requirement in a detailed way.
Thank You
Hello David D.,
I hope this message finds you well. I came across your project and was immediately intrigued. I am fully confident that I am the best fit for this job.
With years of experience in PHP, I've successfully navigated similar projects. My expertise in PHP, Python, Web Scraping, Software Architecture, Google App Engine ensures efficient and high-quality results aligned with your goals.
I value collaboration and will actively involve you in the process to exceed your expectations. Timely delivery is crucial, and I'm committed to meeting deadlines while maintaining exceptional quality.
I invite you to join me for a chat where we can delve deeper into your requirements and discuss your project in detail. Additionally, I'll showcase relevant examples of my work, allowing you to gain insight into my skillset and how it aligns with your needs.
Thank You
Hafiz Asif
Hi, Hope you are doing great today. > Please read it carefully, this is not a generic proposal <
I noticed that you are seeking a Urgent Mobile App Data Extractor Needed. I would like to take this opportunity to share with you that I possess a variety of PHP, Python, Web Scraping, Software Architecture, Google App Engine and have completed numerous projects throughout my career. I am confident that I can deliver the results you need for your project with the same level of professionalism and expertise.
Please keep in mind that I am available to discuss the details of the project with you at any time through chat. Additionally, you can view my profiles to get to know the skills and expertise I possess
https://www.freelancer.com/u/rannasteve
As part of my commitment to delivering high-quality work, I will assign a dedicated Project Manager to your project to provide daily progress reports. You can also count on 24-hour chat support, so time differences won't be an issue. I will ensure that the project is completed within the timeline that we mutually agreed upon before the start of the project. Finally, I offer a one-month maintenance period after the project is completed.
I look forward to hearing back from you and discussing how we can work together to bring your project to life.
Best regards,
Steven
