Immediate Need For Web Scraping Project
$25-50 USD / ชั่วโมง
I have an immediate need for .NET developers to collect (scrape) real estate property and tax data from over 130 public websites in the US. We need to immediately add these sites to our existing server environment described below, with an additional 1,000 sites required immediately following the successful implementation of this project. Beyond that we anticipate to have many more contracts to follow based on our prior performance. The goal is to have the first 130+ sites coded within 30 to 45 days, with incremental deliverables along the way. We are inviting multiple individuals or teams to request an opportunity to participate. Each qualified individual/team that is selected will be awarded two sites to collect data from. Upon successful implementation, determined by the quality of data returned and the timeliness of completion, we will award additional contracts in incremental or bulk volume. The successful candidates that deliver the best performance will be invited to participate on future contracts.
The company I represent has an existing web application running on Windows Server 2012. The application accepts CSV and Excel XLS(X) files that contain search parameters and keys for input used by the scraping applications. These input files are uploaded by our client and converted to jobs in a database. When the client decides to start the saved job, the Job Manager begins executing as many multiple instances of scraper applications as needed, or possible, and instructs them as to which properties it needs to collect data for. The scraper applications are written as .NET service applications that support multi-threaded execution so that many instances of the same county scraping application can be running at the same time. Once the scraper application is done collecting its data it provides it back to the Job Manager as a CSV, XLS(X) or an XML file in the prescribed standard format. The data is then parsed and written into a SQL Server database by another service application and stored there until the client requests the output in CSV, XLS(X), XML or JSON formats, either through the web interface or through our custom API.
Our current scraping applications are written in Visual Basic .NET and Visual C# .NET, and these languages are our first preference. However, other language options will be considered if they are compatible with the .NET framework, can be executed as a service application and/or fit within the existing architecture with little or no modification. Candidates having experience working with any of the following web technologies will be considered first:
• Captchas • Flash
• Java • ASPX
• Flex • Silverlight
• Proxy IP rotation • Image download (.jpg, .png, .bmp, etc.)
• Direct API calls
If you are interested in this project please submit your proposal for development along with relevant experience. Please include which languages you are able to code in along with your years of experience for each, your availability to work on this effort in regards to starting date and number of hours a week you are available to work, which of the web technologies mentioned above you are experienced in working with, and whether or not you are working as an individual or have a team of developers that can develop multiple of these applications at the same time.
Please note that this is listed as an hourly project. We are required to specify a starting hourly range but are willing to compensate appropriately based on performance of the initial task. We will provide more detail about the sites that need to be scraped to all those who reply that have responded appropriately by establishing their relevant experience and qualifications. We will entertain a fixed price bid on future contracts after a candidate has demonstrated their successful implementation on this initial two site effort.
หมายเลขโปรเจค: #7373762
เกี่ยวกับโปรเจกต์
freelancer freelancer 31 คน กำลังเสนอราคาในงานนี้ โดยมีราคาเฉลี่ยอยู่ที่ $30/ชั่วโมง
Hello, I have 12 years of experience in various technologies. I have worked for several years on Flex project QuoteStation, ViaOne CRM and Hero. I have developed several Silverlight projects. The largest Silverlight เพิ่มเติม
Dear customer, I created a lot of scrapers so far, please check my records. I use C# and HtmlAgilityPack to grab data from web sites. Regards Armando
Dear Sir, I'm very much delighted to let you know that i did data scraping with PHP-cURL, Node.js, Selenium from many sites. I just scraped the data from web site and then wrote the data in mysql database or excel o เพิ่มเติม
Hi.. Expert Web Scraper & Data Minor here. I have done too many similar project in past. Having best scraping tools and experience i assure you 100% accurate and good quality work. I have too too scraping experience. y เพิ่มเติม
I have extensive experience in .net and scrapers having delivered many projects I like to discuss this one.
Hi, Please feel free to discuss the project with me ........................................................................... Thanks, Murtaza
lowest bidder lets start the work.............................................................................................. discuss more in pm
Hi there, I'm running an offshore development team in China. Our main skills are ASP.Net, C#, JavaScript related and we have previous experience about website crawling, so I believe we are able to help you. Also we เพิ่มเติม
Hello I have experiences with web scraping by c# and I can update my program for your needs. You can see it through my profile Thank you.
Hi, Is it possible to meet you to understand in brief your requirement? With Regards, Vishal Vishwakarma +919833005478
i have got good experience in .net and sql i am dedicated towards work . will ensure you delivery in time
Hi, As you have detail describe the job need and current process you are following in threaded environment, I think I'm a good candidate for this project, Since I have same experience working with a USA client for s เพิ่มเติม
