In the past few years, the use of Web data mining has become a common practice.
Many companies throughout the world on Web applications to order supplies, search professional services, to track shipments, to book travel or get to a map or driving directions. We've been doing spider-based data-mining for years. We have the tools, the bandwidth, and -- most importantly -- the experience required to get your data reliably, accurately, and quickly.
We create scripts to automate testing of their complex, highly scalable, asynchronous system involving multiple tiers and databases. We have used these scrips and applied techniques to solve business problems in credit card, finance, telecoms, online and offline retail, industrial automation, among other inds.
We can examine any number of websites, from one to millions, to any depth and Integrity. We know how to behave when spidering a site. We are never denied service for abusive access. Most sites never know they were mined at all by our team.