Reward Offered:

Brownie Points

This question is Open

To close the question - pick the best answer from the list below and select "Reward Expert"

what type of a database searching do websites like trip advisor and money supermarket perform?


Asked by : garycoffey
Date asked : 31st October, 2010


Categories:

, ,

Diabled Comment Button

Answers

36

Admin

Probably easy to say it depends ... But if you are talking about how sites 'harvest' or 'scrape' data from multiple sources.. then can be done in two main ways..

  • Using a API (Application Provider Interface) to query the providers data directly and get the appropriate information
  • Using a Screen Scraper - basically creating a piece of software that 'pretends' to be a human making a request and then captures the results

By using multi-threaded environments and clever caching approaches it is possible to make hundreds and thousands of requests to build up a picture of all of the 'prices' and products available from multiple suppliers
I can explain in more detail if this would help

1 year ago