Probably easy to say it depends ... But if you are talking about how sites 'harvest' or 'scrape' data from multiple sources.. then can be done in two main ways..

• Using a API (Application Provider Interface) to query the providers data directly and get the appropriate information
• Using a Screen Scraper - basically creating a piece of software that 'pretends' to be a human making a request and then captures the results

By using multi-threaded environments and clever caching approaches it is possible to make hundreds and thousands of requests to build up a picture of all of the 'prices' and products available from multiple suppliers
I can explain in more detail if this would help