Smart aggregation

A while ago I was thinking about the development of bibliographic record exchange in terms of the following progression: standardization, commoditization, aggregation, syndication. Standardization is necessary for commoditization and aggregation. Syndication involves creating service over the aggregated resource. And this progression seems to apply to other activities as well. I was interested recently to see the following two discussions in the context of Web 3.0 and Media 2.0.
Phil Wainewright describes Web 3.0 in terms of: a commodity API layer (RSS feeds, Amazon and Google APIs, …); a middle layer of aggregation services which reduce the difficulty of working with the highly distributed lower layer; and application services which operate on the middle layer to “bring together functionality from multiple services to help users achieve their objectives in a flexible, intuitive and self-evident way”.
Umair Haque talks about aggregators and smart aggregators [ppt]. Aggregators assemble and rebundle content from a fragmented space. Smart aggregators, in his terms, assemble and rebundle content, but also aggregate information, expectations and preferences about that content. This in turn allows services to efficiently allocate scarce attention through recommendation, collaborative filters, and so on.
In each case, we see a discussion of how aggregation on its own is of limited value.
In the first case there is a discussion about building applications on top of aggregate services. A couple of examples? Well an obvious and simple one relates to RSS aggregation. I mentioned a few days ago that it was tedious keeping track of many individual RSS feeds in an aggregator. However, it is actually quite useful to have Technorati ‘watchlists’ or similar services from others. In this case, a targeted search can be built over the aggregation which is valuable and saves time. Another is the type of light-weight service composition that Jon Udell was talking about in relation to library lookup and xISBN. There is a growing number of interesting services available; what will be really interesting is how they can be stitched together to create compelling new applications.
In the second case, there is a discussion of using what I have called intentional data to make the service more useful in predicting and meeting need. Examples in our domain are ranking search results on the basis of holdings data, recommending books on the basis of circulation data, making selection and presentation decisions on the basis of usage data, and so on.
As more activity moves onto the network, we begin to build up from the lower layers to create richer, more complex structures. We are beginning to see greater value created in our bibliographic applications. What will we do as we aggregate harvested metadata, digitized books, cultural heritage materials? We need to find ways of making aggregation routine, so that we can more readily build out the services that “help users achieve their objectives in a flexible, intuitive and self-evident way”.