Some nice remarks by Peter Suber in his latest SOAN:
As I read the news, both halves of this prediction are coming true. On the one hand, mainstream, non-academic search engines are looking for ways to index scholarly content. Starting with OAI-compliant archives and CrossRef-compliant publications is a natural, both because the data are well-structured and because one negotiation can open the door to many sources. On the other hand, search engines want some quasi-exclusive access to this new content or else their rivals will have equal access to the benefit of their investment. So while Google can eventually strike a similar deal with OAIster, and can already index the surface-web content in OAIster, Yahoo is the only search engine to benefit from the direct OAIster feed of metadata. Similarly, Google will have some kind of exclusive access to the CrossRef content –just as it has with the 1,000,000+ documents in IEEE Xplore and just as it plans to have for the public-domain books from the Stanford library it will pay to digitize through Project Ocean. The only trend in the other direction is the OCLC tool, which will make DSpace metadata equally available to all indexing services. OCLC deserves our thanks for this openness, Google deserves congratulations for seizing the opportunity first, and the other search engines should jump on the bandwagon ASAP. [Suber, SPARC Open Access Newsletter, 5/3/04]
