Crawling library resources

Lorcan 1 min read

A colleague points me to Tony Boston’s discussion of how the National Library of Australia has exposed NLA resources for crawling and indexing by search engines.

The National Library of Australia is making digital copies of special collection materials available over the Internet. About 100,000 collection items including pictures, maps, sheet music, manuscripts, and some books and serials have been made available online. This content is delivered dynamically from a database developed to manage the Library’s digital collections. Since 2002 the Library has been exposing this content to Internet search engines to increase access to the material and provide multiple discovery pathways for Library users. This paper documents lessons learned in exposing the deep web and presents statistics on increased web usage focussing particularly on the Library’s Pictures Collection. Application of technologies which can be used to share deep web content such as the Open Archives Initiative Protocol for Metadata Harvesting are also explored. [Exposing the deep web to increase access to library collections]

More from
The technology career ladder

The technology career ladder

Library leaders should be drawn from across the organization. Any idea that technology leaders are overly specialised or too distant from general library work is outmoded and counter-productive.
Lorcan 7 min read

Lorcan Dempsey dot Net

The social, cultural and technological contexts of libraries, services and networks

Great! You’ve successfully signed up.

Welcome back! You've successfully signed in.

You've successfully subscribed to

Success! Check your email for magic link to sign-in.

Success! Your billing info has been updated.

Your billing was not updated.