Systems

Hwaet! Such swift search, SRU on a cluster

Lorcan 1 min read

My colleagues write about parallel text searching on a Beowulf Cluster in the current DLib Magazine:

This article describes our experience building a scalable, relatively inexpensive, and fast searching framework that demonstrated 172 searches per second on a database of 50 million records. The article should be of interest to anyone seeking an inexpensive, open source, text-searching framework that scales to extremely large databases. The technology described uses the SRW (Search/Retrieve Web) service in a manner nearly identical to federated searching in the metasearch community and should be of interest to anyone doing federated searching. [Parallel Text Searching on a Beowulf Cluster using SRW]

And a topical conclusion:

There is reason to be concerned about the efficiency of SRW and SOAP-based Web Services as opposed to SRU and REST-style services, at least in high-throughput multi-threaded clients. [Parallel Text Searching on a Beowulf Cluster using SRW]

Share
More from LorcanDempsey.net
Two metadata directions
Systems

Two metadata directions

Metadata practice continues to evolve as research and cultural practices diversify. After a brief environmental view, I discuss two important metadata trends here: entification and pluralization.
Lorcan 16 min read
icon

Follow along

Deep dives and quick takes. Libraries, society, culture, technology, ...

Great! You’ve successfully signed up.

Welcome back! You've successfully signed in.

You've successfully subscribed to LorcanDempsey.net.

Success! Check your email for magic link to sign-in.

Success! Your billing info has been updated.

Your billing was not updated.