Systems

Google - behind the scenes

Lorcan 1 min read

My colleague Thom Hickey pointed me at this presentation from Jeff Dean, a Distinguished Engineer in Google’s Systems Lab. It provides an overview of how Google works and looks at some of the engineering issues it faces moving forward. Two things struck me. The first was simply, again, the scale of their operation and the engineering challenges that that poses. The second was also about scale: this time about the service benefits that they can derive from that scale. For example, he describes clustering based on machine learning approaches. At the moment, the ‘intelligence’ that they create is buried in their systems and services – see how they bring together Google News stories, or match ads to web pages. It will be interesting downstream to see if they find a market for services based on this ‘intelligence’. A few moment’s reflection turns up many such services. A very simple example: they could provide a ‘Google thesaurus’ (along the lines of Roget’s thesaurus).

Search is one of the most important applications used on the internet, but it also poses some of the most interesting challenges in computer science. Providing high-quality search requires understanding across a wide range of computer science disciplines, from lower-level systems issues like computer architecture and distributed systems to applied areas like information retrieval, machine learning, data mining, and user interface design. I’ll describe some of the challenges in these areas, discuss some of the applications that Google has developed over the past few years. I’ll also highlight some of the systems that we’ve built at Google, including GFS, a large-scale distributed file system, and MapReduce, a library for automatic parallelization and distribution of large-scale computation. Along the way, I’ll share some interesting observations derived from Google’s web data. Jeff Dean joined Google in 1999 and is currently a Distinguished Engineer in Google’s Systems Lab. [Colloquium Detail]

The presentation takes about an hour to play.

Share
Comments
More from LorcanDempsey.net
The technology career ladder
Institutions

The technology career ladder

Library leaders should be drawn from across the organization. Any idea that technology leaders are overly specialised or too distant from general library work is outmoded and counter-productive.
Lorcan 7 min read
icon

Lorcan Dempsey dot Net

The social, cultural and technological contexts of libraries, services and networks

Great! You’ve successfully signed up.

Welcome back! You've successfully signed in.

You've successfully subscribed to LorcanDempsey.net.

Success! Check your email for magic link to sign-in.

Success! Your billing info has been updated.

Your billing was not updated.