Hallucination and collections

I asked HuggingChat about library collections. It is definitely hallucinating in its responses.
Lorcan 3 min read
Hallucination and collections
The facilitated collection

I realise that my thinking on library collections may not be as well formed as it could be, but really ...? 😉

When I experiment with Chatbot interfaces I tend to have some standard queries. I ask whether the concept of the Irish diaspora is a helpful one in the context of either British or US politics. I also ask about library collections, in particular about the three models I have been writing and speaking about in recent years: collective collections, the inside out collection, and the facilitated collection.

These are the oddest results I have received from my chatbot experiments. Maybe they are having a day off? Maybe I missed a necessary configuration? Maybe my thoughts on library collections are completely incoherent?

I was trying Hugging Chat and it was using the OpenAssistant LLaMa 30B SFT 6 model.

First off, what is the collective collection?

OK, so much for that. What about the inside out collection?

mmm ... OK, and the facilitated collection?

It is of course easy to poke fun. And other results with other models can be so impressive. I do not know what is going on here ... perhaps it is a temporary glitch.

GPT-4 does somewhat better. First with a terse prompt.

A reminder that contra some expectations, GhatGPT is not a search engine. Then with some more guidance ...

This shows the importance of the prompt, giving a fuller answer than the first. Although this puts me in my place ...

It is interesting that it offers an interpretation based on the text itself. That said, Bard does 'know' more ...

And finally ...

I would not pose the inside out collection as competitive with the acquired collection, and I wonder is this partly influenced by somebody else writing about it.

I haven't done a deeper dive to see what I can find out about what is going on here. I note that my website is not part of the Google C4 collection analysed by The Washington Post.

Added 16 May 2023. OpenAI has just expanded the capabilities of ChatGPT to include web browsing. This is an important step. Here is ChatGPT on the collective collection with web browsing switched on:

And here for the first time I actually get named. It is also interesting to see the three collecting modes mentioned, although maybe the second should be collective collections?

Also interesting to see a name-check here. This is an OK answer, but I was especially interested to see the last paragraph which I thought did a good job without directly copying anything I am aware of.

Incidentally, I was interested to come across this article by Leo Lo recently: The CLEAR path: A framework for enhancing information literacy through prompt engineering.

Coda: I had asked HuggingChat about the Irish diaspora in a previous question. The 'memory' of that may have been interfering.

More from
AI, OpenAI and the evolving AI environment

AI, OpenAI and the evolving AI environment

For a few days recently, the news was dominated by reports of organizational upheaval at OpenAI. Here are some reflections on what happened, with some connections to library topics.
Lorcan 12 min read

Lorcan Dempsey dot Net

The social, cultural and technological contexts of libraries, services and networks

Great! You’ve successfully signed up.

Welcome back! You've successfully signed in.

You've successfully subscribed to

Success! Check your email for magic link to sign-in.

Success! Your billing info has been updated.

Your billing was not updated.