AI

Hallucination and collections

I asked HuggingChat about library collections. It is definitely hallucinating in its responses.
Lorcan Dempsey 3 min read
Hallucination and collections
The facilitated collection

I realise that my thinking on library collections may not be as well formed as it could be, but really ...? 😉

When I experiment with Chatbot interfaces I tend to have some standard queries. I ask whether the concept of the Irish diaspora is a helpful one in the context of either British or US politics. I also ask about library collections, in particular about the three models I have been writing and speaking about in recent years: collective collections, the inside out collection, and the facilitated collection.

These are the oddest results I have received from my chatbot experiments. Maybe they are having a day off? Maybe I missed a necessary configuration? Maybe my thoughts on library collections are completely incoherent?

I was trying Hugging Chat and it was using the OpenAssistant LLaMa 30B SFT 6 model.

First off, what is the collective collection?

OK, so much for that. What about the inside out collection?

mmm ... OK, and the facilitated collection?

It is of course easy to poke fun. And other results with other models can be so impressive. I do not know what is going on here ... perhaps it is a temporary glitch.

GPT-4 does somewhat better. First with a terse prompt.

A reminder that contra some expectations, GhatGPT is not a search engine. Then with some more guidance ...

This shows the importance of the prompt, giving a fuller answer than the first. Although this puts me in my place ...

It is interesting that it offers an interpretation based on the text itself. That said, Bard does 'know' more ...

And finally ...

I would not pose the inside out collection as competitive with the acquired collection, and I wonder is this partly influenced by somebody else writing about it.

I haven't done a deeper dive to see what I can find out about what is going on here. I note that my website is not part of the Google C4 collection analysed by The Washington Post.

Added 16 May 2023. OpenAI has just expanded the capabilities of ChatGPT to include web browsing. This is an important step. Here is ChatGPT on the collective collection with web browsing switched on:

And here for the first time I actually get named. It is also interesting to see the three collecting modes mentioned, although maybe the second should be collective collections?

Also interesting to see a name-check here. This is an OK answer, but I was especially interested to see the last paragraph which I thought did a good job without directly copying anything I am aware of.

Incidentally, I was interested to come across this article by Leo Lo recently: The CLEAR path: A framework for enhancing information literacy through prompt engineering.

Coda: I had asked HuggingChat about the Irish diaspora in a previous question. The 'memory' of that may have been interfering.

Share
Comments
More from LorcanDempsey.net
Document chat: an early AI use case
AI

Document chat: an early AI use case

AI-assisted document query and summarization services are becoming more common. This post compares the 'ask a PDF' outputs of several services. Overall responses were good, with reasonable variation, and no significant hallucinations. It also touches on some characteristics of AI-based services.
Lorcan Dempsey 24 min read
icon

Lorcan Dempsey dot net

Deep dives and quick takes: libraries, society, culture and technology

Great! You’ve successfully signed up.

Welcome back! You've successfully signed in.

You've successfully subscribed to LorcanDempsey.net.

Success! Check your email for magic link to sign-in.

Success! Your billing info has been updated.

Your billing was not updated.