Customizing Chatbots and agents - the GPT
OpenAI has made some big announcements about enhancements recently, including the ability to create custom chatbots, called GPTs (which seems like an unnecessarily confusing name?). These can be created through a natural language 'builder' interface, can be asked to carry out particular 'actions', and can work with uploaded specialised data. I built a simple example – Libraries: collaboration and collections – and discuss it further below (see an example response here).
Here is how OpenAI describes it:
We’re rolling out custom versions of ChatGPT that you can create for a specific purpose—called GPTs. GPTs are a new way for anyone to create a tailored version of ChatGPT to be more helpful in their daily life, at specific tasks, at work, or at home—and then share that creation with others. For example, GPTs can help you learn the rules to any board game, help teach your kids math, or design stickers.
Anyone can easily build their own GPT—no coding is required. You can make them for yourself, just for your company’s internal use, or for everyone. Creating one is as easy as starting a conversation, giving it instructions and extra knowledge, and picking what it can do, like searching the web, making images or analyzing data. // OpenAI
At the moment you can 'publish' the static outputs of a GPT to the world (see below), but only 'Plus' subscribers can ask questions of the Chatbot themselves. After the recent organizational upheaval at OpenAI, the Store where one could access custom GPTs has been put on hold. OpenAI also recently paused the creation of new paid 'Plus' ChatGPT accounts, as it works through challenges of increased usage.
Nevertheless, this seems to be important news. It opens the door to the creation of a large number of niche chatbots, combining the LLM capacities of GPT-4 with specialised data sources, and the ability to add some simple directions about behaviors.
It is also a reminder of how difficult it is to predict where things are going. How popular will they be? Will OpenAI be able to scale its infrastructure to support thousands of them? Will they prove useful tools or are they a passing diversion? Could they take off like the Apple app store, which may be the aspirational model, or will they turn out like Alexa skills, individually potentially useful but too much effort to discover and use broadly? I don't know, but it signals how things can suddenly go in a new direction. As I note further below, they also come with performance and policy challenges which may delay further rollout.
Influential researcher and commentator Ethan Mollick sees great potential. He provides an overview of how to build a GPT and notes how it will influence his practice as a teacher and author.
The power here is pretty obvious. I will be creating custom GPTs for every session of the classes I teach. Some will be simulations for students to experience, some will be tutors or mentors, some might even be teammates or assignments. I have been turning my research into GPTs, so that anyone can get advice on how to generate ideas or pitch a business idea by getting feedback from a GPT to which I have given my books as a reference. And I expect this will become a trend in many places, as schools and government agencies and companies build libraries of GPTs that are specialized in solving particular problems in useful ways. // One Useful Thing
Some issues
GPTs are clearly not ready for prime time from a performance point of view. I experienced similar performance issues to those described by Brian Mathews as he worked on a GPT based on CNI transcripts. Document loads appeared to fail (I am not sure if I successfully loaded all that I tried to), it was slow, and responses sometimes fall over before completing.
It also raises important service, policy and social issues, for which they don't really seem prepared. OpenAI has an engineering focus, but as it extends service offerings like this, it will have to evolve policy frameworks around creation and use of GPTs. Will GPTs which claim to be by a particular person be verified? Will you be allowed to create one based on somebody else's publicly available outputs? Will there be any filtering for toxic or misleading GPTs? What documentation will be available about GPTs? What thresholds will GPT creators need to meet to be listed? The recent example of social media has not been good, and it is an area where self-regulation may not be enough.
So, again this could potentially amplify current problematic issues. One could see further fragmentation and polarization of sources of opinion and commentary, alongside more purposeful disinformation, inexpert sources of advice, and so on. But there may also be interesting and useful niche services based on carefully curated specialist materials.
Given the recent events at OpenAI, the discussion about speed of development, and the performance and usage issues they are facing, I would not be surprised if fuller deployment of custom GPTs ended up being pushed even further out. One could see a more robust business offering, where media and communication companies, or others, could offer customised services. What would it mean to have a Martha Stewart chatbot, for example, or a Brené Brown one? Would it be of interest to them or to others? And there is a more general discussion, aligned with the social media discussion, about what accountabilities should be in place for platform companies which distribute content created by users of the platform.
Trying it out
I was curious to try it out, so I created a modest GPT: Libraries: Collaboration and Collections. Some example responses are included below. This is based on a selection of my papers and blog entries mostly about, yes, collaboration and collections. It is very simple - I have not added any actions to it: all it does is answer questions. This particular GPT is only meant as an illustration of what can be done with a few hours work. It is naturally limited in scope and usefulness. However, it indicates a direction.
So, here is an example. I asked it why library collaboration is so hard, and this was the response, also pasted in here:
Library collaboration is a complex and challenging endeavor for several reasons:
1. Diverse Institutional Priorities and Cultures: Libraries often operate within different institutions (like universities, communities, or regions), each with its own unique culture and set of priorities. Aligning these varied interests towards a common goal can be difficult.
2. Resource Allocation and Funding: Collaboration typically involves sharing or reallocating resources, which can be challenging, especially when budgets are tight. Libraries may be reluctant to reallocate resources away from local needs to shared initiatives.
3. Differing Levels of Infrastructure and Technology: Variability in technological infrastructure and digital capabilities among libraries can make it difficult to establish efficient collaboration. This is particularly challenging when trying to integrate systems or share digital resources.
4. Logistical Challenges: Effective collaboration requires coordination of policies, practices, and systems across different libraries. This can be logistically complex, particularly in large-scale collaborations spanning multiple regions or countries.
5. Governance and Decision-Making: Establishing a governance structure that is agreeable to all parties, and that facilitates efficient decision-making, can be challenging. Balancing local autonomy with the collective decision-making necessary for collaboration can be a delicate process.
6. Change Management: Collaboration often requires changes in traditional practices and roles, which can be met with resistance. Managing this change effectively, while maintaining staff engagement and morale, can be a significant challenge.
7. Preservation of Local Identity and Services: Libraries may be concerned about losing their unique identity or diluting the quality of their local services as they engage in broader collaborative efforts.
8. Legal and Policy Constraints: Collaborations often have to navigate complex legal and policy landscapes, especially when dealing with copyright issues, data privacy, or cross-border collaborations.
In summary, while library collaboration offers many potential benefits, including enhanced resource sharing, increased efficiency, and expanded access to information, the process is often impeded by a mix of organizational, cultural, logistical, and technological challenges.
Here are a couple more examples:
- How does the concept of the inside-out collection align with current research and library trends? ↗
- Can you briefly (300 words) outline differences between library collaboration in the UK and the UK? ↗
If you have a 'Plus' ChatGPT subscription, you should be able to ask it questions yourself. It doesn't know very much though. And tread softly ...
Finally, here are some observations about my experience, relating it to more general AI contexts.
First, I needed to spend some time adjusting the prompt. The 'builder' constructs a prompt to guide responses, which you can then edit directly, or ask the builder to make changes. Using the GPT was a bit frustrating at first as it kept answering every question with reference to 'collective collections,' perhaps because of a concentration of that topic in the supplied documents. I tried adjusting the prompt to shift this emphasis, and with more time I am sure I could improve it further. It would probably benefit from additional data and by spending some more time experimenting with the prompt. I ended up explicitly asking it not to talk about collective collections unless the question was explicitly about that topic.
Second, I was not always clear about how the custom GPT worked with GPT-4, as it clearly draws on the general resource. For example, responses from the custom GPT are still shaped by library stereotypes, especially when generating images. I discussed this in relation to the general GPT-4 in my last post. The latter is not too surprising given how LLMs work, and it does say something about how libraries tell their stories and, more importantly, get these stories into general currency. However, I would have expected much greater mitigation in the custom GPT given the explicit focus on libraries, library futures, and so on, but that did not seem to be the case.
Third, while it was quite straightforward, uploading the files was a little tedious. At first I thought it would accept URLs, and fetch the web pages. But that turned out not to be the case; you need to provide the file in a format it understands (HTML, PDF, etc.). It also quite often told me that a load failed. I was not sure whether it meant that the document had not been imported. Nor have I been able to successfully ask it what it has uploaded.
It was an interesting experience building this, despite the immaturity of the tools. It will be interesting to see how much effort OpenAI devotes to this. While OpenAI opened this door, I hope some others go through it. It would be interesting to have, say, Hugging Face, or a group with an interest in scholarly or cultural communication offer some similar capability with more transparency and documentation.
Note on other experiments: Brian Mathews describes his experience creating a prototype GPT – CNI Insights Agent – based on transcripts of CNI meetings in several blog posts. David Lankes describes creating a personalised chatbot — Virtual Virtual Dave – using poe.com here.
Acknowledgements: Much of what I wrote over the last twenty years was in my role at OCLC, and supported its mission of advancing libraries and librarianship. My former OCLC colleagues Brian Lavoie and Constance Malpas are co-authors on several of the documents I uploaded. Thanks to Brian Lavoie, Constance Malpas and Brian Mathews for kindly commenting on a late draft.
Picture: As I note, the picture is generated by DALL·E 3 (within ChatGPT) in response to a prompt asking for an image of an abstract shelf of books based on the style of a Sean Scully picture.
Related posts: See my series of longer posts for a fuller discussion of AI. I plan a fourth in the series at some stage looking at potential impact on library services.
Lorcan
