It may be tempting to think the solution for an LLM project is nothing more than deciding where to place the query input field, but there is definitely an opportunity to bring some product thinking to an LLM solution.
As with any product, start by asking: What is the problem to be solved? Your organization may have a specific problem they need solved (easier access to HR policies), or it may just be that the solution has been identified (we need an LLM!!) and it’s now in search of a problem to be solved. Regardless of the directive, product discovery skills still apply.
The Discovery Kickoff
First, who will be using this product? Is this a tool for company employees only? All employees or a specific business unit? Will outside people, such as customers, have access? Will the user base need it to be multilingual?
With your user set identified, find out their needs and concerns. If in-person chats aren’t feasible, use a survey to learn about their experience and comfort level with using Gen AI (such as ChatGPT). Include a brief explanation of what an LLM can do along with potential problems it can solve, and have the users rank them in order of relevance. Tell them to think of an LLM as their personal intern and ask them to identify any tasks they’d move off their plate to that intern.
(I need to inject a side note here: If you’re not familiar enough with LLMs to speak about their benefits and how they can help your users, now is the perfect time to lean into our free LLM Crash Course. It’s quick, it’s easy, and it’ll give you a good foundation.)
Defining Features and User Stories
The discovery learnings when combined with the business needs and goals will help to define and refine the feature set: Will a history of chats be needed? Will access be per user or are chats visible to all? Can users share chats? Should the LLM be segmented into different instances, each with their own personality and focus (such as Marketing, IT Support, etc.)? Will there be admins who can set up new instances?
Adding in Design
Now that the features and user stories are established, let’s think about design. If this is a standalone product, i.e., not incorporated into an existing app such as Slack, you will need a UI and yes, an input field for the query. How and where you place any additional elements on the page will be based on the users’ comfort level with this new technology. Nonetheless, take this as an opportunity to educate the users on how to use LLMs. Make available hints on how to write a query (btw, it isn’t always a question).
If you’re showing the prompt field along with the query, include info about creating a prompt and why it’s used. Conversely, hide the prompt behind a link along with advanced settings such as temperature or top_p. Your users can immediately use the LLM without being overwhelmed by seeing multiple settings, while the more curious can play around with the other settings to see how they’ll affect the response.
In Summary
It’s a very exciting time to be designing digital products as leaps in technology don’t happen every day. Learn as much as you can about Gen AI/LLMs but remember, product skills and knowledge still apply in defining the problem and finding solutions.
Alice Toth
