So I got to thinking about all the open source llm's out there now (especially the uncensored ones) and was wondering how practical it would be to incorporate that into a visual novel game.

My thoughts on this are a few-fold:

AI lack of 'knowledge' or 'reasoning' means that while they are amazing tools for assisting in ideas, conversations and content generation, they are terrible at 'taking the wheel' themselves. So I am strictly considering them in an 'assistive' capacity.

That means that a VN using AI should, in my opinion, have a predetermined story (issues of branching or linearity aside) and that there should be more or less pre-determined scenes and possible outcomes or 'exits' in scenes.

Where AI could be used is the in-scene comunication or generation. It could be used to generate story and character responses in scene and always be trying to 'steer' a scene to one of the 'exits'.

Obviously a lot would need to be firgured out with a system like this, but I feel like this overarching concept is not unreasonable.

This would of course need to run on a players cpu or gpu to be feasible as a game in my opinion, so the quality of your language models and techniques used are going to be limited.

This is a very open ended thought experiment on my part and I am wondering what the community at large has to say about it.

I had a similar idea which I might pursue for my game. I was going to use the LLM to generate the speech text of an AI personal assistant (similar to Cortana from Halo games). I was also planning to combine this with text-to-speech and speech-to-text models to allow the player to talk with the LLM verbally.

From what I've seen, some of the Llama 2 models (e.g. 7B, 13B) are feasible for this purpose. These have been quantized to 4 bits and converted to run on regular CPUs using 3.5GB and 7GB of memory, respectively, and can generate around 10 tokens per second on a single thread which is fast enough for human speech.

I agree with what you said - the LLM needs to be guided through the story. I was going to provide prompts to the LLM to describe locations the player is visiting and to provide context. Then the LLM can sort through that information and produce an interactive conversation based on the current context.

How well this works in practice remains to be seen. Combined with good text-to-speech models it could be used to replace much of the dialog in games (e.g. random conversations with NPCs).

@undefined Incorporating open-source language models into a visual novel game, especially for scene communication and content generation, is an intriguing idea. Utilizing AI as an assistive tool for dynamic in-scene dialogues and content steering within predetermined storylines can enhance player immersion and engagement. Balancing predetermined outcomes with AI-driven interactions could offer a unique and dynamic storytelling experience. Considerations for player agency and the AI's role in shaping the narrative will be key to a successful implementation. Exciting potential

We've got a feature like this in our Brain~Wave platform being added pretty soon. We are still looking for beta testers, so maybe it's something you'd like to try out for us. I posted about it, open request for beta testers, in the indie showcase channel.

But from our experiences, with the tools available right now, one of the most important things is having a system that allows for a lot of user directionality and improvement.

CaseyAIM said:
I posted about it, open request for beta testers, in the indie showcase channel.

That post was removed. Adverts and calls for beta testers may only be posted in Your Announcements.

I think bad quality filler is worse than no filler. If you use AI to replace the effort of bad writing for dialogues, you can do it, but it will not appeal to players if you didn't tell them it's AI made. What is your UX requirement or goal for conversations with an AI? I think if you want people to really appreciate the dialogues, you need to think really long and hard about each character and what motivates him. You can do some procedural dialogue generation from hand-curated samples to get more variety, but even that can easily make text seem unnatural because the flow of nuances changes. Players will recognise if the conversations have actual intellectual depth behind their design, and an AI does not have that.

@Tom Sloper Thanks for the heads up. Damn, I thought I had a pretty good grasp on where that was supposed to go.

@RmbRT If your assertions were true, I would agree with you. Most of the publicly available/well-marketed AI solutions are indeed pretty lackluster in this regard. However, my team is currently working on one that far exceeds what I think your expectations are. It's built specifically with the goal of all the things that you're concerned about AI not having. And it's remarkable. And it's remarkable.

I think the era of wholesale dismissing AI capabilities is quickly coming to an end.

CaseyAIM said:
I think the era of wholesale dismissing AI capabilities is quickly coming to an end.

The proof will have to be given before agreeing can commence. We are not in awe of chatbots here. And our writers do not welcome the idea of being replaced by a bot. I like what rmbrt said.