LLAMA2 LLM port for Sega Dreamcast

[gameblabla]

Hello guys,
i saw the llama98.c project that was ported to Windows 98 and i couldn't help but feel like this should be possible on the Dreamcast as well.
Turns out yes, and it's decently fast too ! (at least on the smallest LLM model available)
The LLM provided with it is quite limited and can only generate small stories (TinyStories) but if someone ever releases another smaller model that's more flexible, it should be doable as well.
My initial idea was to have some kind of GUI for ChatGPT but when i saw this project, i had to give this a shot

For now i only tried on flycast but this should work on real hardware.

Download it here :
https://github.com/gameblabla/llama2c_D ... es/tag/1.0

[celsowm]

How many token per second?

[gameblabla]

How many token per second?

For 262K model, it's currently around 30~50 tokens per second. For a 15M, i would imagine it would be around 1k/s (that's how it performed on a Pentium 2) so that's why i didn't try it. Next time would be to try something in between like TinyChat (which is around 1776k parameters) or maybe even train my own with my own dataset. Who knows? (I forwarded this to the EXO discord server so we'll see if they have any suggestions)

[pmp174]

Make the terminal green like the computer from courage the cowardly dog.

[pmp174]

Wait I just thought of something fun with this. Set the model to be able to makeup random information and simply lie to the user. A misinformation bot.

There has to be a way to also make this into game. Maybe even like a dungeon crawler or something.

[gameblabla]

Wait I just thought of something fun with this. Set the model to be able to makeup random information and simply lie to the user. A misinformation bot.

There has to be a way to also make this into game. Maybe even like a dungeon crawler or something.

Yeah my hope eventually is to have some kind of fast local LLM that can generate dialogue for other game devs to make use of.
Right now though, this isnt too suitable for that