Hi @ai6667,
I am handling this dynamically now, to avoid going over the maximum amount allowed by the model. I wanted to see if that works well, and it does.
So basically, now you can use the standard max_tokens parameter and that will work 🙂
That said, I have also implemented the fact that only the last 15 questions/answers are kept and used in the prompt.
That number of 15 could also be changed. I forgot what we said before but in this case now, it could be “conversation_buffer” and 15 by default. What do you think?
Thread Starter
Anonymous User 18191698
(@anonymized-18191698)
Being able to set the max tokens per prompt and inputting only the last 15 sounds ok. Although, if the user could change the number in the bot settings, that would be ideal. I do have one problem now.
Thread Starter
Anonymous User 18191698
(@anonymized-18191698)
Hi. Do you think you can add the conversation buffer as a parameter because it would be very useful to control this on a per bot basis? For example, sometimes one would require no buffer for purely text in /text out jobs such as translation, grammar fixing and so on.
Thanks
It’s already in the new version 🙂 Please use it like this:
[mwai_chat max_sentences="10"]
