Hi,
Not yet, but I will add a new Max Tokens field soon. Currently, it’s set automatically to the maximum allowed by the model. By decreasing that number, however, it might cut the reply from the AI before it’s finished, this is what you want?
Thread Starter
Anonymous User 18191698
(@anonymized-18191698)
But wouldn’t it cut it at any rate? I thought that these models were trained to give a complete answer based on x number of tokens. I suppose having control over the number of tokens is always good because we can control costs.
I thought that these models were trained to give a complete answer based on x number of tokens.
It works differently; the answer is only based on the prompt. Max Tokens will not change the length of the reply, it will only cut it 🙂 However, using a smaller number for Max Tokens might increase the ability of the model to generate the answer.
