Ummm... did you try /set parameter num_ctx # and /set parameter num_predict #? Are you using a model that actually supports the context length that you desire...?
theunknownmuncher
joined 9 months ago
Ummm... did you try /set parameter num_ctx # and /set parameter num_predict #? Are you using a model that actually supports the context length that you desire...?
I just stick to AMD, especially on Linux. The official AMD driver is open source on Linux, included in mainline kernel, and performance is better than their Windows diver now