server : forward llama.cpp sampling params on /v1/messages (#30856)

* server : forward llama.cpp sampling params on /v1/messages

* server : pass the request through on /v1/messages instead of copying listed keys

* NIT: Change comment from multiline to single line

* server : accept llama.cpp params under extra_params on /v1/messages

* server : remove test case for trivial functionality

* server : show extra_params in the /v1/messages README example

tools/server/README.md

@@ -1719,6 +1719,8 @@ See [Anthropic Messages API documentation](https://docs.anthropic.com/en/api/mes
 
 `top_k`: Top-k sampling
 
+`extra_params`: llama.cpp parameters that the Anthropic API has no field for, such as `seed`, `min_p` or `repeat_penalty`, as an object (e.g. `{"seed": 42, "min_p": 0.05}`). Its keys are merged into the converted request and behave as on `/v1/chat/completions`.
+
 `stop_sequences`: Array of stop sequences
 
 `stream`: Enable streaming (default: false)
@@ -1739,7 +1741,8 @@ curl http://localhost:9931/v1/messages \
     "system": "You are a helpful assistant.",
     "messages": [
       {"role": "user", "content": "Hello!"}
-    ]
+    ],
+    "extra_params": {"seed": 42, "min_p": 0.05}
   }'
 ```
 

tools/server/server-chat.cpp

@@ -615,6 +615,13 @@ json server_chat_convert_anthropic_to_oai(const json & body) {
         }
     }
 
+    // llama.cpp params (e.g. seed, min_p) go under extra_params and are merged into the converted body
+    if (body.contains("extra_params") && body.at("extra_params").is_object()) {
+        for (const auto & [key, value] : body.at("extra_params").items()) {
+            oai_body[key] = value;
+        }
+    }
+
     return oai_body;
 }