Xuan-Son Nguyen
7adf160c41
Copilot Setup Steps / copilot-setup-steps (push) Has been cancelled
Check Pre-Tokenizer Hashes / pre-tokenizer-hashes (push) Has been cancelled
Python check requirements.txt / check-requirements (push) Has been cancelled
Python Type-Check / python type-check (push) Has been cancelled
Update Operations Documentation / update-ops-docs (push) Has been cancelled
* init * wip * wire it up * make it clear that it's experimental * add props.connect_code * nits * fix windows build * server: do not advertise a connect code without the tunnel --connect-code on its own had no effect other than being served in /props, so the Web UI showed a remote access code that no tunnel was serving. Clear it when --connect is off, and say so once at startup. The teardown comment claimed that terminate() closes the child stdout. It does not: the log thread leaves fgets() only once every writer of that pipe is gone, so a descendant of llama-connect would block the join forever. Describe the real contract instead. Also restore the --reasoning-preserve default in the generated docs. * Update tools/connect/CMakeLists.txt Co-authored-by: Adrien Gallouët <angt@huggingface.co> * server: dial llama-connect on the bound port and a reachable host Pass ctx_http.port so --port 0 and the GCP override work, pick the dial host from --host (wildcards map to loopback, IPv6 is bracketed) and refuse --connect when only unix sockets are bound. --------- Co-authored-by: Pascal <admin@serveurperso.com> Co-authored-by: Adrien Gallouët <angt@huggingface.co>
40 lines
1.1 KiB
C++
40 lines
1.1 KiB
C++
#pragma once
|
|
|
|
#include "common.h"
|
|
|
|
#include <atomic>
|
|
#include <memory>
|
|
#include <string>
|
|
#include <thread>
|
|
|
|
struct common_subproc;
|
|
|
|
// spawns llama-connect, which exposes this server to a remote browser over WebRTC
|
|
// core binary is in a separate project: https://github.com/ggml-org/llama-connect
|
|
struct server_connect {
|
|
server_connect();
|
|
~server_connect();
|
|
|
|
server_connect(const server_connect &) = delete;
|
|
server_connect & operator=(const server_connect &) = delete;
|
|
|
|
// path of the llama-connect binary, empty if not found
|
|
static std::string find_binary();
|
|
|
|
// why --connect cannot work here, empty if it can
|
|
static std::string unavailable_reason(const common_params & params);
|
|
|
|
static std::string resolve_code(const std::string & code);
|
|
|
|
// port is the bound one, it differs from params.port with --port 0 or the GCP override
|
|
bool start(const common_params & params, int port);
|
|
|
|
// idempotent, also called by the destructor
|
|
void stop();
|
|
|
|
private:
|
|
std::unique_ptr<common_subproc> proc;
|
|
std::thread log_thread;
|
|
std::atomic<bool> stopping{false}; // tells the log thread the exit is expected
|
|
};
|