11827 Commits

This Branch

This Branch

All Branches

Author	SHA1	Message	Date
ppAlexa	8f97d82da3	tools : reject truncated GGUF tensor data in split and export tools (#30497) ... * The problem here in the repo is gguf-split can silently produce corrupted tensor output when the input GGUF file is truncated. The fix is to add tensor-data bounds checks and short-read checks in gguf-split.cpp and export-lora.cpp, and a regression test that checks truncated input is rejected and valid tensor data is preserved.# * Bug : GGUF-split can silently produce corrupted tensor output when the input GGUF file is truncated. Fix : add short-read checks in gguf-split.cpp (in copy_file_to_file in split_strategy) and export-lora.cpp (in read_tensor_data of file_input) so that truncated input is rejected. b11727	2026-10-19 09:27:46 +03:00
Georgi Gerganov	27af58168a	tests : update Metal fusion baseline for beam (#30764) ... * tests: add beam fusion rows to MTL baseline Regenerate tests/fusion/MTL.csv with test-fusion --record. The newly added beam architecture has no baseline rows, so test-fusion fails with 2 failing rows (MUL+ADD x6, RMS_NORM+MUL x25). All other arch counts are unchanged. Assisted-by: pi:llama.cpp/ggml-org/Qwen3.8-Flash-Next-GGUF (MacBook, M5 Max, 128GB) * skills: add update-fusion-table skill Add a skill for regenerating a per-device test-fusion baseline. It keeps the workflow from tests/fusion/README.md and the parts that are easy to get wrong: diff the recorded file before applying it, the distinct meanings of a FAIL row, and never recording a partial run. Assisted-by: pi:llama.cpp/ggml-org/Qwen3.8-Flash-Next-GGUF (MacBook, M5 Max, 128GB)	2026-10-19 09:11:38 +03:00
Georgi Gerganov	7db0d3d781	pi : turn .pi/gg into a system prompt addendum (#30762) ... .pi/gg/SYSTEM.md repeated the full agent ruleset. Move it to .pi/gg/APPEND_SYSTEM.md and keep only the llama.cpp-specific rules, so it can be appended to a main system prompt with --append-system-prompt. Drop the .pi/SYSTEM.md symlink setup from wc2wt.sh and pr2wt.sh. Worktrees no longer need it now that the folder holds an addendum instead of a full prompt. Assisted-by: pi:llama.cpp/ggml-org/Qwen3.8-Flash-Next-GGUF (RTX 5090 32GB, DDR5 128GB)	2026-10-19 09:11:13 +03:00
Paul Mairhofer	99db719bdf	CUDA: fix shared-memory WAR hazard in flash_attn_ext_vec (#30573) ... * CUDA: fix KQ write-after-read race in flash_attn_ext_vec Assisted-by: ChatGPT * CUDA: apply review suggestions for KQ sync comments Co-authored-by: Johannes Gäßler <johannesg@5d6.de> --------- Co-authored-by: Johannes Gäßler <johannesg@5d6.de> b11724	2026-10-19 13:25:04 +08:00
Max Krasnyansky	31288c9c66	hexagon: FLASH_ATTN_EXT improvements for longer context and SWA models (#30731) ... * hex-fa: support for masked block skipping to reduce DMA fetching * hex-fa: finetuning the hmx->hvx crossover thresholds (boosting gemma-4) * hex-fa: update hmx gate to account for mdev.count b11723	2026-10-18 18:08:11 -07:00
Jhen-Jie Hong	fe8841a78a	hexagon: allow multistream FA (#30218) b11722	2026-10-18 18:07:53 -07:00
Aman Gupta	ff5a233767	cuda : fuse qwen4exp's SCALE -> SIGMOID -> SCALE -> hc_post chain (#30602) ... Port of the Vulkan HC_POST_GATE fusion (#29520). hc_combine builds the hc_post weights as 2*sigmoid(inject/hc); dsv4_hc_post_f32 now takes a gated template flag and computes scale_out*sigmoid(scale_in*p) from the raw weights, so the chain runs as one kernel instead of four. b11721	2026-10-19 01:48:47 +08:00
Andrea Rossetti	0878f7c304	ggml: support non-contiguous SUM (CPU, CUDA, Vulkan, WebGPU) (#28851) ... * ggml: support non-contiguous SUM (CPU, CUDA) * tests: add non-contiguous SUM cases * tests: SUM over non-contiguous views * vulkan: support non-contiguous SUM * webgpu: support non-contiguous SUM b11720	2026-10-18 18:57:36 +02:00
thelittlefireman	5aeac37a35	ggml-cuda: fattn do not check oob if k_VKQ_max = (k_VKQ_max - nbatch_fa) (#30572) b11719	2026-10-18 10:59:06 +02:00
Todd Malsbary	547208dca2	Restore ESIMD kernels on FFN path (#30239) ... * Reenable ESIMD on the FFN path Signed-off-by: Todd Malsbary <todd.malsbary@intel.com> * Extend ESIMD fusion on FFN path to include Q8_0 Signed-off-by: Todd Malsbary <todd.malsbary@intel.com> * Remove comment explicitly referencing ESIMD in test-backend-ops Signed-off-by: Todd Malsbary <todd.malsbary@intel.com> --------- Signed-off-by: Todd Malsbary <todd.malsbary@intel.com> b11718	2026-10-18 00:02:38 -04:00
Todor Boinovski	848bbda904	hexagon: add support for hadamard MUL_MAT and GET_ROW for flat and tiled Q1_0 and Q2_0 (#30627) ... * hex-hadamard: add support for hadamard ops * hex-mm: reduce pkt counts in the q8_1 quantizer and vec_dot_q1 * hex-q1/q2: add support for flat and tiled GET_ROWS for Q1_0 and Q2_0 --------- Co-authored-by: Max Krasnyansky <maxk@qti.qualcomm.com> b11717	2026-10-17 19:17:33 -07:00
Todor Boinovski	797de63b2f	hexagon: add support for Q1_0 and Q2_0 (#30122) ... * hexagon: add support for q1_0 quant * hexagon: enable Q1_0 weights in the 2D HMX MUL_MAT path * hex-q1: cleanup gates, generalize tiling and remove in-loop divs * hex-tiling: document tiled formats in developer.md and update htp-tensor.h * hex-q2: add support for Q2_0 * hex-q2: remove stale asserts and speacial cases * hex-mm: fix MUL_MAT_NX assumption about square tiling, Q1/Q2 have non-square tiles * hex-q1/2: minor optimizations (unrolls/un-unrolls/...) * hex-q1: optimize Q1_0 layout to reduce unpack/vrmpy overhead * hex-q1/q2_0: further optimize vec_dots --------- Co-authored-by: Max Krasnyansky <maxk@qti.qualcomm.com> b11716	2026-10-17 13:36:38 -07:00
thelittlefireman	9d1878a435	ggml-cuda: HIP: use __builtin_amdgcn_perm for q1_0 unpacking (#29927) b11715	2026-10-17 19:04:39 +02:00
Aleksander Grygier	c0fd786264	ui : Discover & Download Models from HF Hub (#29584) ... * common : resolve <quant>-<sidecar> download tags and list cached sidecars A Q4_0-mtp style tag now resolves the sidecar file when no model file matches it, so a solo draft or mmproj download actually pulls the file. Cached sidecar files list as their own entries so the state survives a restart, and removing such a tag deletes only the sidecar. Assisted-by: pi:zai-org/GLM-5.3 * ui : optional sanitized raw HTML in markdown Add an allowHtml prop to MarkdownContent: raw HTML found in the markdown is rendered after DOMPurify sanitization instead of being escaped as literal text. Default stays escaped. Assisted-by: pi:GLM-5.3-Flash * ui : add the discover models components The Discover surface: the curated-catalog list with its search and skeletons, the details pane with its readme, metadata, Hub stats and download options, and the standalone download progress bar that replaces the plain one. The quant download button sizes its select with a new xs trigger, so the select gains that size and the muted box look. The store stays incomplete when a repo fails, so the next mount retries it. Assisted-by: pi:llama.cpp/DeepSeek-V4.1-Flash * ui : put the discover view in the models manager The manager dialog gains its second view: the Discover Models call to action fades the table out, slides the title with its compass mark, and fades the discover surface in; the arrow button returns to the table. Assisted-by: pi:llama.cpp/DeepSeek-V4.1-Flash # Conflicts: # tools/ui/src/lib/components/ui/button/button.svelte * ui : add the Enable Discover Models setting The flag gates browsing and downloading HuggingFace GGUF models, so it is declared on the discover layer; the repo-tree lookup in the manager layer above reads it from here. Assisted-by: pi:llama.cpp/DeepSeek-V4.1-Flash * ui : resume model downloads from the pause point A stopped download keeps its partial file instead of deleting it, so re-posting the tag resumes from disk. The first progress record now carries the resume offset, the paused snapshot survives into the resumed entry, and a pause settles as soon as it is requested. Assisted-by: pi * ui : lead the discover list with downloads The discover sidebar lists running downloads above the suggested models, and steps aside while a query is active. A paused download stays in the manager table, where it is resumed or dropped. Assisted-by: pi * ui : make resuming a paused download work Re-posting the tag of a paused download hit the router's "already exists" check, because a paused download parks its entry as DOWNLOADED until the next reload: a download entry that is not running is now replaced and resumed. The UI keeps the paused row when the request still fails, instead of dropping it and looking finished. Assisted-by: pi * ui : remember unfinished model downloads A partial file is named after its content hash, so nothing on disk says which model it belongs to once the router restarts. The router now keeps the tags of the downloads it started, clears a tag when the download finishes or is deleted, and lists what is left as still downloading without starting anything. The UI adopts those tags as paused on every list fetch, so a refresh or a fresh open shows an incomplete download and can resume it from the files already on disk. Assisted-by: pi * ui : show a paused download's state on its chip A paused quant chip reads as paused at rest, the way a running one shows its spinner, and offers the resume action on hover. The selector's download row gets a View in Discover action beside its cancel. Assisted-by: pi * ui : collapse the download options by default The copyable download command starts collapsed so the option rows lead; expanding it stays one click away. Assisted-by: pi * ui : keep decision models out of the hub lists Decision models answer with calibrated probabilities instead of text, so llama-ui cannot run them: the hub data layer drops them from the catalog and every search, keyed by the decision-model tag or a text-classification pipeline. Assisted-by: pi * ui : read download tags as quant and sidecar A `<repo>:<quant>-<sidecar>` tag now parses into its quant and sidecar instead of one opaque quant token, and ModelsService.draftSidecarsByRepo collects the router listing's sidecar-only entries per repo, replacing the inline copy in the models store. Assisted-by: pi * ui : cluster draft downloads under their main model Tracked downloads group by repo: the main weights lead and the draft sidecars ride inside their item as compact sub-lines, on the selector, the discover list and the manager table. The main model's controls pause, resume and cancel the whole cluster, and the manager's download rows select into the configuration pane. Assisted-by: pi * ui : act on tracked downloads from the table rows The table passes its clusters' draft sub-lines to the model rows. A row's actions cascade pause and resume, an in-flight row offers Cancel download while a paused one keeps Delete from disk, and the confirm dialog's copy follows the state. Assisted-by: pi * ui : give downloads a configuration pane of their own Selecting a tracked download opens the pane in a download mode: live progress with pause, resume and cancel, and See in Discover, instead of metadata and load actions that a model without files cannot offer. Assisted-by: pi * ui : make the discover details self-fetching and responsive ModelsDiscoverDetailsContainer owns the details, tree and readme fetches so hosts outside the discover view can render the pane, and the details layout adapts to narrow panes with container queries instead of viewport breakpoints. Assisted-by: pi * ui : download a matching draft with the main model A main quant download now pulls the quant-matched draft of the most preferred kind the repo ships (DFlash before MTP), fire-and-forget, so speculative decoding is ready when the main lands. Manual draft chips stay available; the options row also collapses to one column in narrow panes. Assisted-by: pi * ui : keep discover off by default and couple it with the hub setting Discover Models starts unchecked. Enabling it turns the Hugging Face Hub setting on and disabling it turns it back off, so the checkbox never lies; with discover off the hub setting stays available on its own for the display data. Stored and reset state are corrected the same way on load. Assisted-by: pi * ui : open the manage models dialog on the manager view The dialog now opens on the manager instead of returning to the last view, discover entries still land on the discover view while the setting allows it, and the discover surfaces follow the setting's state. Assisted-by: pi * ui : give the discover search room for its focus ring The sticky search bar insets from the panel edges so the focus ring stays inside the scroll container, and rows scroll behind an opaque bar instead of through it. Assisted-by: pi * common: settle sidecar names on the listing and drop unfinished partials An infix or trailing sidecar token is ambiguous: draft heads use it (Model-MTP-BF16.gguf), so do full models with embedded heads (Model-Q8D-MTP.gguf). Such a name is a sidecar only next to token-free main weights of the same family, while mmproj or imatrix anywhere in the name, or a leading token, decides alone (mmproj_f16.gguf). A quant tag always carries its bit count, so a sidecar named after no quant (mtp-Model-it.gguf) gets the bare tag: the listing, the plan and remove all use it. An imatrix tag resolves into a plan slot of its own, downloaded but never wired into the params. remove resolves the tag the way a download does and deletes the partial files of every planned blob, so cancelling a running download or dropping one left unfinished by an earlier run leaves nothing on disk. The router passes its token and releases its lock around the call, and records the unfinished downloads with an atomic write. llama download succeeds on a sidecar-only plan and prints its paths. * ui : move dialog views and draft states into enums and constants The models dialog's views become the ModelsDialogView enum, the draft subline's states join ModelRowDownloadState, the view fade and the copy-feedback reset move to ui constants, and the dialog's compass marks come from DISCOVER_ICON. Assisted-by: pi * ui : drop the unused discover surface The dead openDiscoverModels twin, the commented-out command divider, the details pane's unused onClose and its header's close button, and a redundant type cast leave; the command's copy feedback joins the shared reset constant. Assisted-by: pi * ui : trim the download option comments and highlight the chat template The comment verbosity the options panel and its chips carried shrinks to the non-obvious why, and the chat template dialog renders through SyntaxHighlightedCode as jinja instead of a plain pre block. Assisted-by: pi * common: only a .gguf file is a sidecar, and mmproj/imatrix as whole words A non-GGUF file named like a sidecar (quantization-mmproj-Q8_0.json) is picked as the mmproj, and a sidecar word inside a longer word of a model name (Phi-2-OmniMatrix.i1-Q4_K_M.gguf) drops the model. * ui: dedupe draft sidecar badges per quant The draft sidecar badges deduped by kind and repo without the quant, so with a draft cached at two quants only one badge survived, and a delete of the other main quant found no badge to remove its sidecar with. Also restores the getDownloadEntries doc lost when the clusters method landed above it. Assisted-by: pi (zai-org/GLM-5.3-Flash) * ui: adopt the incomplete download status and drop the paused-ids registry The store inferred an unfinished download from a downloading entry it held no local state for, which misread a download running in another tab as paused and offered a resume the server rejects. The listing carries the "incomplete" status now, so the adoption keys off it directly. The paused ids no longer need to survive in localStorage: the server remembers them across restarts, and the progress snapshot was stale after a reload anyway. The stale localStorage key is removed. Assisted-by: pi (zai-org/GLM-5.3-Flash) * ui: quant-match the pause and cancel cascades Pause, resume and cancel cascaded to every draft sidecar of the repo while delete only took the quant-matched ones, so canceling a Q4_0 download also killed a Q8_0 draft that belongs to another quant's main. Every cascade acts on the quant-matched drafts now, matching what a delete takes with it. Assisted-by: pi (zai-org/GLM-5.3-Flash) * ui: collapse the duplicated draft subline markup The downloaded and in-flight branches repeated the identical connector, label and badge markup; one block with a conditional tail keeps the two states in sync. Assisted-by: pi (zai-org/GLM-5.3-Flash) * ui: scrub positioned styles out of sanitized model card html The style attribute stays allowed for the KaTeX output, which let a third party card draw a fixed overlay over the app. The sanitize hook drops position:fixed and position:absolute values and leaves the KaTeX layout styles alone. Assisted-by: pi (zai-org/GLM-5.3-Flash) * ui: run the discover mount fetch once and reuse the shared repoOf helper fetch() toggles its loading flag synchronously inside the mount effect, which re-ran the fetch on every toggle and retried failed catalog repos endlessly; the effect now untracks the call. The local repoOf copy and the duplicate services import give way to the shared helpers. Assisted-by: pi (zai-org/GLM-5.3-Flash) * ui: narrow the spec-type map to the draft sidecars SPEC_TYPE carried empty entries for the aux sidecars it is never indexed with; the map and the command component now type the badge as a draft sidecar through a draftBadgeOf helper instead of a cast. Assisted-by: pi (zai-org/GLM-5.3-Flash) * ui: name the decision-model tags and markdown cache keys The decision-model hub tag and pipeline tag move into the huggingface constants next to the other pipeline tags, and the markdown transform cache prefixes get names instead of bare one-letter tokens. Assisted-by: pi (zai-org/GLM-5.3-Flash) * ui: read the discover setting through one truthiness check Three places read the setting with !== false and one with Boolean(); all four read it the same way now. Assisted-by: pi (zai-org/GLM-5.3-Flash) * ui: compare the models dialog view against its enum The template compared the view state against the 'manage' literal in three spots while the script used the enum. Assisted-by: pi (zai-org/GLM-5.3-Flash) * nit: Remove `+` before draft sidecar badge * server: list unfinished downloads as incomplete A download that never finished was reported as "downloading", the same status a live download carries, so a client could not tell a running download from a dead one. The tags remembered in unfinished-downloads now list under a dedicated "incomplete" status, without an instance. Assisted-by: pi (zai-org/GLM-5.3-Flash) * server: download a model with its sidecars as -hf does Resolution goes back to master. POST /models takes the sidecar kinds to pull with the model (mmproj by default) and the download child gets them as -hf flags, so the server picks the files. /models lists real models only, each with the sidecars it has on disk, and incomplete downloads keep their sidecars across restarts. A stopped download keeps its partial files for the next attempt, and DELETE purges them through the same plan the download follows. * ui: download sidecars as parameters of their model The download options keep their chips, but a chip no longer posts a sidecar tag: the store maps it to its model and the sidecar kinds to pull, the quant chip takes its companion draft in the same download, and draft badges come from the sidecars each cached model lists. The server adds the sidecars a cached model lacks when it is posted again, and DELETE removes a single sidecar of a cached model. * ui : mirror the server's sidecar lookup in draft badges The server resolves a sidecar by the case-sensitive `<kind>-` token in the same directory as the weights. The UI matched the token loosely, so a badge could offer a draft the server never pulls. Assisted-by: DeepSeek-V4.1-Flash * ui : post the model's sidecar kinds with its download The store keeps one download per model: the chip's kinds ride along with the model's own sidecars, and a download row no longer expands into one row per sidecar. Draft badges carry no quant of their own yet, so a sidecar of a cached model is never labelled with the model's quant. Assisted-by: DeepSeek-V4.1-Flash * ui : join a quant's draft into its download chip One chip per quant, its draft badged inside it and its size covering both files. The draft is the file the server would pick: the same directory, the exact quant, else the closest bit depth, so a 4-bit model can pair with a 16-bit head. Assisted-by: DeepSeek-V4.1-Flash * ui : drop the draft download rows from the manager and selector A download is one row per model again: the drafts ride with it, and the rows read the sidecars on disk off the listing instead of pairing sidecar entries with their model. Pause, resume and cancel act on the model's own download. Assisted-by: DeepSeek-V4.1-Flash * ui : keep the discover download list one row per model The list leads with one row per downloading model, its sidecars as sub-lines. A row badges only the draft kinds the server can resolve, so a badge never outlives the file its lookup would pick. Assisted-by: DeepSeek-V4.1-Flash * ui : keep the discover chips readable in dark mode The metadata chips, the chat template button and the View on Hugging Face link carried a faint border and no surface over the dialog background, so they were near invisible. They take the tertiary button surface now, and their icons size with size-* so the variant's base rule leaves them alone. Assisted-by: DeepSeek-V4.1-Flash * ui : carry the draft through the download options One chip per main quant, its draft badged on it and the chip's downloaded state covering both files. The drafts sit behind an opt-in checkbox with an inline type pick, and each row opens its own command: it pins the row's quant and only names the draft type, so -hfd is gone. Assisted-by: DeepSeek-V4.1-Flash * ui : line the details skeleton up with the loaded pane Same root padding and container context as the loaded view, so the container queries resolve alike. The download options card pads its blocks instead of itself, the readme gets its card, and a description line stands in for the catalog paragraph. Assisted-by: DeepSeek-V4.1-Flash * common : drop the partials of a removed sidecar Removing a sidecar alone left the .downloadInProgress files of the sidecars a download had planned in blobs/. The removal purges them through the same plan the download follows, and the test covers a finished draft beside an unfinished one, whose partial goes while the model keeps its files. Assisted-by: DeepSeek-V4.1-Flash * server : name why a download is refused A model that is already downloading, or that is running, answered "already exists". It now says which: "is already downloading", or "is running, unload it first", both before the resumable check. The router test covers the two. Assisted-by: DeepSeek-V4.1-Flash * ui : post a download once per click Two quick clicks on the same chip posted twice, and the second answer was "already exists". The store tracks the model as pending, returns early while it is pending or in flight, and reads it as in flight before the server answers, so the chip shows the download at once. The new client test pins the flows: one POST per double click, the chip reading downloaded once the list reports the model, a resume keeping its sidecars, a cancel dropping the whole download, and a reload restoring the paused state. Assisted-by: DeepSeek-V4.1-Flash * ui : keep a plain download row the height of the others The sub-lines wrapper of a download row was rendered even with nothing in it, so its padding made the row taller than a plain one and the overlaid progress bar sat in that empty strip. It renders only with sub-lines now, in the selector and in the discover list, and the selector row takes the phone padding its neighbours have. Assisted-by: DeepSeek-V4.1-Flash * ui : badge a repo's draft once The args of a loaded model and the listing of a cached one report the same draft in different words, one with the file's quant and one with the kind alone, so the merge kept both and a repo row showed the same kind twice. The badges merge per kind now, the args one winning as it knows the quant, and the test pins both sources of a real repo. Assisted-by: DeepSeek-V4.1-Flash * server : report the sidecars of a preset or a download on its way A model the preset also names is registered as a preset, so its drafts were never reported and no row could badge them. A download on its way reported none either, though the record of the kinds it pulls is right there: those win over the plan now, the plan resolves through the preset's hf-repo, and a models dir stays out of it. Assisted-by: DeepSeek-V4.1-Flash * ui : badge a download's drafts and open its model from the row A draft still coming down badges the model id now, the way a landed one does, while an on-disk draft keeps its checked sub-line. The row itself opens the model, the status slot beside it pauses or resumes, and the manager resolves a focus on a download row so its panel opens. Assisted-by: DeepSeek-V4.1-Flash * common : report a removed sidecar that only had a partial Removing a sidecar alone dropped the result of the partial purge, so a sidecar that only ever landed as a .downloadInProgress blob answered false and the UI had no reason to refetch. The purge result is folded into the return now, and the test covers a partial-only removal. Assisted-by: DeepSeek-V4.1-Flash * ui : ignore a pause while the download POST is still on its way The chip reads as downloading before the server answers, so a quick second click unloaded a model that had not started and left the store neither in flight nor paused. A pause returns early while the POST is pending, before it touches the feed or the paused marker. The client test holds the POST open and pins it. Assisted-by: DeepSeek-V4.1-Flash * ui : offer a missing draft as its own chip A model already on disk without its draft had no way to add it: the quant chip carried the draft, so it read as a download of both. The quant chip keeps to its own state now, its joined draft shows only while it rides along or is on disk, and a downloaded quant missing its draft gets a chip of its own - badge, size, download - that posts the model with that kind, so the server keeps the files already there and pulls the draft. Assisted-by: DeepSeek-V4.1-Flash * server : list a paused sidecar download by the files on disk A model on disk whose sidecar download stops keeps its unfinished record, and the listing reported the kinds that record plans as if they were on disk: the UI checked a draft that never landed and offered no way to add it. The planned kinds now stand in only while the download runs. * chore: Clean up comments * server : list sidecars only for a model that resolves from a HF repo A preset on a local file has no HF repo, yet the listing resolved its name as one: the plan threw on the invalid repo format and GET /models failed for every model. The sidecars now come from the repo a preset names or a cached model is listed under, and a local file lists without them. * common : fix download cancel regression Signed-off-by: Adrien Gallouët <angt@huggingface.co> * common : preserve model partials when removing sidecars Signed-off-by: Adrien Gallouët <angt@huggingface.co> * server : one sidecar table and an unfinished downloads class Address review from @ggerganov. The sidecar kinds live in one table in common/download, mapping each name to its download option and plan file; the server validates, builds its options and lists cached sidecars through it instead of its own lists. The unfinished downloads move into server_unfinished_downloads, a record with its own mutex and a struct per entry, and server_models exposes them through unfinished_downloads_get and unfinished_downloads_remove. * common: one sidecar table, and removals that keep shared files The resolution, the cached listing and the removal follow a single table of sidecar kinds, so a kind is declared once with its file keyword, whether it is a draft, its option and its plan file. The server reads the same table for the kinds it accepts, the default sidecars and the child flags. Removing a tag keeps the files another cached tag still loads, such as an mmproj named after the removed quant. A sidecar removed on its own goes for every tag that resolves it. An unfinished download records the exact paths of its partials when it starts. A removal deletes those without asking the hub, and keeps a partial another recorded download also writes. The record is taken once the child is spawned, and the files go under the models lock, so a download of the same tag cannot start while they are removed. * server: drop stale partials and refuse a tag that resolves nothing Recording a download again deletes the partials of the earlier record that the new plan no longer writes, such as those of an older hub revision or of a sidecar the new request leaves out. A posted tag that resolves no file of the repo is refused before the download starts. Removing a model follows a single path, whether the reload erased its entry during the wait or not. * server: fix the CI race between the router download tests The pytest workers run in parallel on one shared cache, so a download test could find the file another test had just fetched and see no progress, or start from the unfinished downloads record another test wrote. Each download test now runs on an empty cache of its own. * server: refuse a DELETE of an unknown name and decide downloads in load() A DELETE of a name that is neither a model nor a recorded download went to common_download_remove, and without a tag it wiped the whole repo cache, including the files of a loaded quant or of a download on its way. Such a name is now not found, as it was before. A POST that downloads over an existing entry checked its state outside the lock, so a load of the same model in between got its instance killed. The checks now run in load(), under the lock, right before the spawn. The fake load time of the tests applies to model loads only, since a download worker loads none. * server: add incomplete_sidecars to the /models API A sidecar added to a cached model and then paused kept its record, but the listing showed only the complete model, so a fresh UI lost the pause. The model entry now carries incomplete_sidecars, the kinds its unfinished download has not written yet; a POST with the kind resumes one and a DELETE with it drops its partial. * common : drop a kept partial when a resume gets 416 Now that partials survive between attempts, an old or complete partial makes every retry fail with the same 416. Delete it so the next attempt can start over from zero. Signed-off-by: Adrien Gallouët <angt@huggingface.co> * common: drop a kept partial when a resume gets 200 A server that ignores the range answers 200 to a resume, and the kept partial then fails every attempt the same way a 416 does. Drop it as well, so the next attempt starts over without a range. * server: refuse a download that shares a partial with one on its way Two downloads of quants that share a file, the mmproj or an mtp picked by the closest bits, appended to the same partial and renamed a file with interleaved bytes. A download whose plan writes a partial that a running download also writes is now refused until that one is done. * common : check the saved etag before resuming a partial download Before resuming, compare the etag saved with the partial to the one the server now serves. If it is missing or different, start over, so a resumed download never mixes bytes from two versions of the file. Signed-off-by: Adrien Gallouët <angt@huggingface.co> --------- Signed-off-by: Adrien Gallouët <angt@huggingface.co> Co-authored-by: Pascal <admin@serveurperso.com> Co-authored-by: Adrien Gallouët <angt@huggingface.co> b11714	2026-10-17 17:20:51 +02:00
Aman Gupta	2744e48d1e	CUDA: faster MMVQ for low-bit types (#30517) ... * CUDA: faster MMVQ for low-bit types * ggml-cuda: hip: small nwarps GCN fix on Q3_K(ncols=2,3), Q6_K(ncols=4) --------- Co-authored-by: thelittlefireman <thelittlefireman3@gmail.com> b11713	2026-10-17 15:35:30 +02:00
Pascal	10b8b75534	tests: give the random model generator a per-tensor filler table (#30674) ... * tests: give the random model generator a per-tensor filler table Muse Spark makes test-llama-context fail on state_load, cp_h and cp_d on about six percent of the random seeds, with an NMSE of 1.0e-05 to 1.4e-05 against a threshold of 1.0e-05. The sliding window is what exposes it: a restored state keeps the window alone, so the attention of an SWA layer reduces over a different cell layout and the continuation is not bit exact. Every SWA model carries that, at 1e-08. Muse Spark carried it at 1e-05. The generator is what turns 1e-08 into 1e-05. A norm preserving residual gate holds alpha^2 + beta^2 == 1 and the conversion resolves both weights from one raw weight, while the generator draws every tensor on its own. The pair then scaled the stream and the branch independently, the stream lost an arbitrary factor at each layer, and the model became an amplifier. The weights the test ran on were not a Muse Spark. Per-tensor rules lived inside the write path, where a flag picked the values of a tensor inside each of the two branches that store them, so a new rule meant a new flag threaded through both. The file asks to be rid of that in a TODO of its own. Give each rule a filler, pick one from a table keyed on a substring of the tensor name, and keep a single write path behind it. Mamba moves into the table and Muse Spark joins it: the gate filler draws the raw weight once per block and resolves alpha and beta the way the conversion does. Over a thousand seeds on a CPU build the worst case drops from 1.36e-05 to 2.48e-06 and the median from 4.10e-06 to 5.48e-07, with no seed above the threshold. The other 131 models come out identical to the bit, since no other architecture carries these tensors. ctest is green on a CPU build and on a CUDA build. * tests: draw every tensor filler from fill_normal The fillers repeated the generator and the distribution each time. The normal draw now takes its key and the other fillers transform its values, so the generator lives in one place. The generated models are identical to the bit. b11712	2026-10-17 16:07:55 +03:00
Adrien Gallouët	9d1e03616e	cuda : fix -Wtautological-compare in fattn vec dispatch (#30519) ... Signed-off-by: Adrien Gallouët <angt@huggingface.co> b11711	2026-10-17 14:36:17 +02:00
Sigbjørn Skjæret	f2f667b02d	cuda : add grouped experts top-k fusion (#29181) ... * fuse grouped experts * add tests * hip fix * add to graph_optimize * apply suggestion from am17an * hip fix * fix hip fix * fix fix fix * gate on hip version instead b11710	2026-10-17 13:55:33 +02:00
Xue Zhenliang	62318cd57d	model : add Kolibri-1 (Aleph-Alpha) support (#30673) ... * model : add Kolibri-1 (Aleph-Alpha) support Kolibri-1 is a bilingual German/English MoE reasoning model with 78B parameters and 3.46B active per token: 50 layers, 384 routed experts plus one shared expert. Attention interleaves four sliding layers at a 513 window with one full layer. The sliding layers rotate with NeoX RoPE, the full layers are NoPE. GQA runs 48 query heads against 4 KV heads at head_dim 128 with a QK-RMSNorm, and every block carries four RMSNorms, so the attention and the MLP output are normalized before they rejoin the residual stream. Routing picks the experts from logits + expert_bias and weights them with the sigmoid of the unbiased logits, without renorm. The converter reuses AfmoeModel for the per-expert checkpoint layout and the expert selection bias, and writes the shared expert and the per layer sliding window pattern. The released FP8 checkpoint needs no special code, it goes through the generic weight_scale_inv path. Three tensor names are added to the mapping (post_attn_norm, post_ffn_norm, moe.router.expert_bias) and the 128k vocabulary registers the qwen2 pre-tokenizer by hash. The arch is registered in test-llama-archs, with GQA and the sliding window pattern. Assisted-by: DeepSeek Harness (DeepSeek V4.1 Flash) * Update src/models/kolibri1.cpp Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@huggingface.co> * Add fusion records for kolibri1 * Update src/models/kolibri1.cpp Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@huggingface.co> --------- Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@huggingface.co> b11709	2026-10-17 13:00:17 +02:00
Sandro Hanea	83f9389419	ci : added ABI reports as github artifact (#30619)	2026-10-17 11:41:53 +02:00
Gabe Goodhart	1268edac57	model : Granite switch decision 256 options (#30693) ... * feat(convert): Attempt to come up with 256 single-token, semantic option markers Each switch model needs to identify a collection of single-token option markers that can be mapped to their respective logits for the output probability distribution. The simple upper/lower/numbers set only yields 66 unique markers, but the model can support up to 255. The challenge is that which combinations of characters result in single-token markers is model- specific, so we do a discovery exercise at conversion time by looking through the specific model's vocab for single-token sequences of 1+ plain letter strings (eg `A`, `AA`, `BB`, etc). Branch: GraniteSwitchDecisionFullOptions AI-usage: none Signed-off-by: Gabe Goodhart <ghart@us.ibm.com> * docs: Note granite-switch with 256 option markers Branch: GraniteSwitchDecisionFullOptions AI-usage: none Signed-off-by: Gabe Goodhart <ghart@us.ibm.com> * style: Remove extra newline Branch: GraniteSwitchDecisionFullOptions AI-usage: none Signed-off-by: Gabe Goodhart <ghart@us.ibm.com> * feat(convert): Use the computed labels in the template so c++ side is source of truth Branch: GraniteSwitchDecisionFullOptions AI-usage: full (Claude + Sonnet 5.5) Signed-off-by: Gabe Goodhart <ghart@us.ibm.com> * feat: Use LEV case for GRANITE_SWITCH w/ computed labels Branch: GraniteSwitchDecisionFullOptions AI-usage: full (Claude + Sonnet 5.5) Signed-off-by: Gabe Goodhart <ghart@us.ibm.com> * docs: Consolidate g-s into 255 bucket Branch: GraniteSwitchDecisionFullOptions AI-usage: full (Claude + Sonnet 5.5) Signed-off-by: Gabe Goodhart <ghart@us.ibm.com> b11707	2026-10-17 09:14:32 +02:00
Marcus	f74e2fa6a3	sycl: fix prompt-processing regression after Q4_0/Q8_0 weight reorder (#30374) b11706	2026-10-16 22:03:41 -04:00
Xuan-Son Nguyen	ea0c4c2c85	skills: add slice-model (#30677) ... * skills: add slice-model * exclude from py check * add to agents.md	2026-10-17 02:11:56 +02:00
Zack Li	6420cf5dd5	ci: update device matrix for Snapdragon jobs by removing SM8750 (#30620) ... * ci: update device matrix for Snapdragon jobs by removing SM8750 * ci: increase timeout for QDC device tests and update job timeout parameters	2026-10-16 16:25:05 -07:00
wes	27c7a769da	hexagon: vectorize softplus with HVX (#30435) ... softplus_f32 and tile_softplus_f32 ran a scalar logf(1.0f + expf(x)) loop. Replace both with a shared hvx_softplus_f32_aa() helper built from the backend's existing hvx_vec_exp_f32 / hvx_vec_log_f32 primitives, so no new numerical approximation is introduced. hvx_vec_exp_f32 clamps its input to (-88, 88) before exponentiating, so a naive log(1 + exp(x)) composition would saturate to ~88.72 for large x instead of returning x. The Q6_V_vmux_QVV select keeps the scalar reference's x > 20 guard (ggml_compute_softplus_f32 in ggml-impl.h), which is what makes the vector path match the CPU result. Assisted-by: Claude b11703	2026-10-16 15:20:43 -07:00
Shawn Gu	2f2763db6e	opencl: Add MxFP4 MoE bin kernel for Adreno X1 devices (#30691) ... * opencl: add bin kernel `gemm_moe_mxfp4_q8_1_dp4a_ila_ldck` for X1 * opencl: fix whitespace * opencl: undo unnecessary change * opencl: enable dp4a clc for X1E --------- Co-authored-by: Li He <lih@qti.qualcomm.com> b11702	2026-10-16 15:15:04 -07:00
Daniel Han	01fc3aae84	model: add Beam (Reflection) (#30670) b11701	2026-10-17 00:07:55 +02:00
Gabe Goodhart	23f7aabc16	model : Granite Switch Decision (#30676) ... * feat(convert): Add decision.chat to support decision models that can also chat Branch: GraniteSwitchDecision AI-usage: full (Claude + Sonnet 5.5) Signed-off-by: Gabe Goodhart <ghart@us.ibm.com> * feat(convert): Add conversion for Granite Switch as decision model The template follows from the usage sample: https://huggingface.co/ibm-granite/granite-switch-4.1-8b#guardian-and-decision-side-by-side Branch: GraniteSwitchDecision AI-usage: draft (Claude + Sonnet 5.5) Signed-off-by: Gabe Goodhart <ghart@us.ibm.com> * feat(common): Add support for chat+decision models in common Branch: GraniteSwitchDecision AI-usage: full (Claude + Sonnet 5.5) Signed-off-by: Gabe Goodhart <ghart@us.ibm.com> * feat(common): Add granite switch decision model enum Branch: GraniteSwitchDecision AI-usage: full (Claude + Sonnet 5.5) Signed-off-by: Gabe Goodhart <ghart@us.ibm.com> * feat(server): Plumb through decision_chat bool to support decision+chat models Branch: GraniteSwitchDecision AI-usage: full (Claude + Sonnet 5.5) Signed-off-by: Gabe Goodhart <ghart@us.ibm.com> * feat(server): Add Granite Switch decision support Branch: GraniteSwitchDecision AI-usage: draft (Claude + Sonnet 5.5) Signed-off-by: Gabe Goodhart <ghart@us.ibm.com> * docs: Update server README for granite switch decisions / chat+decisions Branch: GraniteSwitchDecision AI-usage: draft (Claude + Sonnet 5.5) Signed-off-by: Gabe Goodhart <ghart@us.ibm.com> * Revert "feat(convert): Add decision.chat to support decision models that can also chat" This reverts commit 952ef567647e53ac9beda7a1f79d6927240d068d. * Revert "feat(common): Add support for chat+decision models in common" This reverts commit 93ed782e3a8b2fd5968732c6525c7ad29ad4bea2. * Revert "feat(server): Plumb through decision_chat bool to support decision+chat models" This reverts commit 97e9af60ddaccbd82f1eb0a488adff43b16ee095. * fix: If Granite Switch supports decisions, it always supports text too Branch: GraniteSwitchDecision AI-usage: none Signed-off-by: Gabe Goodhart <ghart@us.ibm.com> * docs: Remove reference to decision.chat in server README Branch: GraniteSwitchDecision AI-usage: none Signed-off-by: Gabe Goodhart <ghart@us.ibm.com> b11700	2026-10-16 20:00:41 +02:00
texasich	7182ef71e6	common: honor HTTP_PROXY/HTTPS_PROXY env vars in http client (#21752) ... * common: honor HTTP_PROXY/HTTPS_PROXY env vars in http client common_http_client() was ignoring standard proxy env vars, causing downloads via -hf and similar flags to fail silently in environments behind an HTTP proxy (corporate networks, air-gapped infra, etc.). Checks HTTPS_PROXY/https_proxy for https:// URLs and HTTP_PROXY/http_proxy for http:// URLs, in that order. If a proxy URL is set and parseable, passes it to httplib's set_proxy() and set_proxy_basic_auth() as appropriate. Malformed proxy URLs are silently ignored so existing callers aren't affected. Fixes #21694 * common: drop misleading comment about generic proxy fallback The comment referenced a 'generic one' that doesn't exist in the code — both HTTP_PROXY and HTTPS_PROXY are scheme-specific. Reviewer noted the mismatch. Addresses review feedback from @ServeurpersoCom. * common : warn when the proxy URL from the environment is malformed * common : honor NO_PROXY when selecting the http proxy * ci : retrigger checks (test-thread-safety flake) * common : bypass the proxy for loopback destinations llama cli talks to its spawned server on 127.0.0.1 through common_http_client, so with HTTP_PROXY set and no NO_PROXY the health polling went to the corporate proxy, which rejects loopback targets, and the wait loop never exits ("Loading model..." hangs forever). A remote proxy can never reach the client machine's own loopback, so these destinations are now always contacted directly, before NO_PROXY is consulted: localhost, *.localhost, 127.0.0.0/8, ::1 and 0.0.0.0. Similar to Go's net/http. Proxy handling for downloads and remote --server-base endpoints is unchanged. Signed-off-by: texasich <101962694+texasich@users.noreply.github.com> * docs: move proxy tip from README to server docs per review: the top level README stays lean; the proxy howto now lives with the server docs under Usage Signed-off-by: texasich <101962694+texasich@users.noreply.github.com> --------- Signed-off-by: texasich <101962694+texasich@users.noreply.github.com> Co-authored-by: texasich <texasich@users.noreply.github.com> Co-authored-by: texasich <texasich@gmail.com> Co-authored-by: marecabo <23156476+marecabo@users.noreply.github.com> b11699	2026-10-16 17:01:37 +02:00
aiqubits	2a667032dc	convert : respect Mistral format for DeepseekV2Model (#30669) ... Use the instance's is_mistral_format flag when DeepseekV2Model reloads configuration, allowing Mistral MoE checkpoints to read params.json. Fixes #29899 Signed-off-by: aiqubits <aiqubits@hotmail.com>	2026-10-16 15:17:30 +02:00
Piotr Wilkin (ilintar)	f3363593f4	Add more benchmarks to llama-eval (#26174) ... * llama-eval : add sandboxed execution for code suites Adds a lightweight replacement for the Docker containers usually used to run generated code, so a benchmark can be executed without a daemon, an image or root. A cached venv is provisioned per suite and reused; candidate programs run in a throwaway working directory as their own process group, under the strongest isolation the host offers: bwrap read-only root, tmpfs /tmp and $HOME, no network unshare no network, via --map-current-user rlimit cpu/memory/file-size/process caps only Candidate code must not run as uid 0, which is why "unshare -r" is a last-resort tier that warns: root bypasses file permission bits, so a test asserting that a write to a chmod 0444 file fails would instead see it succeed. Two details worth noting for the bwrap tier. Only paths hidden by the $HOME tmpfs are bound back, because bwrap cannot mount onto a symlink and version-manager alias directories usually are one. And the venv's base interpreter tree is resolved by walking the symlink chain, since a version-managed interpreter is commonly reached through an unversioned alias that startup resolves through. Suites can declare provisioning steps, which run once at build time while the network is still up, so test time never needs it. Assisted-by: Claude Opus 4.8 <noreply@anthropic.com> * llama-eval : add HumanEval and ClassEval code suites Adds two execution-graded suites alongside the existing text-graded ones. The model's reply is parsed for a python code block, combined with the task's own unit tests, and run in the sandbox. --dataset humaneval 164 function-level problems, scored pass@k --dataset classeval 100 class-level tasks, scored at class level (feeding pass@k) and method level, matching upstream pass@k reuses the existing chunk mechanism: --n_cases at a multiple of the dataset size gives that many samples per problem, aggregated with the unbiased estimator from the Codex paper. Program construction and result interpretation are dataset-owned hooks, because the two suites differ: one HumanEval problem is a function judged by a single assert, whereas one ClassEval task is a class judged by many independent test methods, and reducing that to pass/fail would throw away partial credit. Suites also declare their own package set, provisioning steps, interpreter and timeout. The package set is derived from the dataset's own imports rather than a requirements file. HumanEval turns out to need no third-party packages at all; ClassEval needs thirteen, two of which must be declared because no import mentions them -- lxml is selected by string in ClassEval_44, and scipy is a gensim requirement. Notes on ClassEval specifically: * gensim publishes no wheel past cp313, so 3.13 is preferred; on 3.14+ the two pure-Python APIs it uses are provided by a small shim instead. * The three nltk corpora it needs are pre-seeded at venv build time. These are the modern names -- the dataset asks for the since-renamed punkt and averaged_perceptron_tagger. * It defaults to a patched fork, ilintar/ClassEval, because three upstream tests cannot be passed by anything: a hard-coded 2024 date checked against datetime.now(), an assertion on the dataset author's own hostname, and an exact float comparison. Reference solutions go from 96/100 to 100/100 classes and 2182/2196 to 2196/2196 test methods. No skeleton is touched, so what the model is asked to write is unchanged; --dataset-source selects the original. --exec-seed (default 0) seeds the RNG before each test, so suites whose tests draw random inputs score reproducibly. ClassEval_58 fails its own reference solution about 30% of the time without it. The simulator gains both suites, so the harness can be exercised without a model. Assisted-by: Claude Opus 4.8 <noreply@anthropic.com> * llama-eval : fix silent isolation downgrade, generalise the sandbox runner The isolation probe ran the interpreter without binding it back, so whenever this tool is itself started from a venv under $HOME -- the usual case, since it needs `datasets` -- bwrap's tmpfs hid the interpreter, the probe failed, and the sandbox quietly fell back to a tier that does not hide $HOME at all. Nothing in the output looked wrong when that happened, and the existing suites have been running that way. The probe now uses the same interpreter and the same binds the suite will actually get. Two additions the agentic suite needs, neither of which changes the existing suites: * run_argv() runs an arbitrary command in a caller-owned directory that survives the call. run() is now a thin wrapper over it, so HumanEval and ClassEval take exactly the same path as before. A multi-step agent needs its edits to persist across many invocations, and a suite for a language other than Python needs to drive a linter and a test runner rather than a single source string. * RLIMIT_AS is now per-sandbox, defaulting to what it was. It caps reserved address space rather than resident memory, which is the wrong instrument for a runtime that reserves a large region and commits little of it: node's TypeScript type-stripping goes through WebAssembly, reserves gigabytes per instance, and aborts at 4 GiB with an out-of-memory error that reads like a fault in the code under test. HumanEval stays at 164/164 and ClassEval at 100/100. Assisted-by: Claude Opus 5 <noreply@anthropic.com> * llama-eval : add SACB, a multi-turn agentic coding suite `--dataset agentic` is a different shape from the other code suites. Rather than answering a single prompt, the model is given a small but real repository, a narrow tool set and a bug report, drives its own multi-turn conversation, and is graded by running a test suite it was never allowed to see. The corpus is SACB (Simple Agent Coding Benchmark), 60 tasks over three hand-written repositories in Python and TypeScript, hosted at ilintar/SACB. Grading needs no container: a cached venv gives ruff and mypy for Python, and node's own test runner plus tsc for TypeScript, so the whole dependency set is four packages. Three constraints are deliberate: * No shell, and no way to run the tests. The only feedback channel is `lint`. A syntax or type error can be driven out mechanically; a logic error has to be reasoned about. A harness that lets an agent iterate against the test suite measures something closer to search than to understanding. * Two edit tools, one line-addressed and one content-addressed. Which one a model reaches for, and whether it keeps line numbers straight after its own earlier edit, is itself a signal. * The tests never touch the working tree. They are held outside it and copied into a throwaway copy only once the agent has stopped, so they can be neither read nor edited. Every tool path is resolved and confinement-checked against the repository root, so a symlink planted inside the tree cannot reach out of it. A task counts as resolved when every test that was failing passes and nothing that already passed broke. Both sets are derived by running the tests against the defective tree and against the reference fix, never written by hand, so a task whose defect no test exercises cannot enter the corpus. Many tasks carry several independent defects, so the fraction of target tests passed is reported alongside, separating "fixed two of the three faults" from "changed nothing". An episode that arrives with no tool call and no `finish` is nudged rather than ended. Models routinely narrate their analysis mid-task, and treating that as completion ends the episode with no edits and scores a reasoning failure that never happened; about one episode in four exits that way. Nothing here is installed unless this suite is selected, and then only for the language being run: agentic_eval is imported lazily and each toolchain is keyed per language. Assisted-by: Claude Opus 5 <noreply@anthropic.com> * llama-eval : make the agentic hooks typed rather than discovered `ty` flagged nine unresolved attributes, of which the seven in main() predate the agentic suite: `EvalState.dataset` is Optional and was dereferenced without narrowing. * `loaded_dataset` turns the Optional into a value in one place, raising rather than handing back None, so callers do not each have to narrow it. * `is_agentic`, `prepare()` and `run()` are declared on BaseDataset. They were reached through getattr with a default at three call sites, which no checker can see through; `run()` now raises a clear error naming the suite instead of failing as a missing attribute. * The sandbox setup is one branch on `is_agentic` rather than two tests of `grader_type`, which also drops a duplicated condition. No behaviour change: HumanEval stays at 164/164, and `ty check` on examples/llama-eval is clean. Assisted-by: Claude Opus 5 <noreply@anthropic.com> * feat: API key * fix reasoning and response length for agentic runs, add option to save entire transcripts * llama-eval : fix ty unresolved-attribute on loaded_dataset.ae Assisted-by: AI coding agent * llama-eval : trim comments and remove non-ASCII from README Assisted-by: AI coding agent	2026-10-16 11:52:18 +02:00
Jhen-Jie Hong	ceefd16daa	hexagon: correct HMX FA sinks (#30581) ... * hexagon: correct HMX FA sinks * hex-fa: vectorize sinks in hmx kernels --------- Co-authored-by: Max Krasnyansky <maxk@qti.qualcomm.com> b11696	2026-10-16 16:51:14 +08:00
Mikhail Podvitskii	9a16f1dbe4	model : MTP support for Mellum2.1 (#30612) ... * model : MTP support for Mellum * model : inline rope construction in Mellum b11695	2026-10-16 10:20:03 +02:00
Aaron Teo	b84ea5cf5e	convert : speed up model endian conversion and add more datatype support (#30647) ... * convert: speed up model endian conversion Signed-off-by: Aaron Teo <aaron.teo1@ibm.com> * convert: explicitly state noop Signed-off-by: Aaron Teo <aaron.teo1@ibm.com> * convert: add missing data type support Signed-off-by: Aaron Teo <aaron.teo1@ibm.com> --------- Signed-off-by: Aaron Teo <aaron.teo1@ibm.com>	2026-10-16 10:18:29 +02:00
Michael Taylor	a30fb4924d	[SYCL]: fix memory errors in mul_mat, split buffer, host pool (#29889) b11693	2026-10-16 03:28:50 -04:00
Sh1m13	495c5805ce	gguf-py : add numpy Q1_0 and Q2_0 de/quantization support (#30630) ... * gguf-py : add numpy Q1_0 and Q2_0 de/quantization support * cont: added q1_0 / q2_0 to the test list removing the small test Assisted-by: Claude Opus 5.5 <noreply@anthropic.com> --------- Assisted-by: Claude Opus 5.5 <noreply@anthropic.com>	2026-10-16 09:26:14 +02:00
uvos	623c16393c	CUDA: HIP: rms_norm: add variants that load float4 at a time and store x in registers (#29720) ... * CUDA: HIP: rms_norm: load float4 at a time and store x in registers * Add template paramter comments * use ggml_cuda_is_aligned for src * nits * also use ggml_cuda_is_aligned for src in fused variants * fix nits b11691	2026-10-16 10:07:34 +03:00
hayder	13b8048920	common : read ngram cache parts by reference instead of copying (#30061) b11690	2026-10-16 10:06:32 +03:00
FINAL-Bench	0f4376268f	convert : support text-only checkpoints in pplx-decider (#30575) ... * pplx-decider-text-only convert : support text-only checkpoints in pplx-decider * Apply suggestions from code review * convert : remove unused import	2026-10-16 08:56:00 +02:00
Trivalik	3dbcb7fca5	cpu : reduce AVX2 tiled accumulators together (#30296) ... Assisted-by: Codex b11688	2026-10-16 09:48:23 +03:00
Aaron Teo	c804754529	ggml-cpu: refactor s309x repack helper functions (#30588) ... Signed-off-by: Aaron Teo <aaron.teo1@ibm.com> b11687	2026-10-16 09:45:35 +03:00
Neo Zhang	c0083e3052	sycl : reduce the build threads to half (#30236)	2026-10-16 09:44:24 +03:00
Titaniumtown	13de11e78f	sycl: wide stores for the q4_K/q5_K weight dequant (#29696) b11685	2026-10-16 09:43:07 +03:00
Georgi Gerganov	88aaf325a7	tests : catch vanished rows in the fusion baseline (#30618) ... * tests : update the Metal fusion baseline Re-record the MTL fusion baseline with the current build: 276 rows over 131 dummy models. Assisted-by: pi:llama.cpp/ggml-org/Qwen3.8-Flash-Next-GGUF (MacBook, M5 Max, 128GB) * tests : fail test-fusion when a baseline row stops matching The report was built from the counters the run produced, and the baseline was only ever used as a lookup table, so a fusion that stopped matching produced no row and was never compared. qwen4exp and mellum lost their fusions this way while the Fusion / metal job stayed green. Report a baseline row that the run does not reproduce as a regression. Scope the pass to the arches that were actually run so --model keeps working. Assisted-by: pi:llama.cpp/ggml-org/Qwen3.8-Flash-Next-GGUF (MacBook, M5 Max, 128GB) * pi : never write inline comments unless the user asks Assisted-by: pi:llama.cpp/ggml-org/Qwen3.8-Flash-Next-GGUF (MacBook, M5 Max, 128GB) * tests : keep the mellum dummy model softmax-gated mellum reads expert_gating_func from the GGUF since #30578, while the fixture writes sigmoid for every MoE arch, so the Mellum dummy model stopped building the softmax MoE routing fusion. Write softmax for mellum, matching the arch default and the GGUFs that conversion/mellum.py produces. Re-record the MTL baseline: the routing row is back, 277 ok / 0 failed. Assisted-by: pi:llama.cpp/ggml-org/Qwen3.8-Flash-Next-GGUF (MacBook, M5 Max, 128GB) * pi : use emoji in the AI stats disclosure Assisted-by: pi:llama.cpp/ggml-org/Qwen3.8-Flash-Next-GGUF (MacBook, M5 Max, 128GB) * pi : list the disclosure model after the hardware Assisted-by: pi:llama.cpp/ggml-org/Qwen3.8-Flash-Next-GGUF (MacBook, M5 Max, 128GB) * pi : use monochrome symbols in the AI stats disclosure Assisted-by: pi:llama.cpp/ggml-org/Qwen3.8-Flash-Next-GGUF (MacBook, M5 Max, 128GB) * tests : add muse-spark to the Metal fusion baseline Assisted-by: pi:llama.cpp/ggml-org/Qwen3.8-Flash-Next-GGUF (MacBook, M5 Max, 128GB) b11684	2026-10-16 09:35:19 +03:00
Ruben Ortlam	a8d9a7e1e5	vulkan: fix gpu-visible host memory issues (#30468) ... * vulkan: fix log spam if host pointer import fails * exclude MoltenVK from shared memory import * remove unreachable code, assert instead. Shorten warnings * disable host memory buffer sharing on AMD due to amdgpu perf issue b11683	2026-10-16 08:29:23 +02:00
Karthik Vijayan	f12aaa344c	Fix failing RocM tests (#30611) ... Co-authored-by: Karthik Vijayan <kavijay@microsoft.com> b11682	2026-10-16 12:46:55 +08:00
Mikhail Podvitskii	5e03a1ff45	graph : add precision handling for MELLUM architecture (#30584) b11681	2026-10-16 11:32:26 +08:00
Max Krasnyansky	b797b9ca81	hexagon: tensor repacking rework and support for tiled and flat quantized and fp weights (#30635) ... * hex-repack: Remove need_repack hack and streamline the rules -- breaks GET_ROWS tests for K-Quants, will fix in the next round Add support for GET_ROWS with tiled Q5_K Add support for raw k-quant tensors * hex-get-rows: vectorize raw k-quant ukernels * hex-binary: refactor dma-loop thread functions (no need for macros, etc) * hex-quant: move/cleanup flat and tiled dequant ukernels * hex-repack: add support for repacking/tiling F16/F32/BF16 2D weights * hex-get-rows: remove redundant cast * hex-repack: proper filtering for BF16 that were leaking through during the tests * hex-repack: streamling tiling rules (broken MM for now) * hex-repack: fix mmid after tiling updates * hex-repack: rename repacked to tiled in all flags/funcs/kernels/vars * hex-mm: fix valid_rows calc in various kernels * hex-mm: remove unused hvx_mm_nx_2d_flat * hex-mm: cleanup and optimize hvx-tiled vec_dot_f16 * hex-mm: another pass at hvx-tiled f16 matmul * hex-get-rows: optimize small src1 * hex-get-rows: one more pass at recovering get-rows perf regression * hex-repack: export GGML_HEXAGON_RT_REPACK via run.py * hex-mm-tiled: fix accuracy regression with hmx-tiled-f16 matmul * hex-mm: fold tiled_f16 checks into the general flow * hex-get-rows: remove dead checks for buffer_is_hexagon and tiling during supports_op * hex-quant: proper tail handling in new dequant ukernels * hex-get-rows: remove unused code and cleanup support checks * hex-mm: cleanup ne1 padding/tiling checks * hex-mm: cleanup fusion checks for tiled tensors * hex-repack: add a little can_repack helper to reduce duplication * hex-repack: never repack host buffers * hex-repack: factor out repack functions into separate .h and .cpp * hex-tensor: cleanup tensor ifaces for tiled/padded sizes * hex-repack: update k-quant tiling comments and cleanup #defines * hex-dcache: factor out l2/dcache dirty tracking and flushing * hex-tensor: update docs and remove #defs duplication * hex-tensor: remove unused/incorrect get_data interface * hex-get-rows: cleanup vtcm layout build * tests: disable huge MM tests * hex-dcache: add missing header b11680	2026-10-15 20:30:11 -07:00
Aman Gupta	6afa9a3134	llama : share the k-pool input views between layers (#30613) ... The scheduler uploads each view of a host input on its own, so the per-layer reshape of new_pool_idxs (and of sel_mask in glm5-next) was copied once per layer. b11679	2026-10-16 11:14:36 +08:00
Gaurav Garg	b6334e37e6	Limit max CUDA pinned host buffer size to 4 GB (#30449) b11678	2026-10-16 08:20:26 +05:30