Copilot Setup Steps / copilot-setup-steps (push) Has been cancelled
Check Pre-Tokenizer Hashes / pre-tokenizer-hashes (push) Has been cancelled
Python check requirements.txt / check-requirements (push) Has been cancelled
Python Type-Check / python type-check (push) Has been cancelled
Update Operations Documentation / update-ops-docs (push) Has been cancelled
* nemotron : support non-standard GQA, BPE tokenizer and domyn-small pre-type
* gguf-py : write Metaspace prepend_scheme as GGUF metadata
* vocab : parse metaspace prepend_scheme into enum, rename key
* vocab : use metaspace prepend_scheme in detokenize, add TODOs for other tokenizers
* vocab : use constant for metaspace escaped space
---------
Co-authored-by: Martin Cimmino <martin.cimmino@domyn.com>
Copilot Setup Steps / copilot-setup-steps (push) Has been cancelled
Check Pre-Tokenizer Hashes / pre-tokenizer-hashes (push) Has been cancelled
Python check requirements.txt / check-requirements (push) Has been cancelled
Python Type-Check / python type-check (push) Has been cancelled
Update Operations Documentation / update-ops-docs (push) Has been cancelled
* Add Flint model (FlintForCausalLM). Assisted by Codex
* cleanup
* one more
---------
Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@huggingface.co>
Copilot Setup Steps / copilot-setup-steps (push) Has been cancelled
Check Pre-Tokenizer Hashes / pre-tokenizer-hashes (push) Has been cancelled
Python check requirements.txt / check-requirements (push) Has been cancelled
Python Type-Check / python type-check (push) Has been cancelled
Update Operations Documentation / update-ops-docs (push) Has been cancelled
* model : add SenseNova U1 understanding support
Add support for the dense SenseNova U1 understanding branch, including
the U1.5-8B-MoT model.
- Add HF-to-GGUF conversion for NEOChatModel, excluding MoT generation
tensors and concatenating temporal and spatial Q/K normalization
weights.
- Register the sensenova_u1 GGUF architecture and custom spatial RoPE
metadata.
- Implement temporal/spatial Q/K normalization with 4-axis RoPE.
- Keep U1-specific text position handling in the model implementation.
- Add synthetic architecture coverage for sensenova_u1.
Only the dense understanding branch is supported; generation
branches are intentionally rejected during conversion.
* conversion : treat one-expert SenseNova U1 configs as dense
* mtmd : add SenseNova U1 vision support
Add conversion support for the SenseNova U1 vision encoder and implement
its CPU mtmd projector graph.
The graph applies patch embedding, GELU, adjacent-pair 2D RoPE, spatial
downsampling, and dense projection for dynamic-resolution image inputs.
* mtmd : fix SenseNova U1 vision token layout
* model : clean up SenseNova U1 RoPE metadata
Replace the hard-coded spatial RoPE key with shared GGUF and LLM_KV
definitions used by conversion, model loading, and architecture tests.
Clarify why SenseNova U1 retains its custom text-position input: U1 uses
[t, 0, 0, 0], while the generic M-RoPE input uses [t, t, t, 0].
* mtmd : simplify SenseNova U1 projector integration
Map the SenseNova U1 dense projection to the standard mm.0 weight and
bias tensors and reuse the shared projector fields in the loader and graph.
Read vision parameters from the checkpoint configuration and remove the
U1-specific tensor names and model fields while preserving its strided
projection, bias, and row-major token layout.
* mtmd : fix SenseNova U1 image positions and attention
* fix spacing
* mtmd : restore generic test input chunks
* mtmd : restore SenseNova U1 integration after upstream merge
* Update conversion/sensenova_u1.py
Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@huggingface.co>
* Update src/models/sensenova-u1.cpp
Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@huggingface.co>
* model : remove SenseNova U1 BF16 activation casts
Use standard backend precision handling for QKV, FFN, and output
projections instead of casting inputs based on the output tensor type.
Assisted-by: Codex
* Update src/models/sensenova-u1.cpp
Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@huggingface.co>
---------
Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@huggingface.co>
Copilot Setup Steps / copilot-setup-steps (push) Has been cancelled
Check Pre-Tokenizer Hashes / pre-tokenizer-hashes (push) Has been cancelled
Python check requirements.txt / check-requirements (push) Has been cancelled
Python Type-Check / python type-check (push) Has been cancelled
Update Operations Documentation / update-ops-docs (push) Has been cancelled
* model : add H2O-Lightning-4B decision model
Adds the h2o-lightning decision type for h2oai/h2o-lightning-4b
(Qwen3.5-4B with a merged decision LoRA).
None of the existing types match this model. It answers with " A"
rather than "A", uses up to 255 labels, and was trained on Python
json.dumps text, so floats like 12.0 must render as 12.0.
The converter writes the decision metadata and the systemone template
from the model's serve_config.json and refuses a config that doesn't
match. Tests skip unless the tiny test model is available locally.
Assisted-by: Claude Opus 5.5
* rm redundant code
* put labels to gguf
---------
Co-authored-by: Jonathan C. McKinney <jon.mckinney@h2o.ai>
Co-authored-by: Xuan Son Nguyen <son@huggingface.co>
Copilot Setup Steps / copilot-setup-steps (push) Has been cancelled
Check Pre-Tokenizer Hashes / pre-tokenizer-hashes (push) Has been cancelled
Python check requirements.txt / check-requirements (push) Has been cancelled
Python Type-Check / python type-check (push) Has been cancelled
Update Operations Documentation / update-ops-docs (push) Has been cancelled
* mtmd : read LFM2 tiling params from GGUF metadata
Write min_tiles, max_tiles, tile_size and max_pixels_tolerance from
processor_config.json when converting LFM2-VL mmproj, read them in clip.cpp
with the previous constants as defaults. do_image_splitting=false maps to
min_tiles=max_tiles=1 (no tiling).
Rebased version of #25524, original implementation by Michael DM (@michael-dm).
Co-authored-by: Michael DM <usable_siren180@simplelogin.com>
* fix test
---------
Co-authored-by: Michael DM <usable_siren180@simplelogin.com>
Copilot Setup Steps / copilot-setup-steps (push) Has been cancelled
Check Pre-Tokenizer Hashes / pre-tokenizer-hashes (push) Has been cancelled
Python check requirements.txt / check-requirements (push) Has been cancelled
Python Type-Check / python type-check (push) Has been cancelled
Update Operations Documentation / update-ops-docs (push) Has been cancelled
* model : add Kolibri-1 (Aleph-Alpha) support
Kolibri-1 is a bilingual German/English MoE reasoning model with 78B
parameters and 3.46B active per token: 50 layers, 384 routed experts plus
one shared expert.
Attention interleaves four sliding layers at a 513 window with one full
layer. The sliding layers rotate with NeoX RoPE, the full layers are NoPE.
GQA runs 48 query heads against 4 KV heads at head_dim 128 with a
QK-RMSNorm, and every block carries four RMSNorms, so the attention and the
MLP output are normalized before they rejoin the residual stream. Routing
picks the experts from logits + expert_bias and weights them with the
sigmoid of the unbiased logits, without renorm.
The converter reuses AfmoeModel for the per-expert checkpoint layout and
the expert selection bias, and writes the shared expert and the per layer
sliding window pattern. The released FP8 checkpoint needs no special code,
it goes through the generic weight_scale_inv path. Three tensor names are
added to the mapping (post_attn_norm, post_ffn_norm,
moe.router.expert_bias) and the 128k vocabulary registers the qwen2
pre-tokenizer by hash. The arch is registered in test-llama-archs, with GQA
and the sliding window pattern.
Assisted-by: DeepSeek Harness (DeepSeek V4.1 Flash)
* Update src/models/kolibri1.cpp
Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@huggingface.co>
* Add fusion records for kolibri1
* Update src/models/kolibri1.cpp
Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@huggingface.co>
---------
Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@huggingface.co>
Copilot Setup Steps / copilot-setup-steps (push) Has been cancelled
Check Pre-Tokenizer Hashes / pre-tokenizer-hashes (push) Has been cancelled
Python check requirements.txt / check-requirements (push) Has been cancelled
Python Type-Check / python type-check (push) Has been cancelled
Update Operations Documentation / update-ops-docs (push) Has been cancelled
* feat(convert): Attempt to come up with 256 single-token, semantic option markers
Each switch model needs to identify a collection of single-token option
markers that can be mapped to their respective logits for the output
probability distribution. The simple upper/lower/numbers set only yields 66
unique markers, but the model can support up to 255. The challenge is that
which combinations of characters result in single-token markers is model-
specific, so we do a discovery exercise at conversion time by looking
through the specific model's vocab for single-token sequences of 1+ plain
letter strings (eg `A`, `AA`, `BB`, etc).
Branch: GraniteSwitchDecisionFullOptions
AI-usage: none
Signed-off-by: Gabe Goodhart <ghart@us.ibm.com>
* docs: Note granite-switch with 256 option markers
Branch: GraniteSwitchDecisionFullOptions
AI-usage: none
Signed-off-by: Gabe Goodhart <ghart@us.ibm.com>
* style: Remove extra newline
Branch: GraniteSwitchDecisionFullOptions
AI-usage: none
Signed-off-by: Gabe Goodhart <ghart@us.ibm.com>
* feat(convert): Use the computed labels in the template so c++ side is source of truth
Branch: GraniteSwitchDecisionFullOptions
AI-usage: full (Claude + Sonnet 5.5)
Signed-off-by: Gabe Goodhart <ghart@us.ibm.com>
* feat: Use LEV case for GRANITE_SWITCH w/ computed labels
Branch: GraniteSwitchDecisionFullOptions
AI-usage: full (Claude + Sonnet 5.5)
Signed-off-by: Gabe Goodhart <ghart@us.ibm.com>
* docs: Consolidate g-s into 255 bucket
Branch: GraniteSwitchDecisionFullOptions
AI-usage: full (Claude + Sonnet 5.5)
Signed-off-by: Gabe Goodhart <ghart@us.ibm.com>
Copilot Setup Steps / copilot-setup-steps (push) Has been cancelled
Check Pre-Tokenizer Hashes / pre-tokenizer-hashes (push) Has been cancelled
Python check requirements.txt / check-requirements (push) Has been cancelled
Python Type-Check / python type-check (push) Has been cancelled
Update Operations Documentation / update-ops-docs (push) Has been cancelled
* feat(convert): Add decision.chat to support decision models that can also chat
Branch: GraniteSwitchDecision
AI-usage: full (Claude + Sonnet 5.5)
Signed-off-by: Gabe Goodhart <ghart@us.ibm.com>
* feat(convert): Add conversion for Granite Switch as decision model
The template follows from the usage sample:
https://huggingface.co/ibm-granite/granite-switch-4.1-8b#guardian-and-decision-side-by-side
Branch: GraniteSwitchDecision
AI-usage: draft (Claude + Sonnet 5.5)
Signed-off-by: Gabe Goodhart <ghart@us.ibm.com>
* feat(common): Add support for chat+decision models in common
Branch: GraniteSwitchDecision
AI-usage: full (Claude + Sonnet 5.5)
Signed-off-by: Gabe Goodhart <ghart@us.ibm.com>
* feat(common): Add granite switch decision model enum
Branch: GraniteSwitchDecision
AI-usage: full (Claude + Sonnet 5.5)
Signed-off-by: Gabe Goodhart <ghart@us.ibm.com>
* feat(server): Plumb through decision_chat bool to support decision+chat models
Branch: GraniteSwitchDecision
AI-usage: full (Claude + Sonnet 5.5)
Signed-off-by: Gabe Goodhart <ghart@us.ibm.com>
* feat(server): Add Granite Switch decision support
Branch: GraniteSwitchDecision
AI-usage: draft (Claude + Sonnet 5.5)
Signed-off-by: Gabe Goodhart <ghart@us.ibm.com>
* docs: Update server README for granite switch decisions / chat+decisions
Branch: GraniteSwitchDecision
AI-usage: draft (Claude + Sonnet 5.5)
Signed-off-by: Gabe Goodhart <ghart@us.ibm.com>
* Revert "feat(convert): Add decision.chat to support decision models that can also chat"
This reverts commit 952ef567647e53ac9beda7a1f79d6927240d068d.
* Revert "feat(common): Add support for chat+decision models in common"
This reverts commit 93ed782e3a8b2fd5968732c6525c7ad29ad4bea2.
* Revert "feat(server): Plumb through decision_chat bool to support decision+chat models"
This reverts commit 97e9af60ddaccbd82f1eb0a488adff43b16ee095.
* fix: If Granite Switch supports decisions, it always supports text too
Branch: GraniteSwitchDecision
AI-usage: none
Signed-off-by: Gabe Goodhart <ghart@us.ibm.com>
* docs: Remove reference to decision.chat in server README
Branch: GraniteSwitchDecision
AI-usage: none
Signed-off-by: Gabe Goodhart <ghart@us.ibm.com>
Use the instance's is_mistral_format flag when DeepseekV2Model reloads
configuration, allowing Mistral MoE checkpoints to read params.json.
Fixes#29899
Signed-off-by: aiqubits <aiqubits@hotmail.com>
Copilot Setup Steps / copilot-setup-steps (push) Has been cancelled
Check Pre-Tokenizer Hashes / pre-tokenizer-hashes (push) Has been cancelled
Python check requirements.txt / check-requirements (push) Has been cancelled
Python Type-Check / python type-check (push) Has been cancelled
Update Operations Documentation / update-ops-docs (push) Has been cancelled
* model: add Muse Spark Open (text, vision and audio)
Muse Spark Open is a latent space MoE: the residual stream stays at 8192
while the 256 experts live in 4096, with a projection down and up around
them, top-8 sigmoid routing with a correction bias. Attention interleaves
three sliding layers at a 2048 window with rope for one global layer
without rope, with a sigmoid output gate and a weightless QK-norm.
Residuals are norm preserving, their gate is folded at conversion time.
Vision reuses the Muse Glimmer tower. Audio is a causal convolutional
pyramid with no attention and no spectrogram, four stages folding 16, 6,
5 and 4 samples into the channel dim, one embedding per 80 ms of 24 kHz.
The quantizer keeps the latent projections and the attention gate at
Q8_0, a little over 1% of the weights, since the whole residual stream
passes through them and the low-bit types turned the output into a
repeating loop. The model saver writes the post norm epsilon, and the
arch is registered in test-llama-archs.
Co-authored-by: Xuan Son Nguyen <son@huggingface.co>
Co-authored-by: Pedro Cuenca <pedro@huggingface.co>
* Muse Spark: drop the redundant forward expand
* Muse Spark: drop the redundant forward expand and tighten the comments
* Update src/llama-quant.cpp
Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@huggingface.co>
* Update src/llama-graph.cpp
Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@huggingface.co>
* Update conversion/muse_spark.py
Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@huggingface.co>
* Update conversion/muse_spark.py
Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@huggingface.co>
* Update conversion/muse_spark.py
Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@huggingface.co>
* Update conversion/muse_spark.py
Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@huggingface.co>
---------
Co-authored-by: Xuan Son Nguyen <son@huggingface.co>
Co-authored-by: Pedro Cuenca <pedro@huggingface.co>
Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@huggingface.co>
Copilot Setup Steps / copilot-setup-steps (push) Has been cancelled
Check Pre-Tokenizer Hashes / pre-tokenizer-hashes (push) Has been cancelled
Python check requirements.txt / check-requirements (push) Has been cancelled
Python Type-Check / python type-check (push) Has been cancelled
Update Operations Documentation / update-ops-docs (push) Has been cancelled
* spec : add lilicorr support
* convert : use convention for hparam assignment
* add todo for hparam use in llama-batch.cpp
Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
* typo fix
---------
Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
* model : support classifier_activation for rerankers
Assisted-by: Claude Opus 5.5
* model : map classifier gelu to gelu_erf and accept tanh
Assisted-by: Claude Opus 5.5
* model : default act_cls to tanh, ModernBERT falls back to gelu_erf
Assisted-by: Claude Opus 5.5
* model : add LiquidAI/d1-omni-600M decision model
Assisted-by: Claude Opus 5.5
* mtmd : keep conformer GLU sigmoid on CUDA
Assisted-by: Claude Opus 5.5
* server : take d1omni audio through images and input_audio, scope memory-less lfm2 to non-causal
Assisted-by: Claude Opus 5.5
* common : rename decision type d1omni to lfm2-d1-omni, server : make images an alias of files
Assisted-by: Claude Opus 5.5
* model : add LiquidAI/d1-3b decision model
mtmd : read LFM2 image resize algo from GGUF
Assisted-by: Claude Opus 5.5
* common : rename decision type d1 to lfm2-d1
Assisted-by: Claude Opus 5.5
* convert : Fix token configuration for PLaMo-3
The reasoning and tool calling tags in PLaMo-3 consist of three tokens
each. For example, for reasoning:
* reasoning start: `<|plamo:begin_`, `think`, `:plamo|>`
* reasoning end: `<|plamo:end_`, `think`, `:plamo|>`
Registering `<|plamo:begin_`, `<|plamo:end_`, and `:plamo|>` as
`USER_DEFINED` so that they are parsed correctly.
Also PLaMo-3 models use <|plamo:tag|> as EOT, while PLaMo-2 models
use <|plamo:op|>.
Take EOT token as a parameter and look it up so that PLaMo-2 and
PLaMo-3 can use their appropriate ones.
* use NORMAL instead of USER_DEFINED
* vocab : implement PLaMo-3 tokenizer pre-segmentation
The PLaMo-3 tokenizer inserts hard boundaries before running the Unigram
DP, around <|plamo:...|>-looking text, and around runs of at least 4
identical characters or 2 spaces. Without them llama.cpp tokenizes code
indentation and repeated punctuation differently from the reference.
Reproduce the two re.sub() passes in llm_tokenizer_plamo2 by encoding
each segment independently.
* add vocab type "plamo3"
* Update src/llama-vocab.cpp
Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@huggingface.co>
* misc change
---------
Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@huggingface.co>
* model: K2 Horizon gguf conversion code
* model: loading hparams and tensors in k2-horizon.cpp
* model: K2 Horizon compute graph
* model: K2 Horizon compute graph adjustment and registering tokenizers
* model: K2 Horizon chat template and accomodate safetensors naming
* unicode : add the K2-Horizon pre-tokenizer splitter
The K2-Horizon regex had no arm in unicode_regex_split_custom and fell through to the
general std::regex fallback, which fails two ways.
On MSVC std::regex rejects \p{...}, so no K2-Horizon GGUF loads on Windows at all:
llama-quantize, llama-imatrix and llama-perplexity all abort with
regex_error(error_escape) before a token is produced.
Where the fallback does compile it is still wrong. unicode_regex_split collapses each
codepoint to a single byte naming its Unicode category before matching, and U+200C/U+200D
are category Control, which has no entry in k_ucat_cpt, so both become the 0xD0 fallback
byte. The literal and alternatives in K2's regex can then never match and
every ZWNJ or ZWJ ends a letter run.
The splitter is the existing llama3 one with a single rule widened, since K2's regex
differs from llama3's only in that a letter run also takes marks, ZWNJ and ZWJ.
tests/test-unicode.cpp gains a case for this: it fails before the change with
[Amy] [ZWNJ khaham] and passes after with the run intact.
* tests: expand K2 Horizon unicode splitter coverage
* unicode: handle K2 Horizon case folding and empty input
Assisted-by: Codex
* jinja : support sequence indices in selectattr and rejectattr
Assisted-by: Codex
* model : add K2 Horizon dense and MoVA support
Includes the K2 Horizon implementation from ifm-ai/llama.cpp with converter, tensor-parallel and model save/reload fixes.
Assisted-by: Codex
* chat : support K2 Horizon reasoning and tool calls
Assisted-by: Codex
* conversion: remove obsolete K2 Aurora alias
Assisted-by: Codex
* k2-horizon: enforce response schemas and load YaRN betas
Constrain final JSON after reasoning, accept flexible JSON tool envelopes,
enforce XML dialects, and handle repeated or alternate thinking markers.
Load YaRN beta metadata instead of retaining the default values.
Add schema, streaming, continuation, and model reload regressions. Validate
CUDA and CPU builds and 0.9B, 4B, and MoVA conversation/tool round trips.
Assisted-by: Codex
* renaming template fixture
* adressing cisc follows ups
* desloppify the parser / adress aldehir comments
* clean test-chat
* remove fallback : model trained mostly on high anyway
* fix k2 attn_v_exp tn splitting and metal fusion baseline
* k2-horizon : forward expand views before sums
* k2-horizon: copy embds before group norm to fix TP
* disable tesnor parallelism
---------
Co-authored-by: Ryandito Diandaru <ryandito.diandaru@mbzuai.ac.ae>
Co-authored-by: WestWaters <mario.papaleo2013@gmail.com>
Co-authored-by: Natani L. Mayday <71436458+TaskPuppyNatani@users.noreply.github.com>
Co-authored-by: West <100190545+WestWaters@users.noreply.github.com>
Co-authored-by: aaryamonvikram <aaryamonvikram@gmail.com>
Co-authored-by: aaryamonvikram <96529820+aaryamonvikram@users.noreply.github.com>
* server: support vision input for Clef
* move input_attn_causal to private
* extend old server_batch::embd
* server_batch::token::pos to multi dim
* nits
* fix abort
* fix img tokens cap
* fix yield_to_queue mutate data
* init conversion
* convert: ok
* model loaded
* add server code
* improve conversion script
* support shared prompt prefix
* add docs, imorove UX a bit
* add vision support
* add openjev tiny model for testing
* add dev docs
* support lev & kev
* clean up
* fix lev noul
* fix py lint
* nits docs
* clarify about not supporting date_facts
* convert : write Gemma embedding scale for DFlash drafts
A DFlash draft shares the target's token embeddings. Gemma scales them by sqrt(hidden_size) in the forward pass, and the draft config does not state that scale, so the converted draft read unscaled embeddings.
Take the scale from the target config when the draft config has none.
Assisted-by: Claude
* convert : check with get_model_architecture for gemma models
* vocab : honor BOS/EOS settings for PLaMo-2 and PLaMo-3
The original tokenizer configs for PLaMo-2 and PLaMo-3 have
`add_bos_token: true` and `add_eos_token: false` , but
_set_vocab_plamo() did not write the BOS/EOS metadata. The
PLAMO2 tokenizer path also ignored add_bos/add_eos during
tokenization.
Write the settings from tokenizer_config.json and honor them in
the PLAMO2 tokenization path. GGUFs without these keys keep the
previous behavior.
* Update conversion/base.py
Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@huggingface.co>
---------
Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@huggingface.co>
tc-mb