backbone
backbone
¶
BackbonedTrainer: finetune from a pretrained HuggingFace backbone.
Instead of configuring model architecture from scratch, reads two config keys: - architecture/backbone/implementation: "llama", "qwen", or "gpt_neox" - architecture/backbone/weights: HuggingFace model ID (e.g. "TinyLlama/TinyLlama-1.1B-intermediate-step-1431k-3T")
The model class and initial weights are loaded via from_pretrained, bypassing the normal configure() path for architecture parameters.
BackbonedTrainer(spec: ExecutionSpec, base: Optional[Node] = None)
¶
Bases: BaseTrainer[BaseTrainerConfig, Module]
Trainer that initializes from a pretrained HuggingFace backbone.
load_backbone() -> tuple[Any, Any]
¶
Read the declared HuggingFace architecture and its host parameter tree.