Skip to content
theseus
base
Search
Jemoka/thx
theseus
Jemoka/thx
Home
Getting Started
Tutorial
Tutorial
Overview
Local training
Local analysis
Remote training
Your own experiments
What's Next
How-to Guides
How-to Guides
Analysis System
Evaluation System
Remote Configuration
Remote Configuration
Overview
Provider Configuration
JuiceFS Integration
Adding Things
Adding Things
Overview
Adding Components
Adding Components
Overview
Model
Experiment
Dataset
Evaluation
Analysis Job
Optimizer
Learning Rate Schedule
Extending Components
Extending Components
Overview
Trainer
Module
GPT
Transformer Block
Attention
MLP
Mixture of Experts
Design
Design
Overview
Sharp Edges
Experiment DAG and Storage
Available Chips
Sharding Design
Config System
Analysis and Plotting
Reference
Reference
analysis
analysis
Index
attention
base
plots
trace
base
base
Index
axis
chip
dag
hardware
job
topology
checkpoint
cli
cli
Index
app
cleanup
configurator
configurator
Index
fields
steps
configure
interface
interface
Index
actions
age
cache
chart
data
driver
fields
location
log_data
log_list
log_view
navigation
plot
plot_controls
plot_math
plotly_options
run
state
status
timeline
tree
tree_row
view
views
walk
jobs
run
submit
ui
config
data
data
Index
datasets
datasets
Index
addition
alpaca
bbq
ccaligned
cfq
clutrr
dataset
fever
fineweb
fineweb_edu
flan
harmfulqa
longbench
longhealth
mmlu
mnli
mtob
openr1_math
pes2o
pg19
pile
pile_detoxify
pile_injected
qqp
siqa
squad
sst2
winogrande
tokenize
tokenizer
evaluation
evaluation
Index
base
datasets
datasets
Index
addition
alpaca
arc_challenge
arithmetic
bbh
bbq
blimp
ccaligned
cfq
clutrr
fever
gsm8k
hellaswag
longbench
longhealth
math
mmlu
mnli
mtob
perplexity_evals
pes2o
pg19
pg19_lengthgen
pile
pile_injected
qqp
siqa
squad
sst2
tinystories
winogrande
execute
execute
Index
bootstrap
bundle
combobulator
config
dispatch
provider
provider
Index
base
slurm
slurm
Index
availability
provider
ssh
utils
volcano
volcano
Index
config
manifest
transport
run
run
Index
lineage
run
solve
experiments
experiments
Index
benchmark
data
data
Index
blockwise
streaming
forking
forking
Index
pretrain
models
models
Index
gpt
gpt_neox
llama
qwen
qwen_3_5
inference
inference
Index
base
job
model
model
Index
activations
activations
Index
swiglu
attention
attention
Index
base
forking
gated_delta
grouped
rope
axes
block
block
Index
block
forking
gpt_neox
llama
qwen
qwen_3_5
debug
layers
layers
Index
layernorm
mlp
mrope
rmsnorm
rope
masks
models
models
Index
base
contrib
contrib
Index
gpt_neox
llama
marin
qwen
qwen_3_5
qwen_3_5_moe
thoughtbubbles
module
moe
moe
Index
base
shared
utils
quick
registry
store
training
training
Index
backbone
base
contrastive
flywheel
flywheel
Index
contrastive
dataset
io
padded
pmd
strategy
stream
kl_divergence
optimizers
optimizers
Index
adamw
base
base
On this page
base
muon
profiler
schedules
schedules
Index
base
cosine_rewarm
wsd
wsds
utils
About
On this page
base
base
base
¶
An optimizer factory paired with its configuration schema.
Back to top