Mixture of Experts
M
Mixture of Experts
Definition
A neural network architecture that divides the model into multiple specialized sub-networks (experts) and uses a gating mechanism to route each input to the most relevant experts. MoE enables scaling model capacity without proportionally increasing computation.