core/nn/rms_norm library

RMSNorm module — a trainable wrapper around Tensor.rmsNorm.

LayerNorm without the mean-subtract and without the beta bias. Used by Llama, Mistral, Qwen and Phi-3.

Classes

RMSNorm

dart_pytorch package
documentation
core/nn/rms_norm.dart

dart_pytorch package

Libraries
core/coop/lm_factory
core/coop/param_avg
core/coop/shard
core/data/bpe_tokenizer
core/data/char_tokenizer
core/data/csv_dataset
core/data/dataset
core/data/hf_bpe_tokenizer
core/data/image_folder_dataset
core/data/text_token_dataset
core/data/wordpiece_tokenizer
core/nn/aft_transformer
core/nn/attention/aft_attention
core/nn/attention/multi_head_attention
core/nn/attention/multi_head_cross_attention
core/nn/bert
core/nn/bert_hf_loader
core/nn/clip_hf_loader
core/nn/conv2d
core/nn/dropout
core/nn/embedding
core/nn/encoder_decoder
core/nn/ffn/swiglu
core/nn/gpt
core/nn/gpt2_hf_loader
core/nn/gptj
core/nn/gptj_hf_loader
core/nn/kv_cache
core/nn/layer_norm
core/nn/lc0
core/nn/lc0_input
core/nn/lc0_proto
core/nn/linear
core/nn/llama
core/nn/llama_hf_loader
core/nn/llama_vision
core/nn/masks
core/nn/modalities/audio_transformer
core/nn/modalities/multi_modal_classifier
core/nn/modalities/multi_modal_encoder
core/nn/modalities/multi_modal_generator
core/nn/modalities/multi_modal_lm
core/nn/modalities/text_transformer
core/nn/modalities/video_transformer
core/nn/module
core/nn/moe
core/nn/moe_transformer
core/nn/muzero_lm
core/nn/positional
core/nn/pythia
core/nn/pythia_hf_loader
core/nn/rms_norm
core/nn/rotary
core/nn/safetensors
core/nn/sentence/cross_encoder
core/nn/sentence/losses
core/nn/sentence/sentence_encoder
core/nn/sentence/similarity
core/nn/serialize
core/nn/transformer
core/nn/transformer_decoder
core/nn/transformer_decoder_block
core/nn/transformer_encoder
core/nn/transformer_lm
core/nn/vision/clip_vision_model
core/nn/vision/vision_encoder
core/nn/vision/vit_backbone
core/nn/vision/vit_classifier
core/nn/vision/vit_face_embedding
core/nn/vision/vit_object_detector
core/nn/vision_projector
core/optim/adam
core/optim/grad_utils
core/optim/lr_scheduler
core/optim/optimizer
core/optim/sgd
core/tensor/cuda_engine
core/tensor/dtype
core/tensor/native_lib_download
core/tensor/tensor
core/utils/hungarian_algorithm
core/vector_store/auto_tune
core/vector_store/bench
core/vector_store/faiss_io
core/vector_store/gpu_index_flat
core/vector_store/index
core/vector_store/index_binary
core/vector_store/index_binary_flat
core/vector_store/index_binary_ivf
core/vector_store/index_convert
core/vector_store/index_factory
core/vector_store/index_flat
core/vector_store/index_hnsw
core/vector_store/index_id_map
core/vector_store/index_io
core/vector_store/index_ivf_flat
core/vector_store/index_ivf_pq
core/vector_store/index_lsh
core/vector_store/index_pq
core/vector_store/index_pre_transform
core/vector_store/index_refine_flat
core/vector_store/index_replicas
core/vector_store/index_scalar_quantizer
core/vector_store/index_shards
core/vector_store/kmeans
core/vector_store/l2_norm_transform
core/vector_store/pca_transform
core/vector_store/random_rotation_transform
core/vector_store/vector_transform
dart_pytorch