SauTech

Abstract

The model is based on the wav2vec 2.0 architecture which uses CTC loss/decoding. The model consists of a convolutional feature encoder for extracting latent speech representations, a transformer-based context network for modeling long-range dependencies, and a quantization module for discretizing features. This design enables the model to produce rich, context-aware speech representations suitable for downstream speech recognition tasks.