← Back to portfolio

Interspeech 2022 · First author · Sep 2022

K-Wav2vec 2.0: Korean ASR with Grapheme–Syllable Joint Decoding

Role · First author · model design, experiments, writing

01Background & Goals

  • The paper explores and optimizes the components of Wav2vec 2.0 for Korean and proposes K-Wav2vec 2.0.
  • Korean syllables are composed of graphemes, so using both units together can improve accuracy and vocabulary coverage at once.

02Contributions

Grapheme–syllable multi-task head and joint decoder
  1. Designed a multi-task architecture that learns grapheme and syllable outputs on top of pre-trained Wav2vec 2.0
  2. Combined the beam-search scores of both outputs in joint decoding to reflect the syllable–grapheme relation
Further pre-training and experiments
  1. Further pre-trained an English-pretrained model on unlabeled Korean speech
  2. Built the experiment setup in Fairseq and ESPnet and evaluated on KsponSpeech and ClovaCall

03Tech Stack

Framework / Platform
PyTorch, Fairseq, ESPnet
Methodology
Self-supervised learning, Multi-task learning, Joint decoding

04Results

  • Published at Interspeech 2022 (first author); code released
  • Presented at the NAVER CLOVA AI Tech Talk (Nov 2021)

05Reference Material