Knowledge Distillation for Mixture of Experts Models in Speech Recognition
Felipe Cruz Salinas, Kenichi Kumatani, Robert Gmyr, Linquan Liu, Yu Shi
MSR-TR-2022-6 | May 2022
Published by Microsoft
Kenichi Kumatani
PRINCIPAL RESEARCHER
Publications
Publications by Year
-
-
Building a great multi-lingual teacher with sparsely-gated mixture of experts for speech recognition
Kenichi Kumatani, Robert Gmyr, Felipe Cruz Salinas, Linquan Liu, Wei Zuo , Devang Patel, Eric Sun, Yu Shi
CoRR | December 2021, Vol abs/2112.05820
Tackling Dynamics in Federated Incremental Learning with Variational Embedding Rehearsal
Tae Jin Park, Kenichi Kumatani, Dimitrios Dimitriadis
October 2021
One-Shot Voice Conversion with Speaker-Agnostic StarGAN
Sefik Emre Eskimez, Dimitrios Dimitriadis, Kenichi Kumatani, Robert Gmyr
INTERSPEECH 2022 | August 2021
UniSpeech: Unified Speech Representation Learning with Labeled and Unlabeled Data
Chengyi Wang, Yu Wu, Yao Qian, Kenichi Kumatani, Shujie Liu, Furu Wei, Michael Zeng, Xuedong Huang
2021 International Conference on Machine Learning | July 2021
Ensemble Combination between Different Time Segmentations
Jeremy Wong, Dimitrios Dimitriadis, Kenichi Kumatani, Yashesh Gaur, George Polovets, Partha Parthasarathy, Eric Sun, Jinyu Li, Yifan Gong
2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) | June 2021
-
Sequence-Level Self-Learning with Multiple Hypotheses
Kenichi Kumatani, Dimitrios Dimitriadis, Yashesh Gaur, Robert Gmyr, Sefik Emre Eskimez, Jinyu Li, Michael Zeng
Interspeech | October 2020
Federated Transfer Learning with Dynamic Gradient Aggregation
Dimitrios Dimitriadis, Kenichi Kumatani, Robert Gmyr, Yashesh Gaur, Sefik Emre Eskimez
August 2020
Publications by Research Area
-
Knowledge Distillation for Mixture of Experts Models in Speech Recognition
Felipe Cruz Salinas, Kenichi Kumatani, Robert Gmyr, Linquan Liu, Yu Shi
MSR-TR-2022-6 | May 2022
Published by Microsoft
Building a great multi-lingual teacher with sparsely-gated mixture of experts for speech recognition
Kenichi Kumatani, Robert Gmyr, Felipe Cruz Salinas, Linquan Liu, Wei Zuo , Devang Patel, Eric Sun, Yu Shi
CoRR | December 2021, Vol abs/2112.05820
Tackling Dynamics in Federated Incremental Learning with Variational Embedding Rehearsal
Tae Jin Park, Kenichi Kumatani, Dimitrios Dimitriadis
October 2021
One-Shot Voice Conversion with Speaker-Agnostic StarGAN
Sefik Emre Eskimez, Dimitrios Dimitriadis, Kenichi Kumatani, Robert Gmyr
INTERSPEECH 2022 | August 2021
UniSpeech: Unified Speech Representation Learning with Labeled and Unlabeled Data
Chengyi Wang, Yu Wu, Yao Qian, Kenichi Kumatani, Shujie Liu, Furu Wei, Michael Zeng, Xuedong Huang
2021 International Conference on Machine Learning | July 2021
Ensemble Combination between Different Time Segmentations
Jeremy Wong, Dimitrios Dimitriadis, Kenichi Kumatani, Yashesh Gaur, George Polovets, Partha Parthasarathy, Eric Sun, Jinyu Li, Yifan Gong
2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) | June 2021
Sequence-Level Self-Learning with Multiple Hypotheses
Kenichi Kumatani, Dimitrios Dimitriadis, Yashesh Gaur, Robert Gmyr, Sefik Emre Eskimez, Jinyu Li, Michael Zeng
Interspeech | October 2020
Federated Transfer Learning with Dynamic Gradient Aggregation
Dimitrios Dimitriadis, Kenichi Kumatani, Robert Gmyr, Yashesh Gaur, Sefik Emre Eskimez
August 2020
-
Tackling Dynamics in Federated Incremental Learning with Variational Embedding Rehearsal
Tae Jin Park, Kenichi Kumatani, Dimitrios Dimitriadis
October 2021
Federated Transfer Learning with Dynamic Gradient Aggregation
Dimitrios Dimitriadis, Kenichi Kumatani, Robert Gmyr, Yashesh Gaur, Sefik Emre Eskimez
August 2020
-
UniSpeech: Unified Speech Representation Learning with Labeled and Unlabeled Data
Chengyi Wang, Yu Wu, Yao Qian, Kenichi Kumatani, Shujie Liu, Furu Wei, Michael Zeng, Xuedong Huang
2021 International Conference on Machine Learning | July 2021
Ensemble Combination between Different Time Segmentations
Jeremy Wong, Dimitrios Dimitriadis, Kenichi Kumatani, Yashesh Gaur, George Polovets, Partha Parthasarathy, Eric Sun, Jinyu Li, Yifan Gong
2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) | June 2021
-
Ensemble Combination between Different Time Segmentations
Jeremy Wong, Dimitrios Dimitriadis, Kenichi Kumatani, Yashesh Gaur, George Polovets, Partha Parthasarathy, Eric Sun, Jinyu Li, Yifan Gong
2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) | June 2021
-
Ensemble Combination between Different Time Segmentations
Jeremy Wong, Dimitrios Dimitriadis, Kenichi Kumatani, Yashesh Gaur, George Polovets, Partha Parthasarathy, Eric Sun, Jinyu Li, Yifan Gong
2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) | June 2021
Federated Transfer Learning with Dynamic Gradient Aggregation
Dimitrios Dimitriadis, Kenichi Kumatani, Robert Gmyr, Yashesh Gaur, Sefik Emre Eskimez
August 2020
-
Federated Transfer Learning with Dynamic Gradient Aggregation
Dimitrios Dimitriadis, Kenichi Kumatani, Robert Gmyr, Yashesh Gaur, Sefik Emre Eskimez
August 2020
Publications by Type
-
Building a great multi-lingual teacher with sparsely-gated mixture of experts for speech recognition
Kenichi Kumatani, Robert Gmyr, Felipe Cruz Salinas, Linquan Liu, Wei Zuo , Devang Patel, Eric Sun, Yu Shi
CoRR | December 2021, Vol abs/2112.05820
-
One-Shot Voice Conversion with Speaker-Agnostic StarGAN
Sefik Emre Eskimez, Dimitrios Dimitriadis, Kenichi Kumatani, Robert Gmyr
INTERSPEECH 2022 | August 2021
UniSpeech: Unified Speech Representation Learning with Labeled and Unlabeled Data
Chengyi Wang, Yu Wu, Yao Qian, Kenichi Kumatani, Shujie Liu, Furu Wei, Michael Zeng, Xuedong Huang
2021 International Conference on Machine Learning | July 2021
Ensemble Combination between Different Time Segmentations
Jeremy Wong, Dimitrios Dimitriadis, Kenichi Kumatani, Yashesh Gaur, George Polovets, Partha Parthasarathy, Eric Sun, Jinyu Li, Yifan Gong
2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) | June 2021
Sequence-Level Self-Learning with Multiple Hypotheses
Kenichi Kumatani, Dimitrios Dimitriadis, Yashesh Gaur, Robert Gmyr, Sefik Emre Eskimez, Jinyu Li, Michael Zeng
Interspeech | October 2020
-
Knowledge Distillation for Mixture of Experts Models in Speech Recognition
Felipe Cruz Salinas, Kenichi Kumatani, Robert Gmyr, Linquan Liu, Yu Shi
MSR-TR-2022-6 | May 2022
Published by Microsoft
-
Tackling Dynamics in Federated Incremental Learning with Variational Embedding Rehearsal
Tae Jin Park, Kenichi Kumatani, Dimitrios Dimitriadis
October 2021
Federated Transfer Learning with Dynamic Gradient Aggregation
Dimitrios Dimitriadis, Kenichi Kumatani, Robert Gmyr, Yashesh Gaur, Sefik Emre Eskimez
August 2020