Yu Shi

i-Code V2: An Autoregressive Generation Framework over Vision, Language, and Speech Data

Ziyi Yang, Mahmoud Khademi, Yichong Xu, Reid Pryzant, Yuwei Fang, Chenguang Zhu, Dongdong Chen, Yao Qian, Mei Gao, Yi-Ling Chen, Robert Gmyr, Naoyuki Kanda, Noel Codella, Bin Xiao, Yu Shi, Lu Yuan, Takuya Yoshioka, Michael Zeng, Xuedong Huang

North American Chapter of the Association for Computational Linguistics (NAACL) 2024 | May 2023

Publication

i-Code: An Integrative and Composable Multimodal Learning Framework

Ziyi Yang, Yuwei Fang, Chenguang Zhu, Reid Pryzant, Dongdong Chen, Yu Shi, Yichong Xu, Yao Qian, Mei Gao, Yi-Ling Chen, Liyang Lu, Yujia Xie, Robert Gmyr, Noel Codella, Naoyuki Kanda, Bin Xiao, Lu Yuan, Takuya Yoshioka, Michael Zeng, Xuedong Huang

AAAI 2023 | February 2023

Publication

Breaking trade-offs in speech separation with sparsely-gated mixture of experts

Xiaofei Wang, Zhuo Chen, Yu Shi, Jian Wu, Naoyuki Kanda, Takuya Yoshioka

arXiv:2211.06493 | November 2022

Preprint

Knowledge Distillation for Mixture of Experts Models in Speech Recognition

Felipe Cruz Salinas, Kenichi Kumatani, Robert Gmyr, Linquan Liu, Yu Shi

MSR-TR-2022-6 | May 2022

Published by Microsoft

Publication

Building a great multi-lingual teacher with sparsely-gated mixture of experts for speech recognition

Kenichi Kumatani, Robert Gmyr, Felipe Cruz Salinas, Linquan Liu, Wei Zuo , Devang Patel, Eric Sun, Yu Shi

CoRR | December 2021, Vol abs/2112.05820

Publication

Florence: A New Foundation Model for Computer Vision

Lu Yuan, Dongdong Chen, Yi-Ling Chen, Noel Codella, Xiyang Dai, Jianfeng Gao, Houdong Hu, Xuedong Huang, Boxin Li, Chunyuan Li, Ce Liu, Mengchen Liu, Zicheng Liu, Yumao Lu, Yu Shi, Lijuan Wang, Jianfeng Wang, Bin Xiao, Zhen Xiao, Jianwei Yang, Michael Zeng, Luowei Zhou, Pengchuan Zhang

arXiv preprint arXiv:2111.11432 | November 2021

Publication Project

Generating Human Readable Transcript for Automatic Speech Recognition with Pre-trained Language Model

Junwei Liao, Yu Shi, Ming Gong (YIMING), Linjun Shou (寿林钧), Sefik Emre Eskimez, Liyang Lu, Hong Qu, Michael Zeng

ICASSP2021 | June 2021

Publication

Speech-language Pre-training for End-to-end Spoken Language Understanding

Yao Qian, Ximo Bian, Yu Shi, Naoyuki Kanda, Leo Shen, Zhen Xiao, Michael Zeng

ICASSP 2021 | June 2021

Publication

Improving Zero-shot Neural Machine Translation on Language-specific Encoders-Decoders

Junwei Liao, Yu Shi, Ming Gong (YIMING), Linjun Shou (寿林钧), Hong Qu, Michael Zeng

arXiv:2102.06578 | February 2021

Publication

Listen, Look and Deliberate: Visual context-aware speech recognition using pre-trained text-video representations

Shahram Ghorbani, Yashesh Gaur, Yu Shi, Jinyu Li

Spoken Language Technology Workshop | January 2021

Publication

Discriminative Transfer Learning for Optimizing ASR and Semantic Labeling in Task-oriented Spoken Dialog

Yao Qian, Yu Shi, Michael Zeng

InterSpeech2020 | October 2020

Publication

MaP: A Matrix-based Prediction Approach to Improve Span Extraction in Machine Reading Comprehension

Huaishao Luo, Yu Shi, Ming Gong (YIMING), Linjun Shou (寿林钧), Tianrui Li

AACL2020 | September 2020

Publication

Mixed-Lingual Pre-training for Cross-lingual Summarization

Ruochen Xu, Chenguang Zhu, Yu Shi, Michael Zeng, Xuedong Huang

The 1st Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the 10th International Joint Conference on Natural Language Processing (AACL 2020) | September 2020

Publication

Improving Readability for Automatic Speech Recognition Transcription

Junwei Liao, Yu Shi, Sefik Emre Eskimez, Liyang Lu, Ming Gong (YIMING), Linjun Shou (寿林钧), Hong Qu, Michael Zeng

arXiv:2004.04438 | April 2020

Publication

A Study of Discriminative Training For HMM-based Online Handwritten Chinese/Japanese Character Recognition

Yongqiang Wang, Qiang Huo, Yu Shi

ICFHR 2010 | November 2010

Publication

A Study of Irrelevant Variability Normalization Based Training and Unsupervised Online Adaptation for LVCSR

Guangchuan Shi, Yu Shi, Qiang Huo

Interspeech 2010 | September 2010

Publication

A Study of Feature Design for Online Handwritten Chinese Character Recognition based on Continuous-Density Hidden Markov Models

Lei Ma, Qiang Huo, Yu Shi

ICDAR 2009 | July 2009

Publication

A Symbol Graph Based Handwritten Math Expression Recognition

Yu Shi, Frank Soong

ICPR 2008 | December 2008

Publication

GPU-Accelerated Gaussian Clustering for fMPE Discriminative Training

Yu Shi, Frank Seide, Frank Soong

InterSpeech 2008 | September 2008

Publication

Symbol Graph Based Discriminative Training and Rescoring for Improved Math Symbol Recognition

Yu Shi, Frank Soong

ICASSP 2008 | March 2008

Publication

Approximate Word-Lattice Indexing with Text Indexers: Time-Anchored Lattice Expansion

Peng Yu, Yu Shi, Frank Seide, Roger (Peng) Yu

Proc. IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP) | March 2008

Publication

Towards Spoken-Document Retrieval for the Enterprise: Approximate Word-Lattice Indexing with Text Indexers

Frank Seide, Peng Yu, Roger (Peng) Yu, Yu Shi

Proc. IEEE Workshop on Automatic Speech Recognition and Understanding (ASRU) | December 2007

Publication

Robust Voice Activity Detection Based on Noise Eigenspace

Dongwen Ying, Yu Shi, Xuguang Lu, Jianwu Dang, Frank Soong

Acoustical Science and Technology | November 2007, Vol 28(6): pp. 413-423

Publication

A Unified Framework for Symbol Segmentation and Recognition of Handwritten Mathematical Expressions

Yu Shi, Haiyang Li, Frank Soong

ICDAR 2007 | September 2007

Publication

A Segmentation Posterior based Endpointing Algorithm

Yanlu Xie, Yu Shi (yushi), Frank Soong, Beiqian Dai

ICASSP 2007 | April 2007

Publication

A Robust Voice Activity Detection based on Noise Eigenspace Projection

Dongwen Ying, Yu Shi, Frank Soong, Jianwu Dang, Xugang Lu

ISCSLP 2006 | December 2006

Publication

Integrating Hypotheses of Multiple Recognizers for Improving Mandarin LVCSR Performance

Yu Shi, Frank Soong, Jian-Lai Shou

ISCSLP 2006 | December 2006

Publication

Auto-Segmentation based VAD for Robust ASR

Yu Shi, Frank Soong, Jianlai Zhou

Interspeech 2006 - ICSLP | September 2006

Publication

Auto-Segmentation based Partitioning and Clustering Approach to Robust Endpointing

Yu Shi, Frank Soong, Jianlai Zhou

ICASSP 2006 | May 2006

Publication

A Robust VAD Based Upon Noise Eigenspace Projection

Dongwen Ying, Yu Shi, Frank Soong, Jianwu Dang

Part of the Lecture Notes in Computer Science book series | April 2006

Publication Publication

Segmental Tonal Modeling for Phone Set Design in Mandarin LVCSR

Chao Huang, Yu Shi, Jianlai Zhou, Min Chu, Terry Wang, Eric Chang

2004 IEEE International Conference on Acoustics, Speech, and Signal Processing | May 2004

Publication Publication

Studies in Massively Speaker-Specific Speech Recognition

Yu Shi, Eric Chang

IEEE | May 2004

Publication

Tone Articulation Modeling for Mandarin Spontaneous Speech Recognition

Jian-lai Zhou, Ye Tian, Yu Shi, Chao Huang, Eric Chang

IEEE | May 2004

Publication

Spectrogram-Based Formant Tracking via Particle Filters

Yu Shi, Eric Chang

ICASSP 2003 | April 2003

Publication

A System for Spoken Query Information Retrieval on Mobile Devices

Eric Chang, Frank Seide, Helen M. Meng, Zhuoran Chen, Yu Shi, Yuk-Chi Li

IEEE Transactions on Speech and Audio Processing | October 2002, Vol 10(8): pp. 531-541

Publication

Power Spectral Density Based Channel Equalization of Large Speech Database for Concatenative TTS System

Yu Shi, Eric Chang, Hu Peng, Min Chu

ICSLP 2002 | September 2002

Publication

Speech Lab in a Box: A Mandarin Speech Toolbox to Jumpstart Speech Related Research

Eric Chang, Yu Shi, Jianlai Zhouo, Chao Huang

EuroSpeech 2001 | October 2001

Publication

i-Code V2: An Autoregressive Generation Framework over Vision, Language, and Speech Data

Ziyi Yang, Mahmoud Khademi, Yichong Xu, Reid Pryzant, Yuwei Fang, Chenguang Zhu, Dongdong Chen, Yao Qian, Mei Gao, Yi-Ling Chen, Robert Gmyr, Naoyuki Kanda, Noel Codella, Bin Xiao, Yu Shi, Lu Yuan, Takuya Yoshioka, Michael Zeng, Xuedong Huang

North American Chapter of the Association for Computational Linguistics (NAACL) 2024 | May 2023

Publication

i-Code: An Integrative and Composable Multimodal Learning Framework

Ziyi Yang, Yuwei Fang, Chenguang Zhu, Reid Pryzant, Dongdong Chen, Yu Shi, Yichong Xu, Yao Qian, Mei Gao, Yi-Ling Chen, Liyang Lu, Yujia Xie, Robert Gmyr, Noel Codella, Naoyuki Kanda, Bin Xiao, Lu Yuan, Takuya Yoshioka, Michael Zeng, Xuedong Huang

AAAI 2023 | February 2023

Publication

Breaking trade-offs in speech separation with sparsely-gated mixture of experts

Xiaofei Wang, Zhuo Chen, Yu Shi, Jian Wu, Naoyuki Kanda, Takuya Yoshioka

arXiv:2211.06493 | November 2022

Preprint

Knowledge Distillation for Mixture of Experts Models in Speech Recognition

Felipe Cruz Salinas, Kenichi Kumatani, Robert Gmyr, Linquan Liu, Yu Shi

MSR-TR-2022-6 | May 2022

Published by Microsoft

Publication

Building a great multi-lingual teacher with sparsely-gated mixture of experts for speech recognition

Kenichi Kumatani, Robert Gmyr, Felipe Cruz Salinas, Linquan Liu, Wei Zuo , Devang Patel, Eric Sun, Yu Shi

CoRR | December 2021, Vol abs/2112.05820

Publication

Florence: A New Foundation Model for Computer Vision

Lu Yuan, Dongdong Chen, Yi-Ling Chen, Noel Codella, Xiyang Dai, Jianfeng Gao, Houdong Hu, Xuedong Huang, Boxin Li, Chunyuan Li, Ce Liu, Mengchen Liu, Zicheng Liu, Yumao Lu, Yu Shi, Lijuan Wang, Jianfeng Wang, Bin Xiao, Zhen Xiao, Jianwei Yang, Michael Zeng, Luowei Zhou, Pengchuan Zhang

arXiv preprint arXiv:2111.11432 | November 2021

Publication Project

Generating Human Readable Transcript for Automatic Speech Recognition with Pre-trained Language Model

Junwei Liao, Yu Shi, Ming Gong (YIMING), Linjun Shou (寿林钧), Sefik Emre Eskimez, Liyang Lu, Hong Qu, Michael Zeng

ICASSP2021 | June 2021

Publication

Speech-language Pre-training for End-to-end Spoken Language Understanding

Yao Qian, Ximo Bian, Yu Shi, Naoyuki Kanda, Leo Shen, Zhen Xiao, Michael Zeng

ICASSP 2021 | June 2021

Publication

Improving Zero-shot Neural Machine Translation on Language-specific Encoders-Decoders

Junwei Liao, Yu Shi, Ming Gong (YIMING), Linjun Shou (寿林钧), Hong Qu, Michael Zeng

arXiv:2102.06578 | February 2021

Publication

Discriminative Transfer Learning for Optimizing ASR and Semantic Labeling in Task-oriented Spoken Dialog

Yao Qian, Yu Shi, Michael Zeng

InterSpeech2020 | October 2020

Publication

MaP: A Matrix-based Prediction Approach to Improve Span Extraction in Machine Reading Comprehension

Huaishao Luo, Yu Shi, Ming Gong (YIMING), Linjun Shou (寿林钧), Tianrui Li

AACL2020 | September 2020

Publication

Improving Readability for Automatic Speech Recognition Transcription

Junwei Liao, Yu Shi, Sefik Emre Eskimez, Liyang Lu, Ming Gong (YIMING), Linjun Shou (寿林钧), Hong Qu, Michael Zeng

arXiv:2004.04438 | April 2020

Publication

A Study of Discriminative Training For HMM-based Online Handwritten Chinese/Japanese Character Recognition

Yongqiang Wang, Qiang Huo, Yu Shi

ICFHR 2010 | November 2010

Publication

A Study of Irrelevant Variability Normalization Based Training and Unsupervised Online Adaptation for LVCSR

Guangchuan Shi, Yu Shi, Qiang Huo

Interspeech 2010 | September 2010

Publication

A Study of Feature Design for Online Handwritten Chinese Character Recognition based on Continuous-Density Hidden Markov Models

Lei Ma, Qiang Huo, Yu Shi

ICDAR 2009 | July 2009

Publication

A Symbol Graph Based Handwritten Math Expression Recognition

Yu Shi, Frank Soong

ICPR 2008 | December 2008

Publication

Robust Voice Activity Detection Based on Noise Eigenspace

Dongwen Ying, Yu Shi, Xuguang Lu, Jianwu Dang, Frank Soong

Acoustical Science and Technology | November 2007, Vol 28(6): pp. 413-423

Publication

A Unified Framework for Symbol Segmentation and Recognition of Handwritten Mathematical Expressions

Yu Shi, Haiyang Li, Frank Soong

ICDAR 2007 | September 2007

Publication

A Segmentation Posterior based Endpointing Algorithm

Yanlu Xie, Yu Shi (yushi), Frank Soong, Beiqian Dai

ICASSP 2007 | April 2007

Publication

A Robust Voice Activity Detection based on Noise Eigenspace Projection

Dongwen Ying, Yu Shi, Frank Soong, Jianwu Dang, Xugang Lu

ISCSLP 2006 | December 2006

Publication

Integrating Hypotheses of Multiple Recognizers for Improving Mandarin LVCSR Performance

Yu Shi, Frank Soong, Jian-Lai Shou

ISCSLP 2006 | December 2006

Publication

Auto-Segmentation based VAD for Robust ASR

Yu Shi, Frank Soong, Jianlai Zhou

Interspeech 2006 - ICSLP | September 2006

Publication

Auto-Segmentation based Partitioning and Clustering Approach to Robust Endpointing

Yu Shi, Frank Soong, Jianlai Zhou

ICASSP 2006 | May 2006

Publication

Spectrogram-Based Formant Tracking via Particle Filters

Yu Shi, Eric Chang

ICASSP 2003 | April 2003

Publication

Power Spectral Density Based Channel Equalization of Large Speech Database for Concatenative TTS System

Yu Shi, Eric Chang, Hu Peng, Min Chu

ICSLP 2002 | September 2002

Publication

i-Code V2: An Autoregressive Generation Framework over Vision, Language, and Speech Data

Ziyi Yang, Mahmoud Khademi, Yichong Xu, Reid Pryzant, Yuwei Fang, Chenguang Zhu, Dongdong Chen, Yao Qian, Mei Gao, Yi-Ling Chen, Robert Gmyr, Naoyuki Kanda, Noel Codella, Bin Xiao, Yu Shi, Lu Yuan, Takuya Yoshioka, Michael Zeng, Xuedong Huang

North American Chapter of the Association for Computational Linguistics (NAACL) 2024 | May 2023

Publication

i-Code: An Integrative and Composable Multimodal Learning Framework

Ziyi Yang, Yuwei Fang, Chenguang Zhu, Reid Pryzant, Dongdong Chen, Yu Shi, Yichong Xu, Yao Qian, Mei Gao, Yi-Ling Chen, Liyang Lu, Yujia Xie, Robert Gmyr, Noel Codella, Naoyuki Kanda, Bin Xiao, Lu Yuan, Takuya Yoshioka, Michael Zeng, Xuedong Huang

AAAI 2023 | February 2023

Publication

Breaking trade-offs in speech separation with sparsely-gated mixture of experts

Xiaofei Wang, Zhuo Chen, Yu Shi, Jian Wu, Naoyuki Kanda, Takuya Yoshioka

arXiv:2211.06493 | November 2022

Preprint

Generating Human Readable Transcript for Automatic Speech Recognition with Pre-trained Language Model

Junwei Liao, Yu Shi, Ming Gong (YIMING), Linjun Shou (寿林钧), Sefik Emre Eskimez, Liyang Lu, Hong Qu, Michael Zeng

ICASSP2021 | June 2021

Publication

Speech-language Pre-training for End-to-end Spoken Language Understanding

Yao Qian, Ximo Bian, Yu Shi, Naoyuki Kanda, Leo Shen, Zhen Xiao, Michael Zeng

ICASSP 2021 | June 2021

Publication

Discriminative Transfer Learning for Optimizing ASR and Semantic Labeling in Task-oriented Spoken Dialog

Yao Qian, Yu Shi, Michael Zeng

InterSpeech2020 | October 2020

Publication

Improving Readability for Automatic Speech Recognition Transcription

Junwei Liao, Yu Shi, Sefik Emre Eskimez, Liyang Lu, Ming Gong (YIMING), Linjun Shou (寿林钧), Hong Qu, Michael Zeng

arXiv:2004.04438 | April 2020

Publication

A Study of Irrelevant Variability Normalization Based Training and Unsupervised Online Adaptation for LVCSR

Guangchuan Shi, Yu Shi, Qiang Huo

Interspeech 2010 | September 2010

Publication

GPU-Accelerated Gaussian Clustering for fMPE Discriminative Training

Yu Shi, Frank Seide, Frank Soong

InterSpeech 2008 | September 2008

Publication

Towards Spoken-Document Retrieval for the Enterprise: Approximate Word-Lattice Indexing with Text Indexers

Frank Seide, Peng Yu, Roger (Peng) Yu, Yu Shi

Proc. IEEE Workshop on Automatic Speech Recognition and Understanding (ASRU) | December 2007

Publication

Robust Voice Activity Detection Based on Noise Eigenspace

Dongwen Ying, Yu Shi, Xuguang Lu, Jianwu Dang, Frank Soong

Acoustical Science and Technology | November 2007, Vol 28(6): pp. 413-423

Publication

A Segmentation Posterior based Endpointing Algorithm

Yanlu Xie, Yu Shi (yushi), Frank Soong, Beiqian Dai

ICASSP 2007 | April 2007

Publication

A Robust Voice Activity Detection based on Noise Eigenspace Projection

Dongwen Ying, Yu Shi, Frank Soong, Jianwu Dang, Xugang Lu

ISCSLP 2006 | December 2006

Publication

Integrating Hypotheses of Multiple Recognizers for Improving Mandarin LVCSR Performance

Yu Shi, Frank Soong, Jian-Lai Shou

ISCSLP 2006 | December 2006

Publication

Auto-Segmentation based VAD for Robust ASR

Yu Shi, Frank Soong, Jianlai Zhou

Interspeech 2006 - ICSLP | September 2006

Publication

Auto-Segmentation based Partitioning and Clustering Approach to Robust Endpointing

Yu Shi, Frank Soong, Jianlai Zhou

ICASSP 2006 | May 2006

Publication

Spectrogram-Based Formant Tracking via Particle Filters

Yu Shi, Eric Chang

ICASSP 2003 | April 2003

Publication

Power Spectral Density Based Channel Equalization of Large Speech Database for Concatenative TTS System

Yu Shi, Eric Chang, Hu Peng, Min Chu

ICSLP 2002 | September 2002

Publication

i-Code V2: An Autoregressive Generation Framework over Vision, Language, and Speech Data

Ziyi Yang, Mahmoud Khademi, Yichong Xu, Reid Pryzant, Yuwei Fang, Chenguang Zhu, Dongdong Chen, Yao Qian, Mei Gao, Yi-Ling Chen, Robert Gmyr, Naoyuki Kanda, Noel Codella, Bin Xiao, Yu Shi, Lu Yuan, Takuya Yoshioka, Michael Zeng, Xuedong Huang

North American Chapter of the Association for Computational Linguistics (NAACL) 2024 | May 2023

Publication

i-Code: An Integrative and Composable Multimodal Learning Framework

Ziyi Yang, Yuwei Fang, Chenguang Zhu, Reid Pryzant, Dongdong Chen, Yu Shi, Yichong Xu, Yao Qian, Mei Gao, Yi-Ling Chen, Liyang Lu, Yujia Xie, Robert Gmyr, Noel Codella, Naoyuki Kanda, Bin Xiao, Lu Yuan, Takuya Yoshioka, Michael Zeng, Xuedong Huang

AAAI 2023 | February 2023

Publication

Florence: A New Foundation Model for Computer Vision

Lu Yuan, Dongdong Chen, Yi-Ling Chen, Noel Codella, Xiyang Dai, Jianfeng Gao, Houdong Hu, Xuedong Huang, Boxin Li, Chunyuan Li, Ce Liu, Mengchen Liu, Zicheng Liu, Yumao Lu, Yu Shi, Lijuan Wang, Jianfeng Wang, Bin Xiao, Zhen Xiao, Jianwei Yang, Michael Zeng, Luowei Zhou, Pengchuan Zhang

arXiv preprint arXiv:2111.11432 | November 2021

Publication Project

i-Code V2: An Autoregressive Generation Framework over Vision, Language, and Speech Data

Ziyi Yang, Mahmoud Khademi, Yichong Xu, Reid Pryzant, Yuwei Fang, Chenguang Zhu, Dongdong Chen, Yao Qian, Mei Gao, Yi-Ling Chen, Robert Gmyr, Naoyuki Kanda, Noel Codella, Bin Xiao, Yu Shi, Lu Yuan, Takuya Yoshioka, Michael Zeng, Xuedong Huang

North American Chapter of the Association for Computational Linguistics (NAACL) 2024 | May 2023

Publication

i-Code: An Integrative and Composable Multimodal Learning Framework

Ziyi Yang, Yuwei Fang, Chenguang Zhu, Reid Pryzant, Dongdong Chen, Yu Shi, Yichong Xu, Yao Qian, Mei Gao, Yi-Ling Chen, Liyang Lu, Yujia Xie, Robert Gmyr, Noel Codella, Naoyuki Kanda, Bin Xiao, Lu Yuan, Takuya Yoshioka, Michael Zeng, Xuedong Huang

AAAI 2023 | February 2023

Publication

Breaking trade-offs in speech separation with sparsely-gated mixture of experts

Xiaofei Wang, Zhuo Chen, Yu Shi, Jian Wu, Naoyuki Kanda, Takuya Yoshioka

arXiv:2211.06493 | November 2022

Preprint

Generating Human Readable Transcript for Automatic Speech Recognition with Pre-trained Language Model

Junwei Liao, Yu Shi, Ming Gong (YIMING), Linjun Shou (寿林钧), Sefik Emre Eskimez, Liyang Lu, Hong Qu, Michael Zeng

ICASSP2021 | June 2021

Publication

Speech-language Pre-training for End-to-end Spoken Language Understanding

Yao Qian, Ximo Bian, Yu Shi, Naoyuki Kanda, Leo Shen, Zhen Xiao, Michael Zeng

ICASSP 2021 | June 2021

Publication

Improving Zero-shot Neural Machine Translation on Language-specific Encoders-Decoders

Junwei Liao, Yu Shi, Ming Gong (YIMING), Linjun Shou (寿林钧), Hong Qu, Michael Zeng

arXiv:2102.06578 | February 2021

Publication

Listen, Look and Deliberate: Visual context-aware speech recognition using pre-trained text-video representations

Shahram Ghorbani, Yashesh Gaur, Yu Shi, Jinyu Li

Spoken Language Technology Workshop | January 2021

Publication

Discriminative Transfer Learning for Optimizing ASR and Semantic Labeling in Task-oriented Spoken Dialog

Yao Qian, Yu Shi, Michael Zeng

InterSpeech2020 | October 2020

Publication

MaP: A Matrix-based Prediction Approach to Improve Span Extraction in Machine Reading Comprehension

Huaishao Luo, Yu Shi, Ming Gong (YIMING), Linjun Shou (寿林钧), Tianrui Li

AACL2020 | September 2020

Publication

Mixed-Lingual Pre-training for Cross-lingual Summarization

Ruochen Xu, Chenguang Zhu, Yu Shi, Michael Zeng, Xuedong Huang

The 1st Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the 10th International Joint Conference on Natural Language Processing (AACL 2020) | September 2020

Publication

Improving Readability for Automatic Speech Recognition Transcription

Junwei Liao, Yu Shi, Sefik Emre Eskimez, Liyang Lu, Ming Gong (YIMING), Linjun Shou (寿林钧), Hong Qu, Michael Zeng

arXiv:2004.04438 | April 2020

Publication

Towards Spoken-Document Retrieval for the Enterprise: Approximate Word-Lattice Indexing with Text Indexers

Frank Seide, Peng Yu, Roger (Peng) Yu, Yu Shi

Proc. IEEE Workshop on Automatic Speech Recognition and Understanding (ASRU) | December 2007

Publication

Studies in Massively Speaker-Specific Speech Recognition

Yu Shi, Eric Chang

IEEE | May 2004

Publication

Segmental Tonal Modeling for Phone Set Design in Mandarin LVCSR

Chao Huang, Yu Shi, Jianlai Zhou, Min Chu, Terry Wang, Eric Chang

2004 IEEE International Conference on Acoustics, Speech, and Signal Processing | May 2004

Publication Publication

Tone Articulation Modeling for Mandarin Spontaneous Speech Recognition

Jian-lai Zhou, Ye Tian, Yu Shi, Chao Huang, Eric Chang

IEEE | May 2004

Publication

A System for Spoken Query Information Retrieval on Mobile Devices

Eric Chang, Frank Seide, Helen M. Meng, Zhuoran Chen, Yu Shi, Yuk-Chi Li

IEEE Transactions on Speech and Audio Processing | October 2002, Vol 10(8): pp. 531-541

Publication

Speech Lab in a Box: A Mandarin Speech Toolbox to Jumpstart Speech Related Research

Eric Chang, Yu Shi, Jianlai Zhouo, Chao Huang

EuroSpeech 2001 | October 2001

Publication

A Study of Discriminative Training For HMM-based Online Handwritten Chinese/Japanese Character Recognition

Yongqiang Wang, Qiang Huo, Yu Shi

ICFHR 2010 | November 2010

Publication

A Study of Irrelevant Variability Normalization Based Training and Unsupervised Online Adaptation for LVCSR

Guangchuan Shi, Yu Shi, Qiang Huo

Interspeech 2010 | September 2010

Publication

A Study of Feature Design for Online Handwritten Chinese Character Recognition based on Continuous-Density Hidden Markov Models

Lei Ma, Qiang Huo, Yu Shi

ICDAR 2009 | July 2009

Publication

A Symbol Graph Based Handwritten Math Expression Recognition

Yu Shi, Frank Soong

ICPR 2008 | December 2008

Publication

Approximate Word-Lattice Indexing with Text Indexers: Time-Anchored Lattice Expansion

Peng Yu, Yu Shi, Frank Seide, Roger (Peng) Yu

Proc. IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP) | March 2008

Publication

Robust Voice Activity Detection Based on Noise Eigenspace

Dongwen Ying, Yu Shi, Xuguang Lu, Jianwu Dang, Frank Soong

Acoustical Science and Technology | November 2007, Vol 28(6): pp. 413-423

Publication

A Unified Framework for Symbol Segmentation and Recognition of Handwritten Mathematical Expressions

Yu Shi, Haiyang Li, Frank Soong

ICDAR 2007 | September 2007

Publication

Symbol Graph Based Discriminative Training and Rescoring for Improved Math Symbol Recognition

Yu Shi, Frank Soong

ICASSP 2008 | March 2008

Publication

Towards Spoken-Document Retrieval for the Enterprise: Approximate Word-Lattice Indexing with Text Indexers

Frank Seide, Peng Yu, Roger (Peng) Yu, Yu Shi

Proc. IEEE Workshop on Automatic Speech Recognition and Understanding (ASRU) | December 2007

Publication

A System for Spoken Query Information Retrieval on Mobile Devices

Eric Chang, Frank Seide, Helen M. Meng, Zhuoran Chen, Yu Shi, Yuk-Chi Li

IEEE Transactions on Speech and Audio Processing | October 2002, Vol 10(8): pp. 531-541

Publication

A Robust VAD Based Upon Noise Eigenspace Projection

Dongwen Ying, Yu Shi, Frank Soong, Jianwu Dang

Part of the Lecture Notes in Computer Science book series | April 2006

Publication Publication

i-Code V2: An Autoregressive Generation Framework over Vision, Language, and Speech Data

Ziyi Yang, Mahmoud Khademi, Yichong Xu, Reid Pryzant, Yuwei Fang, Chenguang Zhu, Dongdong Chen, Yao Qian, Mei Gao, Yi-Ling Chen, Robert Gmyr, Naoyuki Kanda, Noel Codella, Bin Xiao, Yu Shi, Lu Yuan, Takuya Yoshioka, Michael Zeng, Xuedong Huang

North American Chapter of the Association for Computational Linguistics (NAACL) 2024 | May 2023

Publication

i-Code: An Integrative and Composable Multimodal Learning Framework

Ziyi Yang, Yuwei Fang, Chenguang Zhu, Reid Pryzant, Dongdong Chen, Yu Shi, Yichong Xu, Yao Qian, Mei Gao, Yi-Ling Chen, Liyang Lu, Yujia Xie, Robert Gmyr, Noel Codella, Naoyuki Kanda, Bin Xiao, Lu Yuan, Takuya Yoshioka, Michael Zeng, Xuedong Huang

AAAI 2023 | February 2023

Publication

Generating Human Readable Transcript for Automatic Speech Recognition with Pre-trained Language Model

Junwei Liao, Yu Shi, Ming Gong (YIMING), Linjun Shou (寿林钧), Sefik Emre Eskimez, Liyang Lu, Hong Qu, Michael Zeng

ICASSP2021 | June 2021

Publication

Speech-language Pre-training for End-to-end Spoken Language Understanding

Yao Qian, Ximo Bian, Yu Shi, Naoyuki Kanda, Leo Shen, Zhen Xiao, Michael Zeng

ICASSP 2021 | June 2021

Publication

Improving Zero-shot Neural Machine Translation on Language-specific Encoders-Decoders

Junwei Liao, Yu Shi, Ming Gong (YIMING), Linjun Shou (寿林钧), Hong Qu, Michael Zeng

arXiv:2102.06578 | February 2021

Publication

Listen, Look and Deliberate: Visual context-aware speech recognition using pre-trained text-video representations

Shahram Ghorbani, Yashesh Gaur, Yu Shi, Jinyu Li

Spoken Language Technology Workshop | January 2021

Publication

Discriminative Transfer Learning for Optimizing ASR and Semantic Labeling in Task-oriented Spoken Dialog

Yao Qian, Yu Shi, Michael Zeng

InterSpeech2020 | October 2020

Publication

MaP: A Matrix-based Prediction Approach to Improve Span Extraction in Machine Reading Comprehension

Huaishao Luo, Yu Shi, Ming Gong (YIMING), Linjun Shou (寿林钧), Tianrui Li

AACL2020 | September 2020

Publication

Improving Readability for Automatic Speech Recognition Transcription

Junwei Liao, Yu Shi, Sefik Emre Eskimez, Liyang Lu, Ming Gong (YIMING), Linjun Shou (寿林钧), Hong Qu, Michael Zeng

arXiv:2004.04438 | April 2020

Publication

A Study of Discriminative Training For HMM-based Online Handwritten Chinese/Japanese Character Recognition

Yongqiang Wang, Qiang Huo, Yu Shi

ICFHR 2010 | November 2010

Publication

A Study of Irrelevant Variability Normalization Based Training and Unsupervised Online Adaptation for LVCSR

Guangchuan Shi, Yu Shi, Qiang Huo

Interspeech 2010 | September 2010

Publication

A Study of Feature Design for Online Handwritten Chinese Character Recognition based on Continuous-Density Hidden Markov Models

Lei Ma, Qiang Huo, Yu Shi

ICDAR 2009 | July 2009

Publication

A Symbol Graph Based Handwritten Math Expression Recognition

Yu Shi, Frank Soong

ICPR 2008 | December 2008

Publication

GPU-Accelerated Gaussian Clustering for fMPE Discriminative Training

Yu Shi, Frank Seide, Frank Soong

InterSpeech 2008 | September 2008

Publication

Symbol Graph Based Discriminative Training and Rescoring for Improved Math Symbol Recognition

Yu Shi, Frank Soong

ICASSP 2008 | March 2008

Publication

Approximate Word-Lattice Indexing with Text Indexers: Time-Anchored Lattice Expansion

Peng Yu, Yu Shi, Frank Seide, Roger (Peng) Yu

Proc. IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP) | March 2008

Publication

Towards Spoken-Document Retrieval for the Enterprise: Approximate Word-Lattice Indexing with Text Indexers

Frank Seide, Peng Yu, Roger (Peng) Yu, Yu Shi

Proc. IEEE Workshop on Automatic Speech Recognition and Understanding (ASRU) | December 2007

Publication

A Unified Framework for Symbol Segmentation and Recognition of Handwritten Mathematical Expressions

Yu Shi, Haiyang Li, Frank Soong

ICDAR 2007 | September 2007

Publication

A Segmentation Posterior based Endpointing Algorithm

Yanlu Xie, Yu Shi (yushi), Frank Soong, Beiqian Dai

ICASSP 2007 | April 2007

Publication

A Robust Voice Activity Detection based on Noise Eigenspace Projection

Dongwen Ying, Yu Shi, Frank Soong, Jianwu Dang, Xugang Lu

ISCSLP 2006 | December 2006

Publication

Integrating Hypotheses of Multiple Recognizers for Improving Mandarin LVCSR Performance

Yu Shi, Frank Soong, Jian-Lai Shou

ISCSLP 2006 | December 2006

Publication

Auto-Segmentation based VAD for Robust ASR

Yu Shi, Frank Soong, Jianlai Zhou

Interspeech 2006 - ICSLP | September 2006

Publication

Auto-Segmentation based Partitioning and Clustering Approach to Robust Endpointing

Yu Shi, Frank Soong, Jianlai Zhou

ICASSP 2006 | May 2006

Publication

A Robust VAD Based Upon Noise Eigenspace Projection

Dongwen Ying, Yu Shi, Frank Soong, Jianwu Dang

Part of the Lecture Notes in Computer Science book series | April 2006

Publication Publication

Studies in Massively Speaker-Specific Speech Recognition

Yu Shi, Eric Chang

IEEE | May 2004

Publication

Segmental Tonal Modeling for Phone Set Design in Mandarin LVCSR

Chao Huang, Yu Shi, Jianlai Zhou, Min Chu, Terry Wang, Eric Chang

2004 IEEE International Conference on Acoustics, Speech, and Signal Processing | May 2004

Publication Publication

Tone Articulation Modeling for Mandarin Spontaneous Speech Recognition

Jian-lai Zhou, Ye Tian, Yu Shi, Chao Huang, Eric Chang

IEEE | May 2004

Publication

Spectrogram-Based Formant Tracking via Particle Filters

Yu Shi, Eric Chang

ICASSP 2003 | April 2003

Publication

Power Spectral Density Based Channel Equalization of Large Speech Database for Concatenative TTS System

Yu Shi, Eric Chang, Hu Peng, Min Chu

ICSLP 2002 | September 2002

Publication

Speech Lab in a Box: A Mandarin Speech Toolbox to Jumpstart Speech Related Research

Eric Chang, Yu Shi, Jianlai Zhouo, Chao Huang

EuroSpeech 2001 | October 2001

Publication

Breaking trade-offs in speech separation with sparsely-gated mixture of experts

Xiaofei Wang, Zhuo Chen, Yu Shi, Jian Wu, Naoyuki Kanda, Takuya Yoshioka

arXiv:2211.06493 | November 2022

Preprint

Building a great multi-lingual teacher with sparsely-gated mixture of experts for speech recognition

Kenichi Kumatani, Robert Gmyr, Felipe Cruz Salinas, Linquan Liu, Wei Zuo , Devang Patel, Eric Sun, Yu Shi

CoRR | December 2021, Vol abs/2112.05820

Publication

Florence: A New Foundation Model for Computer Vision

Lu Yuan, Dongdong Chen, Yi-Ling Chen, Noel Codella, Xiyang Dai, Jianfeng Gao, Houdong Hu, Xuedong Huang, Boxin Li, Chunyuan Li, Ce Liu, Mengchen Liu, Zicheng Liu, Yumao Lu, Yu Shi, Lijuan Wang, Jianfeng Wang, Bin Xiao, Zhen Xiao, Jianwei Yang, Michael Zeng, Luowei Zhou, Pengchuan Zhang

arXiv preprint arXiv:2111.11432 | November 2021

Publication Project

Mixed-Lingual Pre-training for Cross-lingual Summarization

Ruochen Xu, Chenguang Zhu, Yu Shi, Michael Zeng, Xuedong Huang

The 1st Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the 10th International Joint Conference on Natural Language Processing (AACL 2020) | September 2020

Publication

Robust Voice Activity Detection Based on Noise Eigenspace

Dongwen Ying, Yu Shi, Xuguang Lu, Jianwu Dang, Frank Soong

Acoustical Science and Technology | November 2007, Vol 28(6): pp. 413-423

Publication

A System for Spoken Query Information Retrieval on Mobile Devices

Eric Chang, Frank Seide, Helen M. Meng, Zhuoran Chen, Yu Shi, Yuk-Chi Li

IEEE Transactions on Speech and Audio Processing | October 2002, Vol 10(8): pp. 531-541

Publication

Knowledge Distillation for Mixture of Experts Models in Speech Recognition

Felipe Cruz Salinas, Kenichi Kumatani, Robert Gmyr, Linquan Liu, Yu Shi

MSR-TR-2022-6 | May 2022

Published by Microsoft

Publication