Chenyang Huang

selected publications

1. ACL

   Learning Non-Autoregressive Models from Search for Unsupervised Sentence Summarization

   Puyuan Liu,  Chenyang Huang, and Lili Mou

   ACL 2022

   Abs PDF Code

   Text summarization aims to generate a short summary for an input text. In this work, we propose a Non-Autoregressive Unsupervised Summarization (NAUS) approach, which does not require parallel data for training. Our NAUS first performs edit-based search towards a heuristically defined score, and generates a summary as pseudo-groundtruth. Then, we train an encoder-only non-autoregressive Transformer based on the search result. We also proposed a dynamic programming approach for length-control decoding, which is important for the summarization task. Experiments on the Gigaword headline generation and DUC2004 datasets show that NAUS achieves state-of-the-art performance for unsupervised summarization, yet largely improving inference efficiency. Further, our algorithm is able to perform length-transfer summary generation.

2. AAAI

   Non-Autoregressive Translation with Layer-Wise Prediction and Deep Supervision

   Chenyang Huang, Hao Zhou, Osmar R Zaı̈ane, Lili Mou, and Lei Li

   AAAI 2022

   Abs PDF Code

   How do we perform efficient inference while retaining high translation quality? Existing neural machine translation models, such as Transformer, achieve high performance, but they decode words one by one, which is inefficient. Recent non-autoregressive translation models speed up the inference, but their quality is still inferior. In this work, we propose DSLP, a highly efficient and high-performance model for machine translation. The key insight is to train a non-autoregressive Transformer with Deep Supervision and feed additional Layer-wise Predictions. We conducted extensive experiments on four translation tasks (both directions of WMT’14 EN-DE and WMT’16 EN-RO). Results show that our approach consistently improves the BLEU scores compared with respective base models. Specifically, our best variant outperforms the autoregressive model on three translation tasks, while being 14.8 times more efficient in inference.

3. NAACL

   Seq2Emo: A Sequence to Multi-Label Emotion Classification Model

   Chenyang Huang, Amine Trabelsi, Xuebin Qin, Nawshad Farruque, Lili Mou, and Osmar R Zaiane

   NAACL 2021

   Abs PDF Code

   Multi-label emotion classification is an important task in NLP and is essential to many applications. In this work, we propose a sequence-to-emotion (Seq2Emo) approach, which implicitly models emotion correlations in a bi-directional decoder. Experiments on SemEval’18 and GoEmotions datasets show that our approach outperforms state-of-the-art methods (without using external data). In particular, Seq2Emo outperforms the binary relevance (BR) and classifier chain (CC) approaches in a fair setting.

4. CIKM

   Simulated Annealing for Emotional Dialogue Systems

   Chengzhang Dong,  Chenyang Huang, Osmar Zaı̈ane, and Lili Mou

   CIKM 2021

   Abs PDF Code

   Explicitly modeling emotions in dialogue generation has important applications, such as building empathetic personal companions. In this study, we consider the task of expressing a specific emotion for dialogue generation. Previous approaches take the emotion as a training signal, which may be ignored during inference. Here, we propose a search-based emotional dialogue system by simulated annealing (SA). Specifically, we first define a scoring function that combines contextual coherence and emotional correctness. Then, SA iteratively edits a general response, and search for a generation with a high score. In this way, we enforce the presence of the desired emotion. We evaluate our system on the NLPCC2017 dataset. The proposed method shows about 12% improvements in emotion accuracy compared with the previous state-of-the-art method, without hurting the generation quality (measured by BLEU).

5. ACL

   Optimizing Deeper Transformers on Small Datasets

   Peng Xu, Dhruv Kumar, Wei Yang, Wenjie Zi, Keyi Tang,  Chenyang Huang, Jackie Chi Kit Cheung, Simon JD Prince, and Yanshuai Cao

   ACL 2021

   Abs PDF Code

   It is a common belief that training deep transformers from scratch requires large datasets. Consequently, for small datasets, people usually use shallow and simple additional layers on top of pre-trained models during fine-tuning. This work shows that this does not always need to be the case: with proper initialization and optimization, the benefits of very deep transformers can carry over to challenging tasks with small datasets, including Text-to-SQL semantic parsing and logical reading comprehension. In particular, we successfully train layers of transformers, comprising fine-tuned layers from pre-trained RoBERTa and relation-aware layers trained from scratch. With fewer training steps and no task-specific pre-training, we obtain the state-of-the-art performance on the challenging cross-domain Text-to-SQL parsing benchmark Spider. We achieve this by deriving a novel Data-dependent Transformer Fixed-update initialization scheme (DT-Fixup), inspired by the prior T-Fixup work. Further error analysis shows that increasing depth can help improve generalization on small datasets for hard cases that require reasoning and structural understanding.

6. SPNLP@ACL

   A Globally Normalized Neural Model for Semantic Parsing

   Chenyang Huang, Wei Yang, Yanshuai Cao, Osmar Zaı̈ane, and Lili Mou

   SPNLP@ACL 2021

   Abs PDF

   In this paper, we propose a globally normalized model for context-free grammar (CFG)-based semantic parsing. Instead of predicting a probability, our model predicts a real-valued score at each step and does not suffer from the label bias problem. Experiments show that our approach outperforms locally normalized models on small datasets, but it does not yield improvement on a large dataset.

7. PR

   U^2-Net: Going Deeper with Nested U-Structure for Salient Object Detection

   Xuebin Qin, Zichen Zhang,  Chenyang Huang, Masood Dehghan, Osmar R Zaiane, and Martin Jagersand

   Pattern Recognition 2020

   Abs PDF Code

   In this paper, we design a simple yet powerful deep network architecture, U2-Net, for salient object detection (SOD). The architecture of our U2-Net is a two-level nested U-structure. The design has the following advantages: (1) it is able to capture more contextual information from different scales thanks to the mixture of receptive fields of different sizes in our proposed ReSidual U-blocks (RSU), (2) it increases the depth of the whole architecture without significantly increasing the computational cost because of the pooling operations used in these RSU blocks. This architecture enables us to train a deep network from scratch without using backbones from image classification tasks. We instantiate two models of the proposed architecture, U2-Net (176.3 MB, 30 FPS on GTX 1080Ti GPU) and U2-Net† (4.7 MB, 40 FPS), to facilitate the usage in different environments. Both models achieve competitive performance on six SOD datasets.

8. CVPR

   Basnet: Boundary-Aware Salient Object Detection

   Xuebin Qin, Zichen Zhang,  Chenyang Huang, Chao Gao, Masood Dehghan, and Martin Jagersand

   CVPR 2019

   Abs PDF Code

   Deep Convolutional Neural Networks have been adopted for salient object detection and achieved the state-of-the-art performance. Most of the previous works however focus on region accuracy but not on the boundary quality. In this paper, we propose a predict-refine architecture, BASNet, and a new hybrid loss for Boundary-Aware Salient object detection. Specifically, the architecture is composed of a densely supervised Encoder-Decoder network and a residual refinement module, which are respectively in charge of saliency prediction and saliency map refinement. The hybrid loss guides the network to learn the transformation between the input image and the ground truth in a three-level hierarchy – pixel-, patch- and map- level – by fusing Binary Cross Entropy (BCE), Structural SIMilarity (SSIM) and Intersection-over-Union (IoU) losses. Equipped with the hybrid loss, the proposed predict-refine architecture is able to effectively segment the salient object regions and accurately predict the fine structures with clear boundaries. Experimental results on six public datasets show that our method outperforms the state-of-the-art methods both in terms of regional and boundary evaluation measures. Our method runs at over 25 fps on a single GPU.

9. NAACL

   Automatic Dialogue Generation with Expressed Emotions

   Chenyang Huang, Osmar R Zaiane, Amine Trabelsi, and Nouha Dziri

   NAACL 2018

   Abs HTML Code

   Dialogue systems, also known as Conversational Agent (CA), are designed to mimic coherent conversations with humans. Most conversational agents are specialized for a specific domain such as travel booking and are typically finite state-based or template-based. Open domain dialogue systems have seen a growing interest in recent years thanks to neural dialogue generation systems, based on deep learning models. These systems basically learn to predict the words and the sentence to respond based on the previous utterances. However, while such a system can generate grammatically correct and human-like answers, the responses are often generic and non-committal instead of being specific and emotionally intelligent. In this work, the objective is to tackle two main problems that are essential towards building emotionally intelligent chatbots:“How to detect the emotions expressed by the human accurately?” and “How can a chatbot express an emotion?”