Publications

GTS: Inference-Time Scaling of Latent Reasoning with a Learnable Gaussian Thought Sampler

Minghan Wang, Ye Bai, Thuy-Trang Vu, Ehsan Shareghi, Gholamreza Haffari

Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing (EMNLP 2026)

Abstract. Inference-time scaling (ITS) for latent reasoning requires sampling multiple candidate latent trajectories, yet current methods typically rely on heuristic perturbations such as dropout or fixed Gaussian noise. We show that stronger perturbations do not necessarily improve candidate-set quality: they often induce larger distribution shifts without producing more decision-useful reasoning paths. This limitation arises because heuristic perturbations inject stochasticity through fixed or hand-designed mechanisms rather than an optimizable, context-conditioned sampling policy, making latent exploration difficult to control. We focus on the sampling side of latent ITS and propose the Gaussian Thought Sampler (GTS), a lightweight module that predicts context-dependent Gaussian perturbations over continuous reasoning states. Trained with GRPO-style policy optimization while keeping the backbone frozen, GTS turns heuristic perturbation into a trainable, context-conditioned sampling policy. Experiments across two latent reasoning backbones and multiple benchmarks show that GTS consistently improves pass@N over heuristic baselines, suggesting that effective sampling for latent ITS requires controlled, optimizable exploration rather than simply amplified stochasticity.

Towards Inference-time Scaling for Continuous Space Reasoning

Minghan Wang, Thuy-Trang Vu, Ehsan Shareghi, Gholamreza Haffari

Findings of the Association for Computational Linguistics: ACL 2026 (64th Annual Meeting of the Association for Computational Linguistics)

Abstract. Inference-time scaling through multiple sample generation in combination with Process- or Outcome-Reward Model (PRM or ORM) re-ranking has proven effective for text-based reasoning in large language models. This paper investigates whether such established techniques can be successfully adapted to reasoning in the continuous space, using COCONUT (CITATION) continuous space reasoning LM as the backbone. We demonstrate the feasibility of generating diverse reasoning paths through dropout-based sampling. Our Pass@N analysis on the generated samples reveals the potential that could enable a significant gain in performance akin to gains observed in the discrete space. However, we highlight unique challenges faced for materializing this gain in the continuous thought space. In particular, working recipes for data generation and training PRM and ORM models in the discrete space unlocks only marginal improvements in the continuous space. Through probing various aspects including geometric properties and trajectory dynamics, we identify the underlying reasons that prevent effective discrimination between correct and incorrect reasoning (essential for the functioning of PRM and ORM). Our findings reveal that current limitations stem from the absence of key inductive biases in continuous thought representations.

Discrete Minds in a Continuous World: Do Language Models Know Time Passes?

Minghan Wang, Ye Bai, Thuy-Trang Vu, Ehsan Shareghi, Gholamreza Haffari

Findings of the Association for Computational Linguistics: EMNLP 2025

Abstract. While Large Language Models (LLMs) excel at temporal reasoning tasks like event ordering and duration estimation, their ability to perceive the actual passage of time remains unexplored. We investigate whether LLMs perceive the passage of time and adapt their decision-making accordingly through three complementary experiments. First, we introduce the Token-Time Hypothesis, positing that LLMs can map discrete token counts to continuous wall-clock time, and validate this through a dialogue duration judgment task. Second, we demonstrate that LLMs could use this awareness to adapt their response length while maintaining accuracy when users express urgency in question answering tasks. Finally, we develop BombRush, an interactive navigation challenge that examines how LLMs modify behavior under progressive time pressure in dynamic environments. Our findings indicate that LLMs possess certain awareness of time passage, enabling them to bridge discrete linguistic tokens and continuous physical time, though this capability varies with model size and reasoning abilities. This work establishes a theoretical foundation for enhancing temporal awareness in LLMs for time-sensitive applications.

SpeechDialogueFactory: A Framework for Natural Speech Dialogue Generation

Minghan Wang, Ye Bai, Yuxia Wang, Thuy-Trang Vu, Ehsan Shareghi, Gholamreza Haffari

Proceedings of the 26th Annual Conference of the International Speech Communication Association (Interspeech 2025)

Abstract. High-quality speech conversational datasets are essential for developing and evaluating Speech-LLMs. However, collecting real-world recordings presents significant challenges including high costs, privacy concerns, and inconsistent quality, while existing synthetic approaches often lack authenticity due to limited acoustic variety and insufficient paralinguistic information. We present SpeechDialogueFactory, a framework that addresses these limitations through a three-stage pipeline: generating comprehensive metadata, creating detailed scripts, and producing utterances enriched with paralinguistic features. Our framework retrieves speaker voices from a voice bank and leverages paralinguistic tags for expressive TTS. We also introduce an automated evaluation protocol that shows strong correlation with human assessments. Experimental results demonstrate that our synthesized dialogues achieve quality comparable to human recordings while offering greater flexibility and control.

Conversational SimulMT: Efficient Simultaneous Translation with Large Language Models

Minghan Wang, Thuy-Trang Vu, Yuxia Wang, Ehsan Shareghi, Gholamreza Haffari

Proceedings of the 22nd International Conference on Spoken Language Translation (IWSLT 2025)

Abstract. Simultaneous machine translation (SimulMT) presents a challenging trade-off between translation quality and latency. Recent studies have shown that LLMs can achieve good performance in SimulMT tasks. However, this often comes at the expense of high inference costs and latency. In this paper, we propose a conversational SimulMT framework to enhance the inference efficiency of LLM-based SimulMT through multi-turn-dialogue-based decoding where source and target chunks interleave in translation history, enabling the reuse of Key-Value cache. To adapt LLMs to the proposed conversational decoding, we create supervised fine-tuning training data by segmenting parallel sentences using an alignment tool and a novel augmentation technique to enhance generalization. Our experiments with Llama2-7b-chat on three SimulMT benchmarks demonstrate that the proposed method empowers the superiority of LLM in translation quality, meanwhile achieving comparable computational latency with specialized SimulMT models.

Proverbs Run in Pairs: Evaluating Proverb Translation Capability of Large Language Model

Minghan Wang, Viet Thanh Pham, Farhad Moghimifar, Thuy-Trang Vu

Findings of the Association for Computational Linguistics: ACL 2025 (63rd Annual Meeting of the Association for Computational Linguistics)

Abstract. Despite achieving remarkable performance, machine translation (MT) research remains underexplored in terms of translating cultural elements in languages, such as idioms, proverbs, and colloquial expressions. This paper investigates the capability of state-of-the-art neural machine translation (NMT) and large language models (LLMs) in translating proverbs, which are deeply rooted in cultural contexts. We construct a translation dataset of standalone proverbs and proverbs in conversation for four language pairs. Our experiments show that the studied models can achieve good translation between languages with similar cultural backgrounds, and LLMs generally outperform NMT models in proverb translation. Furthermore, we find that current automatic evaluation metrics such as BLEU, CHRF++ and COMET are inadequate for reliably assessing the quality of proverb translation, highlighting the need for more culturally aware evaluation metrics.

Exploring the Potential of Multimodal LLM with Knowledge-Intensive Multimodal ASR

Minghan Wang, Yuxia Wang, Thuy-Trang Vu, Ehsan Shareghi, Reza Haf

Findings of the Association for Computational Linguistics: EMNLP 2024

Abstract. Recent advancements in multimodal large language models (MLLMs) have made significant progress in integrating information across various modalities, yet real-world applications in educational and scientific domains remain challenging. This paper introduces the Multimodal Scientific ASR (MS-ASR) task, which focuses on transcribing scientific conference videos by leveraging visual information from slides to enhance the accuracy of technical terminologies. Realized that traditional metrics like WER fall short in assessing performance accurately, prompting the proposal of severity-aware WER (SWER) that considers the content type and severity of ASR errors. We propose the Scientific Vision Augmented ASR (SciVASR) framework as a baseline method, enabling MLLMs to improve transcript quality through post-editing. Evaluations of state-of-the-art MLLMs, including GPT-4o, show a 45% improvement over speech-only baselines, highlighting the importance of multimodal information integration.

Simultaneous Machine Translation with Large Language Models

Minghan Wang, Thuy-Trang Vu, Jinming Zhao, Fatemeh Shiri, Ehsan Shareghi, Gholamreza Haffari

Proceedings of the 22nd Annual Workshop of the Australasian Language Technology Association (ALTA 2024)

Abstract. Real-world simultaneous machine translation (SimulMT) systems face more challenges than just the quality-latency trade-off. They also need to address issues related to robustness with noisy input, processing long contexts, and flexibility for knowledge injection. These challenges demand models with strong language understanding and generation capabilities which may not often equipped by dedicated MT models. In this paper, we investigate the possibility of applying Large Language Models (LLM) to SimulMT tasks by using existing incremental-decoding methods with a newly proposed RALCP algorithm for latency reduction. We conducted experiments using the Llama2-7b-chat model on nine different languages from the MUST-C dataset. The results show that LLM outperforms dedicated MT models in terms of BLEU and LAAL metrics. Further analysis indicates that LLM has advantages in terms of tuning efficiency and robustness. However, it is important to note that the computational cost of LLM remains a significant obstacle to its application in SimulMT.

Zephyr: Zero-Shot Punctuation Restoration

Minghan Wang, Yinglu Li, Jiaxin Guo, Xiaosong Qiao, Chang Su, Min Zhang, Shimin Tao, Hao Yang

ICASSP 2023 - 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

Abstract. Punctuation restoration can be crucial for the cascade speech translation system. Traditional approaches typically treat it as a sequential tagging problem, predicting which punctuation mark should follow a given word. However, this often requires significant computational and storage resources for full-stage training or fine-tuning. Our argument is that pretrained language models (PLMs) can directly leverage their learned knowledge for punctuation generation, making additional training unnecessary. In this paper, we propose the Zephyr algorithm, which utilizes PLMs to perform zero-shot and few-shot punctuation restoration for both offline and streaming scenarios. Our experimental results demonstrate that, in comparison to fine-tuning-based baselines, Zephyr achieves competitive performance while requiring little to no training cost and exhibiting better generalizability in zero-shot and few-shot settings.

Code repository to be added; Code currently opens GitHub profile.

WhiSLU: End-to-End Spoken Language Understanding with Whisper

Minghan Wang, Yinglu Li, Jiaxin Guo, Xiaosong Qiao, Zongyao Li, Hengchao Shang, Daimeng Wei, Shimin Tao, Min Zhang, Hao Yang

Proceedings of the 24th Annual Conference of the International Speech Communication Association (Interspeech 2023)

Abstract. Spoken Language Understanding (SLU) systems commonly use cascading structures. However, these systems are prone to error propagation, information loss, high costs, and latency, leading researchers to explore end-to-end (E2E) SLU as a hot topic. However, E2E SLU faces the challenge of insufficient data, resulting in most previous work relying on pretrained acoustic models. Nevertheless, pre-training task and SLU task solution spaces are often substantially different, making it difficult for E2E SLU models to surpass cascading models. To address this, we propose using OpenAI's Whisper model for SLU tasks. We employ the Sequence-level Multitask Learning (SML) paradigm, which encodes multiple ASR-related tasks into a sequence for learning. Our method significantly outperforms the E2E baseline by a large margin (with a 10% improvement in EM score) and even outperforms cascading models, achieving a 77% EM score on the STOP dataset, demonstrating its effectiveness.

Code repository to be added; Code currently opens GitHub profile.

Diformer: Directional Transformer for Neural Machine Translation

Minghan Wang, Jiaxin Guo, Yuxia Wang, Daimeng Wei, Hengchao Shang, Yinglu Li, Chang Su, Yimeng Chen, Min Zhang, Shimin Tao, Hao Yang

Proceedings of the 23rd Annual Conference of the European Association for Machine Translation (EAMT 2022)

Abstract. Autoregressive (AR) and Non-autoregressive (NAR) models have their own superiority on the performance and latency, combining them into one model may take advantage of both. Current combination frameworks focus more on the integration of multiple decoding paradigms with a unified generative model, e.g. Masked Language Model. However, the generalization can be harmful on the performance due to the gap between training objective and inference. In this paper, we aim to close the gap by preserving the original objective of AR and NAR under a unified framework. Specifically, we propose the Directional Transformer (Diformer) by jointly modelling AR and NAR into three generation directions (left-to-right, right-to-left and straight) with a newly introduced direction variable, which works by controlling the prediction of each token to have specific dependencies under that direction. The unification achieved by direction successfully preserves the original dependency assumption used in AR and NAR, retaining both generalization and performance. Experiments on 4 WMT benchmarks demonstrate that Diformer outperforms current united-modelling works with more than 1.5 BLEU points for both AR and NAR decoding, and is also competitive to the state-of-the-art independent AR and NAR models.