| Title |
Beyond classical positional encodings: a learnable QFT-inspired framework for transformer language models |
| Authors |
Tehsin, Sara ; Akram, Tallha ; Naqvi, Syed Rameez ; Alharbi, Meshal ; Alabduljabbar, Abdulrahman |
| DOI |
10.32604/cmc.2026.081304 |
| Full Text |
|
| Is Part of |
Computers, materials and continua.. Henderson, NV : Tech Science press. 2026, vol. 88, iss. 3, art. no. 6, p. 1-24.. ISSN 1546-2218. eISSN 1546-2226 |
| Keywords [eng] |
hybrid quantum-classical models ; LLMs ; near-term quantum devices ; positional embeddings ; positional encoding ; Quantum Fourier Transform ; quantum transformer |
| Abstract [eng] |
Transformers have become the dominant architecture for sequence modeling in natural language processing; however, their effectiveness critically depends on how positional information is encoded. Conventional positional encodings, while effective, may have limited structural flexibility for capturing complex global sequence relationships. Recent quantum-inspired approaches have sought to address this limitation, yet many either oversimplify quantum principles or introduce substantial computational or hardware overhead. We introduce a novel Quantum Fourier Transform (QFT)-inspired positional encoding scheme for transformers, motivated by the structured frequency representation of the QFT. Unlike prior approaches that either emulate quantum operations superficially or require complex circuit constructions, the proposed method provides a learnable hybrid encoding that preserves quantum-inspired structure while remaining aligned with hardware-efficient circuit primitives and structurally compatible with future near-term quantum implementations. Experiments on WikiText-103 indicate that the proposed encoding achieves competitive perplexity, improved robustness to input scrambling, and stable training behavior relative to alternative quantum-inspired baselines under the evaluated settings. Preliminary circuit-level simulations further suggest favorable noise resilience of the associated encoding primitives. These findings support the potential utility of incorporating quantum-inspired design principles into deep learning architectures and provide a foundation for future exploration at the interface of quantum computing and transformer-based natural language processing (NLP). |
| Published |
Henderson, NV : Tech Science press |
| Type |
Journal article |
| Language |
English |
| Publication date |
2026 |
| CC license |
|
