Tampere University of Technology

TUTCRIS Research Portal

Entropy analysis of word-length series of natural language texts: Effects of text language and genre

Research output: Contribution to journalArticleScientificpeer-review

Details

Original languageEnglish
Article number1250223
JournalINTERNATIONAL JOURNAL OF BIFURCATION AND CHAOS
Volume22
Issue number9
DOIs
Publication statusPublished - Sep 2012
Publication typeA1 Journal article-refereed

Abstract

We estimate the n-gram entropies of natural language texts in word-length representation and find that these are sensitive to text language and genre. We attribute this sensitivity to changes in the probability distribution of the lengths of single words and emphasize the crucial role of the uniformity of probabilities of having words with length between five and ten. Furthermore, comparison with the entropies of shuffled data reveals the impact of word length correlations on the estimated n-gram entropies.

Keywords

  • N-gram entropies, Quantitative linguistics, Symbolic dynamics, Time series, Word-length representation