TUTCRIS - Tampereen teknillinen yliopisto

TUTCRIS

Entropy analysis of word-length series of natural language texts: Effects of text language and genre

Tutkimustuotosvertaisarvioitu

Yksityiskohdat

AlkuperäiskieliEnglanti
Artikkeli1250223
JulkaisuINTERNATIONAL JOURNAL OF BIFURCATION AND CHAOS
Vuosikerta22
Numero9
DOI - pysyväislinkit
TilaJulkaistu - syyskuuta 2012
OKM-julkaisutyyppiA1 Alkuperäisartikkeli

Tiivistelmä

We estimate the n-gram entropies of natural language texts in word-length representation and find that these are sensitive to text language and genre. We attribute this sensitivity to changes in the probability distribution of the lengths of single words and emphasize the crucial role of the uniformity of probabilities of having words with length between five and ten. Furthermore, comparison with the entropies of shuffled data reveals the impact of word length correlations on the estimated n-gram entropies.