Tampere University of Technology

TUTCRIS Research Portal

On Renyi's entropy estimation with one-dimensional Gaussian kernels

Research output: Chapter in Book/Report/Conference proceedingConference contributionScientificpeer-review


Original languageEnglish
Title of host publication2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)
Number of pages5
ISBN (Print)9781479999880
Publication statusPublished - 18 May 2016
Publication typeA4 Article in a conference publication
EventIEEE International Conference on Acoustics, Speech and Signal Processing -
Duration: 1 Jan 19001 Jan 2000

Publication series

ISSN (Electronic)2379-190X


ConferenceIEEE International Conference on Acoustics, Speech and Signal Processing


Rényi's entropies play a significant role in many signal processing applications. Plug-in kernel density estimation methods have been employed to estimate such entropies with good results. However, they become computationally intractable in higher dimensions, because of the requirement to store intermediate probability density values for a large number of data points. We propose a method to reduce the number of the samples in a plug-in kernel density estimation method for Rényi's entropies of real exponents and to improve the result of the standard plug-in kernel density method. To this end, we derive a univariate estimator, using an Hermite expansion of sums of Gaussian kernels and a hierarchical clustering of the samples. On simulated data from a univariate Gaussian distribution, our method performs better than a k-nearest neigbour algorithm and other kernel density estimation methods.


  • Gaussian kernels, Hermite expansion, hierarchical clustering, Rényi's entropy estimation

Publication forum classification

Field of science, Statistics Finland