TUTCRIS - Tampereen teknillinen yliopisto

TUTCRIS

A thorough evaluation of the Language Environment Analysis (LENA) system

Tutkimustuotosvertaisarvioitu

Standard

A thorough evaluation of the Language Environment Analysis (LENA) system. / Cristia, Alejandrina; Lavechin, Marvin; Scaff, Camila; Soderstrom, Melanie; Rowland, Caroline; Räsänen, Okko; Bunce, John; Bergelson, Elika.

julkaisussa: BEHAVIOR RESEARCH METHODS, 2020.

Tutkimustuotosvertaisarvioitu

Harvard

Cristia, A, Lavechin, M, Scaff, C, Soderstrom, M, Rowland, C, Räsänen, O, Bunce, J & Bergelson, E 2020, 'A thorough evaluation of the Language Environment Analysis (LENA) system', BEHAVIOR RESEARCH METHODS. https://doi.org/10.3758/s13428-020-01393-5

APA

Cristia, A., Lavechin, M., Scaff, C., Soderstrom, M., Rowland, C., Räsänen, O., ... Bergelson, E. (2020). A thorough evaluation of the Language Environment Analysis (LENA) system. BEHAVIOR RESEARCH METHODS. https://doi.org/10.3758/s13428-020-01393-5

Vancouver

Cristia A, Lavechin M, Scaff C, Soderstrom M, Rowland C, Räsänen O et al. A thorough evaluation of the Language Environment Analysis (LENA) system. BEHAVIOR RESEARCH METHODS. 2020. https://doi.org/10.3758/s13428-020-01393-5

Author

Cristia, Alejandrina ; Lavechin, Marvin ; Scaff, Camila ; Soderstrom, Melanie ; Rowland, Caroline ; Räsänen, Okko ; Bunce, John ; Bergelson, Elika. / A thorough evaluation of the Language Environment Analysis (LENA) system. Julkaisussa: BEHAVIOR RESEARCH METHODS. 2020.

Bibtex - Lataa

@article{fdd361e825854c1b9040136ebdc3363f,
title = "A thorough evaluation of the Language Environment Analysis (LENA) system",
abstract = "In the previous decade, dozens of studies involving thousands of children across several research disciplines have made use of a combined daylong audio-recorder and automated algorithmic analysis called the LENAⓇ system, which aims to assess children’s language environment. While the system’s prevalence in the language acquisition domain is steadily growing, there are only scattered validation efforts on only some of its key characteristics. Here, we assess the LENAⓇ system’s accuracy across all of its key measures: speaker classification, Child Vocalization Counts (CVC), Conversational Turn Counts (CTC), and Adult Word Counts (AWC). Our assessment is based on manual annotation of clips that have been randomly or periodically sampled out of daylong recordings, collected from (a) populations similar to the system’s original training data (North American English-learning children aged 3-36 months), (b) children learning another dialect of English (UK), and (c) slightly older children growing up in a different linguistic and socio-cultural setting (Tsimane’ learners in rural Bolivia). We find reasonably high accuracy in some measures (AWC, CVC), with more problematic levels of performance in others (CTC, precision of male adults and other children). Statistical analyses do not support the view that performance is worse for children who are dissimilar from the LENAⓇ original training set. Whether LENAⓇ results are accurate enough for a given research, educational, or clinical application depends largely on the specifics at hand. We therefore conclude with a set of recommendations to help researchers make this determination for their goals.",
keywords = "Adult Word Count, Agreement, Child Vocalization Count, Conversational Turn Count, English, Human transcription, LENA, Measurement error, Method comparison, Reliability, Speech technology, Tsimane’",
author = "Alejandrina Cristia and Marvin Lavechin and Camila Scaff and Melanie Soderstrom and Caroline Rowland and Okko R{\"a}s{\"a}nen and John Bunce and Elika Bergelson",
year = "2020",
doi = "10.3758/s13428-020-01393-5",
language = "English",
journal = "BEHAVIOR RESEARCH METHODS",
issn = "1554-351X",
publisher = "Springer Verlag",

}

RIS (suitable for import to EndNote) - Lataa

TY - JOUR

T1 - A thorough evaluation of the Language Environment Analysis (LENA) system

AU - Cristia, Alejandrina

AU - Lavechin, Marvin

AU - Scaff, Camila

AU - Soderstrom, Melanie

AU - Rowland, Caroline

AU - Räsänen, Okko

AU - Bunce, John

AU - Bergelson, Elika

PY - 2020

Y1 - 2020

N2 - In the previous decade, dozens of studies involving thousands of children across several research disciplines have made use of a combined daylong audio-recorder and automated algorithmic analysis called the LENAⓇ system, which aims to assess children’s language environment. While the system’s prevalence in the language acquisition domain is steadily growing, there are only scattered validation efforts on only some of its key characteristics. Here, we assess the LENAⓇ system’s accuracy across all of its key measures: speaker classification, Child Vocalization Counts (CVC), Conversational Turn Counts (CTC), and Adult Word Counts (AWC). Our assessment is based on manual annotation of clips that have been randomly or periodically sampled out of daylong recordings, collected from (a) populations similar to the system’s original training data (North American English-learning children aged 3-36 months), (b) children learning another dialect of English (UK), and (c) slightly older children growing up in a different linguistic and socio-cultural setting (Tsimane’ learners in rural Bolivia). We find reasonably high accuracy in some measures (AWC, CVC), with more problematic levels of performance in others (CTC, precision of male adults and other children). Statistical analyses do not support the view that performance is worse for children who are dissimilar from the LENAⓇ original training set. Whether LENAⓇ results are accurate enough for a given research, educational, or clinical application depends largely on the specifics at hand. We therefore conclude with a set of recommendations to help researchers make this determination for their goals.

AB - In the previous decade, dozens of studies involving thousands of children across several research disciplines have made use of a combined daylong audio-recorder and automated algorithmic analysis called the LENAⓇ system, which aims to assess children’s language environment. While the system’s prevalence in the language acquisition domain is steadily growing, there are only scattered validation efforts on only some of its key characteristics. Here, we assess the LENAⓇ system’s accuracy across all of its key measures: speaker classification, Child Vocalization Counts (CVC), Conversational Turn Counts (CTC), and Adult Word Counts (AWC). Our assessment is based on manual annotation of clips that have been randomly or periodically sampled out of daylong recordings, collected from (a) populations similar to the system’s original training data (North American English-learning children aged 3-36 months), (b) children learning another dialect of English (UK), and (c) slightly older children growing up in a different linguistic and socio-cultural setting (Tsimane’ learners in rural Bolivia). We find reasonably high accuracy in some measures (AWC, CVC), with more problematic levels of performance in others (CTC, precision of male adults and other children). Statistical analyses do not support the view that performance is worse for children who are dissimilar from the LENAⓇ original training set. Whether LENAⓇ results are accurate enough for a given research, educational, or clinical application depends largely on the specifics at hand. We therefore conclude with a set of recommendations to help researchers make this determination for their goals.

KW - Adult Word Count

KW - Agreement

KW - Child Vocalization Count

KW - Conversational Turn Count

KW - English

KW - Human transcription

KW - LENA

KW - Measurement error

KW - Method comparison

KW - Reliability

KW - Speech technology

KW - Tsimane’

U2 - 10.3758/s13428-020-01393-5

DO - 10.3758/s13428-020-01393-5

M3 - Article

JO - BEHAVIOR RESEARCH METHODS

JF - BEHAVIOR RESEARCH METHODS

SN - 1554-351X

ER -