The accuracy and validity of self-reported social media use measures among adolescents
Keywords
Accuracy
Ambulatory assessment
Log data
Self-report
Social network site use
Validity
Publication details
| Year: | 2021 |
| DOI: | 10.1016/j.chbr.2021.100090 |
| Issued: | 2021 |
| Language: | English |
| Volume: | 3 |
| Start Page: | 1 |
| End Page: | 11 |
| Editors: | |
| Authors: | Verbeij T.; Pouwels J.; Beyens I.; Valkenburg P. |
| Type: | Journal article |
| Journal: | Computers in Human Behavior Reports |
| Publisher: | Elsevier BV |
| Topics: | Researching children online: methodology and ethics |
| Sample: | "we included 388 participants in the project. Of these 388 participants, 300 Instagram, WhatsApp, or Snapchat users also participated in the second three-week ESM study that was part of this larger project. The second ESM wave started on 3 June 2020, which coincidentally happened to be the day that the mandated school closures due to COVID-19 in the Netherlands ended after 2.5 months. As tracking software could only track Android phones, the potential sample of 300 adolescents was reduced to 171 Android users, of whom 131 (44%) provided active consent to track their app usage and had their app usage continuously tracked throughout the 21-day ESM period. Of these 131 participants, 125 (42%) also participated in the second ESM wave. The final sample of this study therefore consisted of 125 middle adolescents (Mage ¼ 14.1 years, SDage ¼ .72, 48% girls) of whom 98.5% identified themselves as Dutch. The educational levels of our sample were representative of the south of Netherlands: 38.4% were enrolled in the prevocational secondary education track, 32.8% in the intermediate general secondary education track, and 28.8% in the academic preparatory education track." (Verbeij et al., 2021, p. 3) |
Abstract
A growing number of studies have tried to assess the effects of social media on adolescents, who are among the most avid social media users. To establish the effects of social media use, we need accurate and valid instruments to measure adolescents’ time spent with these media. The aim of this preregistered study was to examine the accuracy and convergent validity of retrospective surveys and experience sampling method (ESM) surveys, by
comparing adolescents’ responses to these self-report measures with their digital trace data. The sample consisted of 125 adolescents (48% girls; Mage ¼ 14.1) with Android smartphones. In both retrospective surveys and ESM, adolescents overestimated their time spent on social media. They more accurately estimated their time spent on platforms that are used in a less fragmented way (Instagram) than on platforms that are used in a more fragmented way (Snapchat). The between-person convergent validity of adolescents’ time estimates according to retrospective surveys and ESM reached the threshold for minimum acceptable convergent validity (r ranged from .55 to .65). The within-person convergent validity of adolescents’ ESM estimates of their time spent on social media was unacceptable (r ¼ .32). The between- and within-person convergent validity of ESM estimates decreased over time (i.e., fatigue effect).
Outcome
"In a sample of 125 adolescents with Android smartphones, we investigated the accuracy and convergent validity of their estimates of the time spent on social media according to
retrospective surveys and ESM. We found that (1) adolescents overestimated their time spent on social media on all self-report measures; (2) their retrospective survey (previous week) and ESM estimates were more accurate for Instagram than for Snapchat and WhatsApp; (3) the between-person convergent validity of the various self-report measures reached the threshold for minimum acceptable convergent validity; (4) the within-person convergent validity for ESM was unacceptable; and (5) the convergent validity of the ESM estimates decreased over time (fatigue effect)." (Verbeij et al., 2021, p. 7)