Assessment of Criteria for Specific Internet-use Disorders (ACSID-11): Introduction of a new screening instrument capturing ICD-11 criteria for gaming disorder and other potential Internet-use disorders (2022)

Introduction

The distribution of and easy access to the Internet make online services particularly attractive and offer many advantages. Besides the benefits for most people, online behaviors may take an uncontrolled addictive form in some individuals (e.g., King & Potenza, 2019; Young, 2004). Especially gaming becomes more and more a public health issue (Faust & Prochaska, 2018; Rumpf et al., 2018). After the recognition of ‘Internet gaming disorder’ in the fifth revision of the Diagnostic and Statistical Manual of Mental Disorders (DSM-5; American Psychiatric Association, 2013) as a condition of further study, gaming disorder has now been included as an official diagnosis (6C51) in the 11th revision of the International Classification of Diseases (ICD-11; World Health Organization, 2018). This is an important step in addressing the global challenges posed by the harmful use of digital technologies (Billieux, Stein, Castro-Calvo, Higushi, & King, 2021). The worldwide prevalence of gaming disorder is estimated 3.05%, which is comparable to other mental disorders such as substance-use disorders or obsessive-compulsive disorders (Stevens, Dorstyn, Delfabbro, & King, 2021). However, the prevalence estimates vary greatly depending on the screening instrument used (Stevens et al., 2021). At present, the landscape of instruments is manifold. Most measures are based on DSM-5 criteria for Internet gaming disorder and none seems clearly preferable (King et al., 2020). Similar applies to other potential addictive behaviors on the Internet, such as the problematic use of online pornography, social networks, or online shopping. These problematic online behaviors may occur together with gaming disorder (Burleigh, Griffiths, Sumich, Stavropoulos, & Kuss, 2019; Müller et al., 2021), but may also be an own entity. Recent theoretical frameworks such as the Interaction of Person-Affect-Cognition-Execution (I-PACE) model (Brand, Young, Laier, Wölfling, & Potenza, 2016; Brand et al., 2019) assume that similar psychological processes underlie the different types of (online) addictive behaviors. The assumptions are in line with earlier approaches that can be used to explain commonalities between addictive disorders, e.g., regarding neuropsychological mechanisms (Bechara, 2005; Robinson & Berridge, 1993), genetic aspects (Blum et al., 2000), or common components (Griffiths, 2005). However, a comprehensive screening tool for (potential) specific Internet-use disorders based on the same criteria does currently not exist. Uniform screenings across different types of disorders due to addictive behaviors are important to determine commonalities and differences more validly.

In the ICD-11, gaming disorder is listed beyond gambling disorder in the category ‘disorders due to addictive behaviors’. The proposed diagnostic criteria (for both) are: (1) impaired control over the behavior (e.g., onset, frequency, intensity, duration, termination, context); (2) increasing priority given to the behavior to the extent that the behavior takes precedence over other interests and everyday activities; (3) continuation or escalation of the behavior despite negative consequences. Although not directly mentioned as additional criteria, it is mandatory for the diagnosis that the behavioral pattern leads to (4) functional impairment in important areas of daily life (e.g., personal, family, educational, or social issues) and/or marked distress (World Health Organization, 2018). Therefore, both components should be included when studying potential addictive behaviors. Overall, these criteria may also be applied to the category ‘other specified disorders due to addictive behaviors’ (6C5Y), in which buying-shopping disorder, pornography-use disorder, and social-networks-use disorder may potentially be categorized (Brand et al., 2020). Online buying-shopping disorder can be defined by excessive, maladaptive online buying of consumer goods that occurs recurrently despite negative consequences and thus may constitute a specific Internet-use disorder (Müller, Laskowski, et al., 2021). Pornography-use disorder is characterized by diminished control over the consumption of (online) pornographic content, which is separable from other compulsive sexual behaviors (Kraus, Martino, & Potenza, 2016; Kraus et al., 2018). Social-networks-use disorder can be defined by excessive use of social networks (including social networking sites and other online communication applications) characterized by diminished control over the use, increasing priority given to the use, and continuation of the use of social networks despite experiencing negative consequences (Andreassen, 2015). All three potential behavioral addictions constitute clinically relevant phenomena that show similarities with other addictive behaviors (e.g., Brand et al., 2020; Griffiths, Kuss, & Demetrovics, 2014; Müller et al., 2019; Stark, Klucken, Potenza, Brand, & Strahler, 2018).

Instruments assessing specific types of Internet-use disorders are mainly based either on earlier concepts, such as modified versions of Young’s Internet Addiction Test (e.g., Laier, Pawlikowski, Pekal, Schulte, & Brand, 2013; Wegmann, Stodt, & Brand, 2015) or the “Bergen” scales based on Griffiths’ addiction components (e.g., Andreassen, Torsheim, Brunborg, & Pallesen, 2012; Andreassen et al., 2015), or they measure unidimensional constructs based on DSM-5 criteria for gaming disorder (e.g., Lemmens, Valkenburg, & Gentile, 2015; Van den Eijnden, Lemmens, & Valkenburg, 2016) or gambling disorder (for a review see Otto et al., 2020). Some earlier measures have been adopted from measures for gambling disorder, substance-use disorders or have been developed atheoretically (Laconi, Rodgers, & Chabrol, 2014). Many of these instruments show psychometric weaknesses and inconsistencies as highlighted in different reviews (King, Haagsma, Delfabbro, Gradisar, & Griffiths, 2013; Lortie & Guitton, 2013; Petry, Rehbein, Ko, & O’Brien, 2015). King et al. (2020) identified 32 different instruments assessing gaming disorder, which illustrates the inconsistency in the research field. Even the most cited and widely used instruments, such as Young’s Internet Addiction Test (Young, 1998), do not adequately represent the diagnostic criteria for gaming disorder, neither of the DSM-5 nor of the ICD-11. King et al. (2020) further point at psychometric weaknesses, for example, a lack of empirical validation and that most instruments were designed based on the assumption of a unimodal construct. It indicates that the sum of the individual symptoms is counted instead of looking at the frequency and experienced intensity individually. The Ten-Item Internet Gaming Disorder Test (IGDT-10; Király et al., 2017) currently seems to adequately capture the DSM-5 criteria but overall none of the instruments appeared to be clearly preferable (King et al., 2020). Recently, a number of scales were introduced as first screening instruments capturing the ICD-11 criteria for gaming disorder (Balhara et al., 2020; Higuchi et al., 2021; Jo et al., 2020; Paschke, Austermann, & Thomasius, 2020; Pontes et al., 2021) as well as for social-networks-use disorder (Paschke, Austermann, & Thomasius, 2021). In general, it could be assumed that not each symptom is necessarily experienced equally, for example, equally often or equally intensively. It thus seems desirable that screening instruments be able to capture both, the overall symptom experiences, and the totality of symptoms per se. Rather, a multidimensional approach may investigate which symptom contributes decisively, or in different phases, to the development and maintenance of a problematic behavior, is associated with higher level of suffering, or whether it is just a matter of even significance.

Similar problems and inconsistencies become apparent when looking at instruments assessing other types of potential specific Internet-use disorders, namely online buying-shopping disorder, online pornography-use disorder, and social-networks-use disorder. These potential specific Internet-use disorders are not formally classified in ICD-11 in contrast to gaming and gambling disorders. Especially in the case of gambling disorder, numerous screening instruments already exist, but most of them lack adequate evidence (Otto et al., 2020), and neither address the ICD-11 criteria for gambling disorder nor focus on predominantly online gambling disorder (Albrecht, Kirschner, & Grüsser, 2007; Dowling et al., 2019). The ICD-11 lists Compulsive Sexual Behavior Disorder (CSBD), for which many assume that problematic pornography use is a main behavioral symptom, as an impulse-control disorder. Compulsive buying-shopping disorder is listed as an example under the category ‘other specified impulse control disorders’ (6C7Y) but without differentiating between online and offline variants. This differentiation is also not made in the most widely used questionnaires measuring compulsive buying (Maraz et al., 2015; Müller, Mitchell, Vogel, & de Zwaan, 2017). Social-networks-use disorder has not yet been considered in the ICD-11. However, there are evidence-based arguments for each of the three disorders to be rather classified as addictive behaviors (Brand et al., 2020; Gola et al., 2017; Müller et al., 2019; Stark et al., 2018; Wegmann, Müller, Ostendorf, & Brand, 2018). Besides a lack of consensus regarding classification and definitions of these potential specific Internet-use disorders, there are also inconsistencies in the use of screening instruments (for reviews see Andreassen, 2015; Fernandez & Griffiths, 2021; Hussain & Griffiths, 2018; Müller et al., 2017). For example, there are more than 20 instruments supposed to measure problematic pornography use (Fernandez & Griffiths, 2021) but none adequately covers the ICD-11 criteria for disorders due to addictive behaviors, which are very close to the ICD-11 criteria for CSBD.

Furthermore, some specific Internet-use disorders seem likely to co-occur, especially disordered gaming and social-networks use (Burleigh et al., 2019; Müller et al., 2021). Using latent profile analysis, Charzyńska, Sussman, and Atroszko (2021) identified that disordered social-networking and shopping as well as disordered gaming and pornography use often occurred together respectively. The profile including high levels on all Internet-use disorders showed lowest well-being (Charzyńska et al., 2021). This also emphasizes the importance of a comprehensive and uniform screening across different Internet-use behaviors. There have been attempts to use similar sets of items across different Internet-use disorders, such as the Problematic Pornography Consumption Scale (Bőthe et al., 2018), the Bergen Social Media Addiction Scale (Andreassen, Pallesen, & Griffiths, 2017) or the Online Shopping Addiction Scale (Zhao, Tian, & Xin, 2017). However, these scales were designed on the basis of the components model by Griffiths (2005) and do not cover the current proposed criteria for disorders due to addictive behaviors (cf. World Health Organization, 2018).

In summary, the ICD-11 proposed diagnostic criteria for disorders due to (predominantly online) addictive behaviors, namely gambling disorder and gaming disorder. Problematic online pornography use, online buying-shopping, and social-networks use may be assigned to the ICD-11 subcategory ‘other specified disorders due to addictive behaviors’ for which the same criteria can be applied (Brand et al., 2020). To date, the landscape of screening instruments for these (potential) specific Internet-use disorders is highly inconsistent. However, consistent measurement of the different constructs is essential to advance research on commonalities and differences across different types of disorders due to addictive behaviors. Our aim was to develop a short but comprehensive screening instrument for different types of (potential) specific Internet-use disorders covering the ICD-11 criteria for gaming disorder and gambling disorder, to assist with early identification of (potential) specific problematic online behaviors.

Methods

Measures

Assessment of Criteria for Specific Internet-use Disorders: ACSID-11

With the ACSID-11 we aimed to invent a tool for assessing specific Internet-use disorders in a short but comprehensive, and consistent manner. It was developed based on theory by an expert group of addiction researchers and clinicians. The items were derived in multiple discussions and consensus meetings based on ICD-11 criteria for disorders due to addictive behaviors, as they are described for gaming and gambling, assuming a multifactorial structure. Findings of a Talk-Aloud Analysis were used to optimize content validity and comprehensibility of the items (Schmidt et al., submitted).

The ACSID-11 comprises 11 items that capture the ICD-11 criteria for disorders due to addictive behaviors. The three main criteria, impaired control (IC), increased priority given to the online activity (IP), and continuation/escalation (CE) of Internet use despite negative consequences, are represented by three items each. Two additional items were created to assess functional impairment in daily life (FI) and marked distress (MD) due to the online activity. In a pre-query, participants were instructed to indicate which activities on the Internet they have used at least occasionally in the past 12 months. The activities (i.e., ‘gaming’, ‘online shopping’, ‘use of online pornography’, ‘use of social-networks’, ‘online gambling’, and ‘other’) were listed with corresponding definitions and the response options ‘yes’ or ‘no’. Participants who answered ‘yes’ only to the ‘other’ item were screened out. All others received the ACSID-11 items for all those activities that were answered with ‘yes’. This multibehavioral approach is based on the WHO’s Alcohol, Smoking and Substance Involvement Screening Test (ASSIST; WHO ASSIST Working Group, 2002), which screens for the major categories of substance use and its negative consequences as well as signs of addictive behavior in a consistent way across specific substances.

In analogy to the ASSIST, every item is formulated in a way so that it could be answered directly for the respective activity. We used a two-part response format (see Fig. 1), in which participants should indicate per item for each activity how often they had the experience in the last 12 months (0: ‚never‘, 1: ‚rarely‘, 2: ‚sometimes‘, 3: ‚often‘), and if at least “rarely”, how intense each experience was in the last 12 months (0: ‚not at all intense‘, 1: ‚rather not intense‘, 2: ‚rather intense‘, 3: ‚intense‘). By assessing the frequency as well as the intensity of each symptom, it is possible to investigate the occurrence of a symptom, but also to control for how intense symptoms are perceived beyond the frequency. The items of the ACSID-11 (proposed English translation) are shown in Table 1. The original (German) items including pre-query and instructions can be found in the Appendix (see Appendix A).

VIEW FULL SIZE

 

Exemplary item of the ACSID-11 (proposed English translation of the German original item) illustrating the measurement of frequency (left columns) and intensity (right columns) of situations related to specific online activities. Notes. The figure shows an exemplary item of the factor Impaired Control (IC) as displayed A) to an individual who uses all five online activities as indicated in the pre-query (see Appendix A) and B) to an individual who indicated to use online shopping and social networks only.

Citation: Journal of Behavioral Addictions 2022; 10.1556/2006.2022.00013

• Download Figure
• Download figure as PowerPoint slide

Table 1.

Items of the ACSID-11 screener for specific Internet-use disorders (proposed English translation).

Item	Question
IC1	In the past 12 months, have you had trouble keeping track of when you started the activity, for how long, how intensely, or in what situation you did it, or when you stopped?
IC2	In the past 12 months, have you felt the desire to stop or restrict the activity because you noticed you were using it too much?
IC3	In the past 12 months, have you tried to stop or restrict the activity and failed with it?
IP1	In the past 12 months, have you given the activity an increasingly higher priority than other activities or interests in your daily life?
IP2	In the past 12 months, have you lost interest in other activities you used to enjoy because of the activity?
IP3	In the past 12 months, have you neglected or given up other activities or interests that you used to enjoy because of the activity?
CE1	In the past 12 months, have you continued or increased the activity even though it has threatened or caused you to lose a relationship with someone important to you?
CE2	In the past 12 months, have you continued or increased the activity even though it has caused you problems in school/training/work?
CE3	In the past 12 months, have you continued or increased the activity even though it has caused you physical or mental complaints/diseases?
FI1	Thinking about all areas of your life, has your life been noticeably affected by the activity in the past 12 months?
MD1	Thinking about all areas of your life, did the activity cause you suffering in the past 12 months?

Notes. IC = impaired control; IP = increased priority; CE = continuation/escalation; FI = functional impairment; MD = marked distress; The original German items can be found in Appendix A.

Ten-Item Internet Gaming Disorder Test: IGDT-10 – ASSIST version

As a measure of convergent validity, we used the ten-item IGDT-10 (Király et al., 2017) in an extended version. The IGDT-10 operationalizes the nine DSM-5 criteria for Internet gaming disorder (American Psychiatric Association, 2013). In this study, we extended the original gaming specific version so that all forms of specific Internet-use disorders were assessed. To implement this, and to keep the methodology comparable, we also used the multibehavioral response format on the example of ASSIST here. For this, the items were modified so that ’gaming’ was replaced by ’the activity’. Every item was then answered for all online activities that the participants had previously indicated to use (from a selection of ‘gaming’, ‘online shopping’, ‘use of online pornography’, ‘use of social networks’, and ‘online gambling’). Per item, each activity was rated on a three-point Likert scale (0 = ‘never’, 1 = ‘sometimes’, 2 = ‘often’). The scoring was the same as the original version of the IGDT-10: Each criterion received a score of 0 if the response was ‘never’ or ‘sometimes’ and a score of 1 if the response was ‘often’. Items 9 and 10 represent the same criterion (i.e., ‘jeopardy or losing a significant relationship, job, or educational or career opportunity because of participation in Internet games’) and count together one point if one or both items are met. A final sum score was calculated for each activity. It could range from 0 to 9 with higher scores indicating higher symptom severity. Regarding gaming disorder, a score of five or more indicates clinical relevance (Király et al., 2017).

Patient Health Questionnaire-4: PHQ-4

The Patient Health Questionnaire-4 (PHQ-4; Kroenke, Spitzer, Williams, & Löwe, 2009) is a brief measure of symptoms of depression and anxiety. It consists of four items taken from the Generalized Anxiety Disorder–7 scale and the PHQ-8 module for depression. Participants should indicate the frequency of occurrence of certain symptoms on a four-point Likert scale ranging from 0 (‘not at all’) to 3 (‘nearly every day’). The total score can range between 0 and 12 indicating none/minimal, mild, moderate, and severe levels of psychological distress with scores from 0–2, 3–5, 6–8, 9–12, respectively (Kroenke et al., 2009).

General well-being

General life satisfaction was assessed using the Life Satisfaction Short Scale (L-1) in the German original version (Beierlein, Kovaleva, László, Kemper, & Rammstedt, 2015) answered on an 11-point Likert scale ranging from 0 (‘not at all satisfied’) to 10 (‘completely satisfied’). The single item scale is well-validated and correlates strongly with multiple-item-scales assessing satisfaction with life (Beierlein et al., 2015). We additionally asked for specific life satisfaction in the domain of health (H-1): ‘All things considered, how satisfied are you with your health these days?’ answered on the same 11-point scale (cf. Beierlein et al., 2015).

Results

Within the current sample, the specific Internet-use behaviors were distributed as follows: Gaming was indicated by 440 (45.9%) individuals (age: M = 43.59, SD = 14.66; 259 male, 180 female, 1 divers), 944 (98.5%) of the individuals engaged in online shopping (age: M = 47.58, SD = 14.49; 491 male, 452 female, 1 divers), 340 (35.5%) of the individuals used online-pornography (age: M = 44.80, SD = 14.96; 263 male, 76 female, 1 divers), 854 (89.1%) of the individuals used social networks (age: M = 46.52, SD = 14.66; 425 male, 428 female, 1 divers), and 200 (20.9%) individuals engaged in online gambling (age: M = 46.91, SD = 13.67; 125 male, 75 female, 0 divers). The minority of participants (n = 61; 6.3%) indicated to use only one activity. Most participants (n = 841; 87.8%) used at least online shopping together with social-networks and 409 (42.7%) of them also indicated to play online games. Sixty-eight (7.1%) of the participants indicated to use all of the mentioned online activities.

Given that gaming and gambling disorders are the two types of disorders due to addictive behaviors that are officially recognized and given that the number of individuals in our sample who reported to do online gambling was rather limited, we will first concentrate on the results regarding the assessment of criteria for gaming disorder with the ACSID-11.

Confirmatory factor analysis

Assumed four-factor model

We tested the assumed four-factorial structure of ACSID-11 by means of multiple CFAs, one per specific Internet-use disorder and separately for frequency and intensity ratings. The factors (1) Impaired Control, (2) Increased Priority, and (3) Continuation/Escalation were formed by the respective three items. The two additional items measuring functional impairment in daily life and marked distress due to the online activity formed the additional factor (4) Functional Impairment. The four-factorial structure of the ACSID-11 is supported by the data. The fit indices indicate a good fit between the models and the data for all types of specific Internet-use disorders assessed by ACSID-11, namely gaming disorder, online buying-shopping disorder, and social-networks-use disorder, online pornography-use disorder, and online gambling disorder (see Table 3). Regarding online pornography-use disorder and online gambling disorder, TLI and RMSEA could be biased due to small sample sizes (Hu & Bentler, 1999). The factor loadings and residual covariances for the CFAs applying a four-factor model are shown in Fig. 2. To note, some of the models show singular anomalous values (i.e., negative residual variance for a latent variable or correlations of equal to or greater than 1).

Table 3.

Fit indices of the four-factor, unidimensional, and second-order CFA models for specific (potential) Internet-use disorders measured by ACSID-11.

		Gaming disorder
		Frequency					Intensity
Model	df	CFI	TLI	SRMR	RMSEA	χ2/df	CFI	TLI	SRMR	RMSEA	χ2/df
Four-factor model	38	0.991	0.987	0.031	0.051	2.13	0.993	0.990	0.029	0.043	1.81
Unidimensional model	27	0.969	0.961	0.048	0.087	4.32	0.970	0.963	0.047	0.082	3.99
Second-order factor model	40	0.992	0.988	0.031	0.047	1.99	0.992	0.989	0.032	0.045	1.89

		Online buying-shopping disorder
		Frequency					Intensity
Model	df	CFI	TLI	SRMR	RMSEA	χ2/df	CFI	TLI	SRMR	RMSEA	χ2/df
Four-factor model	38	0.996	0.994	0.019	0.034	2.07	0.995	0.992	0.020	0.037	2.30
Unidimensional model	27	0.981	0.976	0.037	0.070	5.58	0.986	0.982	0.031	0.056	3.98
Second-order factor model	40	0.996	0.994	0.021	0.036	2.19	0.994	0.992	0.023	0.038	2.40

		Online pornography-use disorder
		Frequency					Intensity
Model	df	CFI	TLI	SRMR	RMSEA	χ2/df	CFI	TLI	SRMR	RMSEA	χ2/df
Four-factor model	38	0.993	0.989	0.034	0.054	1.99	0.987	0.981	0.038	0.065	2.43
Unidimensional model	27	0.984	0.979	0.044	0.075	2.91	0.976	0.970	0.046	0.082	3.27
Second-order factor model	40	0.993	0.991	0.033	0.049	1.83	0.984	0.979	0.039	0.068	2.59

		Social-networks-use disorder
		Frequency					Intensity
Model	df	CFI	TLI	SRMR	RMSEA	χ2/df	CFI	TLI	SRMR	RMSEA	χ2/df
Four-factor model	38	0.993	0.990	0.023	0.049	3.03	0.993	0.989	0.023	0.052	3.31
Unidimensional model	27	0.970	0.963	0.048	0.096	8.89	0.977	0.972	0.039	0.085	7.13
Second-order factor model	40	0.992	0.989	0.027	0.053	3.39	0.991	0.988	0.025	0.056	3.64

		Online gambling disorder
		Frequency					Intensity
Model	df	CFI	TLI	SRMR	RMSEA	χ2/df	CFI	TLI	SRMR	RMSEA	χ2/df
Four-factor model	38	0.997	0.996	0.027	0.059	1.70	0.997	0.996	0.026	0.049	1.47
Unidimensional model	27	0.994	0.992	0.040	0.078	2.20	0.991	0.989	0.039	0.080	2.28
Second-order factor model	40	0.997	0.996	0.029	0.054	1.58	0.997	0.995	0.029	0.053	1.55

Notes. Sample sizes varied for gaming (n = 440), online shopping (n = 944), online-pornography use (n = 340), social-networks use (n = 854), and online gambling (n = 200); ACSID-11 = Assessment of Criteria for Specific Internet-use Disorders, 11-items.

VIEW FULL SIZE

 

Factor loadings and residual covariances of the four-factor models of the ACSID-11 (frequency) for (A) gaming disorder, (B) online gambling disorder, (C) online buying-shopping disorder, (D) online pornography-use disorder, and (E) social-networks-use disorder. Notes. Sample sizes varied for gaming (n = 440), online shopping (n = 944), online-pornography use (n = 340), social-networks use (n = 854), and online gambling (n = 200); The intensity scale of the ACSID-11 showed similar results. ACSID-11 = Assessment of Criteria for Specific Internet-use Disorders, 11-items; Values represent standardized factor loadings, factor covariances, and residual covariances. All estimates were significant at p < 0.001.

Citation: Journal of Behavioral Addictions 2022; 10.1556/2006.2022.00013

• Download Figure
• Download figure as PowerPoint slide

Unidimensional model

Due to high intercorrelations between the different factors, we additionally tested unidimensional solutions with all items loading on one factor, as implemented, e.g., in the IGDT-10. The unidimensional models of the ACSID-11 showed acceptable fit, but with RMSEA and/or χ²/df being above the suggested cutoffs. For all behaviors, the model fits for the four-factor models are better compared to the respective unidimensional models (see Table 3). Consequently, the four-factor solution appears to be superior to the unidimensional solution.

Second-order factor model and bifactor model

An alternative to account for the high intercorrelations is to include a general factor representing the general construct, which is comprised of related subdomains. This can be implemented via second-order factor model and bifactor model. In the second-order factor model, a general (second-order) factor is modeled in an attempt to explain the correlations among the first-order factors. In the bifactor model, it is assumed that the general factor accounts for the commonality between the related domains and that, additionally, there are multiple specific factors, each of which has unique effects on and beyond the general factor. This is modeled so that each item is allowed to load on the general factor as well as on its specific factor where all factors (including correlations between general factor and specific factors) are specified to be orthogonal. The second-order factor model is more constrained than the bifactor model and is nested within the bifactor model (Yung, Thissen, & McLeod, 1999). In our samples, the second-order factor models show similar good fit as the four-factor models (see Table 3). For all behaviors, the four (first-order) factors load high on the (second-order) general factor (see Appendix B), which justifies the use of an overall score. As with the four-factor models, some of the second-order factor models show occasional anomalous values (i.e., negative residual variance for a latent variable or correlations of equal to or greater than 1). We also tested complementary bifactor models which showed comparably superior fit, however, not for all behaviors a model could be identified (see Appendix C).

Discussion and Conclusions

This report introduced the ACSID-11 as a new tool for the easy and comprehensive screening of major types of specific Internet-use disorders. The results of the study indicate that ACSID-11 is suitable to capture the ICD-11 criteria for gaming disorder in a multifaceted structure. Positive correlations with a DSM-5 based assessment tool (IGDT-10) further indicated construct validity.

The assumed multifactorial structure of the ACSID-11 was confirmed by the results of the CFA. The items fit well with a four-factor model representing the ICD-11 criteria (1) impaired control, (2) increased priority, (3) continuation/escalation despite negative consequences, as well as the additional components (4) functional impairment and marked distress to be considered as relevant for addictive behaviors. The four-factor solution showed superior fit compared to the unidimensional solution. The multidimensionality of the scale is a unique feature compared to other scales covering ICD-11 criteria for gaming disorder (cf. King et al., 2020; Pontes et al., 2021). Furthermore, the equally superior fit of the second-order factor model (and partly bifactor model) indicate that the items assessing the four related criteria comprise a general “disorder” construct and justifies the use of an overall score. The results were similar for online gambling disorder and the other potential specific Internet-use disorders measured by ACSID-11 in the multibehavioral format on the example of the ASSIST, namely online buying-shopping disorder, online pornography-use disorder, social-networks-use disorder. For the latter, there are hardly any instruments based on WHO criteria for disorders due to addictive behaviors, although researchers recommend this classification for each of them (Brand et al., 2020; Müller et al., 2019; Stark et al., 2018). New comprehensive measures, such as the ACSID-11, can help to overcome the methodological difficulties and enable systematic analyses of commonalities and differences between these different types of (potential) addictive behaviors.

The reliability of the ACSID-11 is high. For gaming disorder, the internal consistency is comparable or higher than that of most other instruments (cf. King et al., 2020). Reliability in terms of internal consistency is also good for the other specific Internet-use disorders measured by both ACSID-11 and IGDT-10. From this we can conclude that an integrated response format, such as that of the ASSIST (WHO ASSIST Working Group, 2002) is suitable for a joint assessment of different types of behavioral addictions. In the current sample, the ACSID-11 total score was highest for social-networks-use disorder. This fits with the relatively high prevalence of this phenomenon which is currently estimated at 14% for individualist countries and 31% for collectivist countries (Cheng, Lau, Chan, & Luk, 2021).

Convergent validity is indicated by medium to large positive correlations between ACSID-11 and IGDT-10 scores despite different scoring formats. Furthermore, the moderate positive correlations between ACSID-11 scores and the PHQ-4 measuring symptoms of depression and anxiety supports the criterion validity of the new assessment tool. The results are consistent with previous findings on associations between (comorbid) mental problems and specific Internet-use disorders including gaming disorder (Mihara & Higuchi, 2017; but see; Colder Carras, Shi, Hard, & Saldanha, 2020), pornography-use disorder (Duffy, Dawson, & Das Nair, 2016), buying-shopping disorder (Kyrios et al., 2018), social-networks-use disorder (Andreassen, 2015), and gambling disorder (Dowling et al., 2015). Also, the ACSID-11 (especially online gambling disorder and social-networks-use disorder) was inversely correlated with the measure of life satisfaction. This result is consistent with previous findings on associations between impaired well-being and symptom severity of specific Internet-use disorders (Cheng, Cheung, & Wang, 2018; Duffy et al., 2016; Duradoni, Innocenti, & Guazzini, 2020). Studies suggest well-being to be particularly impaired when multiple specific Internet-use disorders co-occur (Charzyńska et al., 2021). The joint occurrence of specific Internet-use disorders is not infrequent (e.g., Burleigh et al., 2019; Müller et al., 2021) which may partly explain the relatively high intercorrelations between the disorders measured by ACSID-11 and IGDT-10 respectively. This underscores the importance of a uniform screening tool to determine commonalities and differences more validly across different types of disorders due to addictive behaviors.

A main limitation of the current study is the non-clinical, relatively small and non-representative sample. Thus, with this study, we cannot show whether ACSID-11 is suitable as a diagnostic tool, as we cannot provide clear cutoff scores, yet. Furthermore, the cross-sectional design did not allow to make inferences about test-retest reliability or causal relationships between ACSID-11 and the validating variables. The instrument needs further validation to verify its reliability and suitability. However, the results from this initial study suggest that it is a promising tool that may be worth testing further. To note, a larger data base is needed not only for this instrument, but for the entire field of research to determine which of these behaviors can be considered diagnostic entities (cf. Grant & Chamberlain, 2016). The structure of the ACSID-11 appears to work well as confirmed by the results of the current study. The four specific factors and the general domain were adequately represented across the different behaviors, although each item was answered for all indicated online activities done at least occasionally in the last twelve months. We already discussed that specific Internet-use disorders are likely to co-occur, nevertheless, this must be confirmed in follow-up studies as the reason for the moderate to high correlations of ACSID-11 scores across behaviors. Furthermore, occasional anomalous values might indicate that for some behaviors the model specification needs to be optimized. The criteria used are not necessarily equally relevant to all of the included types of potential disorders. It may be possible that ACSID-11 cannot adequately cover disorder-specific features in symptom manifestations. Measurement invariance across the different versions should be tested with new independent samples including patients with diagnosed specific Internet-use disorders. Furthermore, the results are not representative of the general population. The data approximately represent Internet users in Germany and there was no lockdown at the time of the data collection; nevertheless, the COVID-19 pandemic has a potential influence on stress levels and (problematic) Internet use (Király et al., 2020). Although the single-item L-1 scale is well-validated (Beierlein et al., 2015), (domain-specific) life satisfaction could be captured more comprehensively in future studies using the ACSID-11.

In conclusion, the ACSID-11 proved to be suitable for the comprehensive, consistent, and economic assessment of symptoms of (potential) specific Internet-use disorders including gaming disorder, online buying-shopping disorder, online pornography-use disorder, social-networks-use disorder, and online gambling disorder based on ICD-11 diagnostic criteria for gaming disorder. Further evaluation of the assessment tool should be conducted. We hope that the ACSID-11 may contribute to a more consistent assessment of addictive behaviors in research and that it may become helpful also in clinical practice in the future.

Funding sources

Deutsche Forschungsgemeinschaft (DFG, German Research Foundation) – 411232260.

Authors’ contribution

SMM: Methodology, Formal analysis, Writing – Original Draft; EW: Conceptualization, Methodology, Writing – Review & Editing; AO: Methodology, Formal analysis; RS: Conceptualization, Methodology; AM: Conceptualization, Methodology; CM: Conceptualization, Methodology; KW: Conceptualization, Methodology; HJR: Conceptualization, Methodology; MB: Conceptualization, Methodology, Writing – Review & Editing, Supervision.

Conflict of interest

The authors report no financial or other conflict of interest relevant to the subject of this article.

Acknowledgements

The work on this article was carried out in the context of the Research Unit ACSID, FOR2974, funded by the Deutsche Forschungsgemeinschaft (DFG, German Research Foundation) – 411232260.

Supplementary material

Supplementary data to this article can be found online at https://doi.org/10.1556/2006.2022.00013.