In this study, a valid and reliable Farsi tool was prepared using a questionnaire design method to assess the level of satisfaction with the quality of dental services and communication with the dentist among adult patients referred to the School of Dentistry of Tehran University of Medical Sciences (TUMS) in 2021–2022. Data collection was completed from January 31,2021 to March 10, 2021. We corresponded with the authors of the original SERVQUAL instrument [8], and they officially authorized us to use their tool. Accordingly, we considered the SERVQUAL instrument and its domains as the main framework for our new instrument. To identify appropriate items for the questionnaire, a comprehensive review of existing instruments related to patient satisfaction and healthcare service quality was conducted [1, 5, 7, 8, 11,12,13,14,15,16,17,18,19,20,21]. Items were selected based on their conceptual alignment with the five dimensions of the SERVQUAL model and their applicability to dental care services. According to this search, 48 initial items were identified. Two specialists in community oral health reviewed the items to assess content relevance, cultural appropriateness, and linguistic clarity. Based on their feedback, 15 items were removed or revised due to redundancy, ambiguity, or lack of relevance, resulting in a reduced pool of 33 items. Further modifications included rewording for clarity and adjusting terminology to better reflect the dental care context. Face and content validity of the remaining items were then assessed. As a result of this process, two additional items were removed, and the final version of the preliminary questionnaire consisted of 31 items distributed across five SERVQUAL dimensions: tangibility (5 items), reliability (12 items), responsiveness (3 items), assurance (7 items), and empathy (4 items). The CVR values ranged from 0.60 to 1.00, indicating acceptable necessity of the retained items, while the CVI values ranged from 0.86 to 1.00, confirming strong content relevance and clarity.
The assessment of validity and reliability will be presented in detail:
Face validity
The researcher (K.Sh) conducted interviews with ten adult patients referred to the School of Dentistry to assess the clarity of the questions. The questions were read aloud, and modifications were made as needed. Additionally, experts qualitatively evaluated the questions for their appearance and comprehensibility [22].
Content validity
Ten experts, including eight specialists in community oral health, one psychologist, and one epidemiologist, evaluated each question for relevance, clarity, and simplicity to calculate the Content Validity Index (CVI). They also assessed the necessity of the questions (categorized as necessary, useful, or unnecessary) to determine the Content Validity Ratio (CVR). Questions with a CVI below 0.7 were either deleted or, if deemed important, revised and re-scored [11]. Additionally, the average necessity rating of the questions was calculated, with a target of exceeding the acceptable level of 60%. The reason for selecting 10 experts is based on Lawshe’s method, which states that with a panel of 10, a minimum of 60% agreement (CVR ≥ 0.62) is sufficient to determine the necessity of an item, making it a statistically acceptable and commonly used standard in content validity assessments [23].
Convergent validity
In assessing convergent validity, two tools that measure the same construct are compared, and they are expected to have a high correlation with each other. In other words, the results of the designed tool are compared with another valid tool that measures the same construct, and if the new tool has a high correlation with the standard tool, it is assumed that these two instruments measure a single construct. As a result, the new tool is also considered valid for the target construct. For this purpose, the Pearson correlation coefficient between the scores obtained from the two tools was calculated. Values between 0.81 and 1 were considered excellent, 0.61 to 0.80 as very good, 0.41 to 0.60 as good, 0.21 to 0.40 as moderate, and 0 to 0.20 as poor correlation [24].
In this study, the Persian version of the Dental Satisfaction Questionnaire (DSQ) was used as a valid and reliable tool to check convergent validity [6]. The DSQ questionnaire included domains of dentist skills, quality of services, ease and availability of services, pain and discomfort during treatment, cost, empathy and responsibility of dentists, and accessibility.
Construct validity
Confirmatory Factor Analysis is a method that shows how many items to measure a construct are correctly selected. In fact, in this method, it is determined whether the items chosen in a questionnaire to measure each factor are suitable. In confirmatory factor analysis, the goal is to ensure the existence of a regular factor structure (25). In this study, factor analysis was performed using IBM SPSS Amos (Version 6.0; IBM Corp., Armonk, NY, USA).
Regarding the RMSEA (Root Mean Square Error) index, values less than 0.05 indicate a good fit of the model with the data, and values less than 0.10 for this index indicate an acceptable fit of the model with the data. Additionally, values between 0.05 and 0.08 are acceptable, and values between 0.08 and 0.1 are considered borderline values [25]. Regarding the CFI (Comparative Fit Index) and GFI (Goodness of Fit Index) indices, values above 0.9 indicate an appropriate and desirable model fit [26, 27].
The reliability of the instrument
The test-retest method was used to assess the reliability of the questionnaire in terms of stability over time. For this purpose, 56 adult patients referred to the School of Dentistry of Tehran University of Medical Sciences (TUMS) were asked to complete the questionnaire twice at a two-week interval. In the first stage, the questionnaire was completed through face-to-face interviews, and in the second stage, it was completed over the phone due to the possibility of the patients not being able to attend again and the spread of the coronavirus.
In this study, two indices were used to evaluate test-retest reliability: the intra-class correlation coefficient (ICC) and the percentage of agreement. To evaluate the reliability of the overall questionnaire, the ICC was calculated, measuring the consistency of total scores between the two time points. In addition, to assess the reliability of individual items, the percentage of agreement between the two administrations was computed. This item-level analysis allowed for identifying any inconsistencies in patient responses to specific questions. Coefficients were interpreted as follows: values below 0.4 indicated poor reproducibility, values between 0.4 and 0.75 indicated good reproducibility, and values above 0.75 indicated very good reproducibility [28].
Internal agreement was assessed using Cronbach’s alpha, a measure of how closely related a set of items are as a group. Items with a Cronbach’s alpha of ≥ 0.7 were retained in the final instrument, following established reliability thresholds [29]. For questions that did not meet this criterion, modifications were made (e.g., rewording or removal), and reliability was re-evaluated to ensure acceptable consistency.
Sample size
Sampling was conducted among available adult patients (aged 18 years and older) referred to the clinical departments of the School of Dentistry at Tehran University of Medical Sciences, excluding the Department of Pediatric Dentistry. Two different sample sizes were used in this study, each determined by the specific statistical analysis it supported.
To carry out Confirmatory Factor Analysis (CFA), a technique used to test whether the data fit a hypothesized measurement model, a sample size of at least 200 participants was required. This criterion was based on the common recommendation of including 10 to 20 participants per questionnaire item to ensure statistical adequacy. The appropriateness of the sample size was also confirmed using the Kaiser-Meyer-Olkin (KMO) measure of sampling adequacy, which verified the suitability of the data for factor analysis [30]. A total of 208 participants were ultimately included for this purpose.
In contrast, a smaller sample was appropriate for analyses focused on test-retest reliability and convergent validity, which assess the stability and agreement of the instrument rather than its underlying factor structure. These analyses typically require fewer participants. The necessary sample size was calculated using Power Analysis and Sample Size (PASS) software, version 11.0, based on the intra-class correlation coefficient (ICC) method and assuming a statistical power of at least 80% [31, 32]. Based on this calculation, a sample of 56 participants was considered sufficient. It should be noted that these 56 participants were a subset of the total sample of 208 participants used for CFA.
Statistical analysis
Data were analyzed using IBM SPSS Statistics for Windows, Version 25.0 (IBM Corp., Armonk, NY, USA). and IBM SPSS AMOS, Version 6.0. Psychometric evaluation of the instrument included assessments of validity (content, convergent, and construct) and reliability (test-retest stability, internal consistency, and item-level agreement), all conducted using standard statistical methods as previously described.
link