Investigating the Validity and Reliability of the Persian Version of the Short Form of the Foreign Language Enjoyment Scale (S-FLES)

Introduction

Currently, language learning has become one of the main goals for families, students and individuals who need to communicate effectively with others to connect with the outside world or find a job. In this regard, in addition to a sense of necessity, a positive attitude towards language learning and enjoying the process per se can improve the quality and effectiveness of learning. According to some researchers [1,2], the enjoyment of learning a foreign language motivates learners to solve language problems and increases their desire to learn.

Similarly, the enjoyment of language learning occurs when learners are aware of their psychological needs in language classes [3-5]. These points of view emphasize the important role of enjoyment in language learning, noting its ability to reduce the level of anxiety associated with the learning process.

In fact, the origins of working with the emotions and feelings of language learners can be traced back to theory of positive psychology [6-9]. Among all emotions, anxiety receives the most attention. However, it is noted that focusing on the enjoyment of language learning can contribute to learners’ progress and help them achieve their academic goals [10].

To assess the level of language learning enjoyment among learners, the Foreign Language Enjoyment (FLE) scale was developed and its psychometric properties were validated. Language is the most developed intellectual tool used for communication between people and human communities. The FLE initially consisted of 21 items covering various factors influencing FLE [11, 12]. Such sets of instruments can lead to relatively long questionnaires, which in turn can negatively affect the response rate [13]. Therefore, to improve convenience without compromising reliability and validity, a shortened version of this scale was developed to achieve an optimal balance between psychometric reliability and time efficiency [14]. In this study, we attempted to validate the Short Form of the Foreign Language Enjoyment Scale (S-FLES).

Methods

The study design is presented graphically in Figure 1.

Figure 1. Flowchart of the study design.

Participants

Data collected from 304 participants were analyzed. Of these, 140 (46.1%) were female and 164 (53.9%) were male. The participants’ ages ranged from 18 to 43 years, with a mean age of 25.56±3.86 years. Most participants were single (219; 72%), while 85 (28%) were married. Education level varied: 38 participants (12.5%) had an associate’s degree, 90 (29.6%) had a bachelor’s degree, 26 (8.6%) had a master’s degree, and 150 (49.3%) were studying in the field of general medicine, dentistry, or pharmacy. As shown in Table 1, the total score for FLE was higher in females than in males, although this difference was not statistically significant (p>0.05).

Table 1. Comparison of foreign language enjoyment scores in samples of men and women

Inclusion and exclusion criteria

Participants were eligible for inclusion if they met the following criteria:

·          Age 18 years or above;

·          Currently studying at Ahvaz Jundishapur University of Medical Sciences;

·          Willing to participate in the study;

·          Proficiency in Persian;

·          Completion of an English language course.

Participants were excluded from the study if they:

·          Did not consent to participate in the study;

·          Suffered from a severe mental disorder such as bipolar disorder or schizophrenia;

·          Were experiencing suicidal ideation.

Instruments

The Rosenberg Self-Esteem Scale (RSE) [17] was used in this study. This instrument is a 10-item self-reporting questionnaire. Each item in the scale has four response options ranging from strongly agree (4 pts) to strongly disagree (1 pt). The test-retest reliability coefficients at two weeks, five months, and one year were 0.84, 0.86, and 0.62, respectively. Mousavi et al. [18] assessed the validity and reliability of this scale using internal consistency, test-retest reliability, and content validity methods. RSE, standardized in Persian, is one of the most widely and frequently used scales in Persian in various studies, demonstrating excellent psychometric properties. The reason for using the RSE to assess the convergent validity of the S-FLES was that previous studies have shown a direct correlation between self-esteem and enjoyment of foreign language learning. Therefore, we used this scale to measure convergent validity.

S-FLES is a 9-item scale used to measure enjoyment of learning a foreign language. Its original version includes 21 items. Each one is rated on a standard 5-point Likert scale ranging from 1 pt (strongly disagree) to 5 pts (strongly agree). This scale was developed and validated by Botes et al. [14] on 1,603 foreign language learners. It consists of three subscales: teacher appreciation, personal enjoyment, and social enjoyment. The S-FLES demonstrates good internal consistency, convergent validity, and discriminant validity [14]. The internal consistency of the 9-item S-FLES scale using Cronbach’s alpha is 0.81. In addition, each factor (social enjoyment, teacher appreciation, and personal enjoyment) demonstrated high internal consistency [14].

The Depression, Anxiety and Stress Scale – 21 Items (DASS-21) was developed by Lovibond [19]. It is a set of three self-reported scales for assessing the negative emotional states of anxiety, depression, and stress. Each subscale consists of 7 questions. Higher scores on this scale reflect higher levels of anxiety, depression, and stress in respondents. In the study by Antoni et al., the alpha coefficients for depression, anxiety, and stress were 0.97, 0.92, and 0.95, respectively. The correlation coefficients were 0.48 between depression and stress, 0.53 between anxiety and stress, and 0.28 between anxiety and depression [20]. In Iran, this questionnaire was examined by Samani and Jokar, who calculated test-retest reliability coefficients for stress, anxiety, and depression as 0.77, 0.76, and 0.80, respectively [21].

The DASS-21 standardized in Persian is another widely used scale in Persian language research, showing good psychometric properties. The reason for using the DASS-21 to assess the divergent validity of the S-FLES is that previous studies revealed an inverse correlation between the DASS-21 components (depression, anxiety, and stress) and foreign language enjoyment. Thus, this scale was used to assess the divergent validity. Their role in our study is related to measuring self-esteem for assessing convergent validity and DASS-21 for assessing divergent validity.

Data analyses

Our study was descriptive in its nature and focused on psychometric assessments. The following descriptive and inferential methods were used for statistical analysis:

1. Independent Student’s t-tests were used to compare men’s and women’s enjoyment scores using SPSS25.

2. Cronbach’s alpha coefficient was calculated to determine internal consistency using SPSS25.

3. Pearson correlation tests were performed to examine relationships between variables using SPSS25.

4. Confirmatory factor analysis was conducted using LISREL 10.

The following fit indices were considered to assess the fit of the model to the factor structure of the questionnaire: chi-square to degrees of freedom ratio (X²/df), comparative fit index (CFI), goodness of fit index (GFI), adjusted goodness of fit index (AGFI), incremental fit index (IFI), relative fit index (RFI), normed fit index (NFI), non-normed fit index (NNFI), standardized root mean square residual (SRMR), and root mean square error of approximation (RMSEA). As a rule, an X²/df ratio below 3 implies good model fit, although this index is highly dependent on sample size; hence, values above 3 may still indicate acceptable fit depending on sample size. Generally, RMSEA value below 0.08, SRMR value below 0.09 and values of fit indices (CFI, GFI, AGFI, IFI, RFI, NFI and NNFI) above 0.90 indicate acceptable model fit, while values of the latter in the range of 0.80-0.90 are considered marginally acceptable [22, 23]. AGFI values above 0.85 in confirmatory factor analysis further confirm acceptable model fit. Pearson correlation using SPSS-25 was employed for data analysis. Confirmatory factor analysis was conducted using LISREL 8.80 software.

Results

Data from 304 participants were analyzed. Among them, 140 (46.1%) were female and 164 (53.9%) were male. The age range of the participants was 18-43 years with a mean ± standard deviation of 25.56±3.86 years. Among the participants, 219 (72%) were single and 85 (28%) were married. In this study, 38 participants (12.5%) had an associate’s degree, 90 (29.6%) had a bachelor’s degree, 26 (8.6%) had a master’s degree, and 150 (49.3%) were studying general medicine, dentistry, or pharmacy. As shown in Table 1, the total score for the enjoyment of learning a foreign language was higher in females than in males, but the difference was not statistically significant (p>0.05).

Correlation coefficients of individual subscale scores with the total score ranged from 0.61 to 0.73 (Table 2).

Table 2. Correlations between subscale scores and the total score

Table 2 demonstrates a statistically significant correlation between the subscale scores and the total score. Additionally, we observed significant associations between the subscale scores.

Reliability

Internal consistency was assessed using Cronbach’s alpha, which was 0.85 for the total scale, 0.84 for the personal enjoyment score, 0.81 for the teacher appreciation score, and 0.81 for the social enjoyment score.

Confirmatory factor analysis

The results of confirmatory factor analysis for the first-order three-factor model and the second-order three-factor model are presented in Tables 3 and 4 respectively. It is clear that both the first-order three-factor model and the second-order three-factor model of the S-FLES demonstrate acceptable fit (Figures 2 and 3).

Table 3. Fit indices for the first-order three-factor model of S-FLES

Table 4. Fit indices for the second-order three-factor model of S-FLES

Figure 2. Three-factor model of S-FLES.

Chi-Square=34.11, df=24, P-value=0.08265, RMSEA=0.037; S-FLES, Short Form of the Foreign Language Enjoyment Scale; RMSEA, root mean square error of approximation; Teacher, teacher appreciation score; personal, personal enjoyment score; social, social enjoyment score. 

Figure 3. Three-factor model of S-FLES.

Chi-Square=34.11, df=24, P-value=0.08265, RMSEA=0.037; S-FLES, Short Form of the Foreign Language Enjoyment Scale; Teacher, teacher appreciation score; personal, personal enjoyment score; social, social enjoyment score.

The fit indices for the second-order factor analysis model are presented in Table 4. As shown in the table, the three-factor structure of the S-FLES demonstrates acceptable fit to the model.

Convergent and divergent validity

Table 5 confirms good convergent and divergent validity of the S-FLES through its statistically significant relationship with self-esteem (p<0.05), along with an inverse significant correlation with depression, stress and anxiety (p<0.05).

Table 5. Convergent and divergent validity of the S-FLES

Discussion

No other study compared the S-FLES scale in other languages. The results of the original study in comparison with our study are discussed below.

Enjoyment of learning a foreign language is a complex and multidimensional feeling. It is a new, promising, but at the same time unexplored area of research. Studying the enjoyment of learning a foreign language from a positive psychology perspective can help us understand why some people succeed in learning a language, while others perceive it as a long and difficult process. Our goal was to examine the psychometric properties of the S-FLES. The results confirmed the presence of a second-order factor structure and three first-order factors for the 9-item S-FLES scale. The three underlying FLE factors in the S-FLES were teacher appreciation, personal enjoyment, and social enjoyment.

This three-factor structure is similar to the findings of Jin and Zhang [11], Lee et al. [12], and Botes et al. [14]. In fact, the personal and social enjoyment aspects form the core of the FLE in most studies examining the factor structure of the FLE [14]. The crucial role of a teacher (mentor) appreciation by a foreign language learner was also found in other higher-order measurement models of the FLE [11, 12]. Thus, the three-factor structure suggests a nuanced understanding of enjoyment in the English classroom, focusing not only on the individual but also on peers and teachers. Future uses of the scale could employ the total S-FLES score or the three S-FLES subscale scores, depending on the research questions.

In the field of applied linguistics, enjoyment of foreign language learning has been recognized in recent years as a cornerstone of research of individual differences [24, 25]. FLE is broadly defined as a positive emotion experienced when psychological needs of an individual are met during challenging language learning activities [26]. FLE can be viewed as a manifestation of positive psychological principles in applied linguistics [27]. FLE is associated with higher academic achievement [26], faster development of second language (L2) proficiency [8], higher perceived competence [28], stronger motivation [8], greater readiness to communicate in the target language [29], lower anxiety when learning a foreign language in the classroom [26, 30], and reduced boredom when learning a foreign language [31].

In our study, the structure of the S-FLES demonstrated good fit. In addition, the S-FLES demonstrated good reliability and validity. The internal consistency of the scale, both at the level of individual subscales and as a whole, was acceptable. Evidence was obtained for both convergent and divergent validity of the scale. Therefore, the use of the S-FLES is recommended for studies investigating the enjoyment of foreign language learning under time constraints. Given the limited number of items, this instrument can be easily incorporated into studies examining multiple variables. In ongoing positive psychology research in applied linguistics, the S-FLES methodology may serve as a valuable tool for enhancing FLE and overcoming its negative counterparts such as classroom anxiety.

These findings offer an opportunity for English teachers, English learners, and English course directors to pay special attention to social interaction and creating a positive classroom atmosphere. This may potentially increase students’ FLE. Given that higher FLE correlates with higher proficiency in foreign language on such measures as academic achievement [27] and faster development of second language comprehension [8], increasing students’ FLE should be a priority. Teachers should consider their role in developing students’ interest in learning English, both through interactions with English language learners and by creating a positive social environment in the classroom. Teacher behaviors such as humor and unpredictability are linked to FLE as well [25, 29].

Furthermore, the use of specific teaching methods such as interactive games [32] and cooperative learning [33] can contribute to creating a positive and joyful classroom environment. Positive participation in the classroom social environment undoubtedly influences the social enjoyment and teacher appreciation factors associated with FLE. Ultimately, the S-FLES serves as an easy-to-use tool for teachers and researchers interested in implementing measures to improve a positive classroom environment. It allows them to assess whether their efforts result in a significant impact.

The current study has a correlational descriptive design; hence, causal relationships cannot be established. In addition, the student sample in this study may not be representative of the general population, as it was conducted among medical students in Ahvaz, excluding individuals of school age and other age groups. These individuals may serve as a target population for future studies. Test-retest reliability was not assessed in this study; therefore, there is insufficient information on the steadiness of results over time. The study sample was predominantly composed of students, young adults, and women, which may limit the generalizability of the results and create limitations on their applicability to a wider population.

Conclusion

The results of our study support the validity and reliability of the S-FLES. Since the scale is new to the Persian language, further validation studies and examination of its psychometric properties on different samples are needed.

Also, our results support the structural validity, reliability, as well as convergent and divergent validity of the S-FLES, a multidimensional 9-item instrument. The S-FLES is a reliable and valid instrument with the advantages of being cost- and time-saving due to the reduced number of items, ease of its interpretation due to the clearly structured measurement model, and reliable for both participants and researchers. Therefore, we propose that this instrument can be used for the advancement of emotion research in applied linguistics.

Ethical approval

All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Declaration of Helsinki and its later amendments or comparable ethical standards.

Conflict of interest

There is no conflict of interest.

Funding Statement

This work is under the support of Ahvaz Jundishapur University of Medical Sciences, Ahvaz, Iran [U-03069] with approved Ethical Code: IR.AJUMS.REC.1403-012.