Establishing the HLS-M-Q18 short version of the European health literacy survey questionnaire for the Malaysian context

An earlier population-based cross-sectional study [7] has validated the suitability of HLS-EU-Q47 for the Malaysian context. The present study also employed a cross-sectional design on the Malaysian population based on ethnic distribution to ensure that the short version instrument reflects the country’s varied ethnicities. Participants were between the age of 18 to 60 years old. Adapted measures from the HLS-EU-47 [8] validated in English and Malay in a previous study [7] were utilised. However, a 3-level face validation process was conducted and our researchers have restructured the sequence of some items to allow for better comprehension and reduce confusion for respondents. Some items were also reworded upon recommendation by health education experts through the face validation stage.

The survey was administered by well-trained interviewers working for the Ministry of Health Malaysia. Three states were selected (Selangor, Kuala Lumpur and Sarawak) to represent the distribution of multiple ethnicities, as well as the distribution of urban and rural areas. The selection of areas were made based on referral and advice by the District of Jurisdiction Malaysia, Rural Master Plan Malaysia, and previous literature [9].

This study was submitted for ethical review and received ethical approval from the National Medical Ethics Committee Malaysia which governs all medical/health related research in Malaysia. The National Medical Research Registration (NMRR) ID obtained for this study is 41882 and approval number is NMRR-18-1320/41882. The NMRR approval is the only ethical approval needed as this project was submitted under the Ministry of Health Malaysia and the National Medical Ethics Committee Malaysia is the Ministry’s Institutional Regulatory Board for Ethical Approval.

All respondents were above 18 years old and therefore involved no minors. All respondents also signed a written consent form clearly stating their rights and nature of participation in the study before being asked to answer the survey. This consent form was also submitted and approved by the National Medical Ethics Committee Malaysia.

Multi-stage random sampling was used in this study. Specifically, there were three stages involved, utilising several sampling techniques (quota sampling, cluster sampling and simple random sampling) to allow random data collection. The three stages are as in Fig. 1. The researchers made the decision to prioritise an inclusive Malaysian sample based on ethnicity and urban/rural strata due to constraints in resources. This was to ensure that the smaller groups were adequately represented in the sample. The list of states, ethnicities and urban/rural distribution required for this study are as presented in Table 1.

In stage 1, quota sampling based on ethnicities and urban/rural distribution were used to select three Malaysian states. Ethnic distribution should be a standard in sampling multiracial populations to ensure inclusivity of the sample [10]. States from both Peninsular Malaysia and Borneo were selected to represent the diverse ethnicities in Malaysia. For the purpose of urban and rural distribution, Kuala Lumpur was selected to represent the urban area majority. This is justified as Kuala Lumpur has the highest urban population in Malaysia. Sarawak was selected to represent the rural distribution, as well as to give more balanced representation of the minority ethnic groups. Selangor represents both urban and rural areas and has a balanced ratio in ethnic group distribution. Selection of the three states was decided upon discussion between researchers and the Ministry of Health Malaysia.

In stage two, researchers utilised cluster sampling to determine districts of choice. District sampling for Selangor was determined based on the demographic distribution list published by the Selangor Economic Development Unit, as well as extant literature [9]. For selection of districts in Kuala Lumpur, researchers used data provided by the Department of Information, Ministry of Communications and Multimedia Malaysia; and for selection of districts in Sarawak, the selection of districts was guided by data provided by the State Director of the Fire and Rescue Department. The definitions of rural and urban were determined by the National Department of Statistics and The Rural Master Plan, published by the Ministry of Rural Development Malaysia.

In stage three, respondents were selected based on a simple random sampling technique based on several criteria (i.e., Malaysian, aged 18 and above, resident in the chosen state, able to make health decisions for themselves). This is also the same protocol criteria used by the Asian Health Literacy Consortium. Only one person was selected in each household, in which the eldest household member would be chosen if there was more than one person who met the respondent selection criteria (similar technique used in literature) [11].

Sample size was calculated based on the minimum requirement for CFA rule of ten [12]. Based on the 47 items, the study would need a minimum of N = 470 in order to perform CFA with a 95% confidence level. However, the researchers decided to increase the number of respondents to improve the confidence interval, thereby reducing the margin of error [13].

Data collection was conducted between 25th June 2018 to 14th July 2018, involving 18 enumerators. Respondents took an average of 30–40 min to fill in the questionnaire.

The health literacy survey questionnaire (HLS-EU-Q47) contained 47 items measuring health literacy. The perceived difficulty of each item was rated on a 4-point Likert scale which ranged from 1 = very difficult to 4 = very easy. The HLS-EU-Q47 was developed based on a conceptual model of health literacy which measures four individual competencies (the ability to access, understand, appraise, and apply health information) across three domains i.e.: health care, disease prevention and health promotion [8].

The enumerators went from house to house within the selected areas and provided the self-reported questionnaire to be answered. A consent form was filled in and obtained from each participant. Although the researchers aimed to collect responses from 800 respondents, a total of 866 complete responses with no missing data were obtained throughout the data collection period and analysed. All trained interviewers wore the Ministry of Health uniform and identity card to avoid misunderstanding and protect the interest of both researchers and respondents.

A CFA was conducted to produce a model with good fit indices and a set of items which are suitable for the measurement of health literacy. To validate the construct, the researchers referred to the CR, AVE, MSV, and ASV values [17]. CR values above 0.7 and AVE values exceeding 0.5 indicate good convergent validity whereas MSV and ASV values that are smaller than AVE indicate good discriminant validity of the constructs.

The test results indicate that the model fit did not adhere to goodness-of-fit indices (Table 3). The analysis indicated that the first tier of CFA resulted in χ2 = 21,718.433, df = 6867, p = 0.000, χ2/df = 8.007, GFI = 0.664, CFI = 0.683, NFI = 0.654, TLI = 663. and RMSEA = 0.090. Meanwhile the CMIN/df showed that the value exceeded 5. These results illustrated that the model was not compatible with the data of the study. Nevertheless, in the aspect of construct validity, the model had produced good convergent and discriminant validity (Table 4).

To improve the model fit, a modification model was developed to determine the measurable items that complied with goodness of fit indices and displayed good convergent and discriminant validity.

To determine whether the model developed met the goodness-of-fit indices, modifications were made to the original model. The result in Table 3 exhibited that the modification model fulfilled the goodness-of-fit indices with χ2 = 395.956, df = 121, p = 0.000, χ2/df = 3.272, GFI = 0.950, CFI = 0.955, NFI = 0.937, TLI = 0.943 and RMSEA = 0.051. This finding further illustrated that the items used to measure the health literacy model were suitable with 18 items with the data of the study.

Furthermore, the modification model showed strong construct validity in the aspects of convergent and discriminant validity (Table 5). Even so, the developed model would not be able to retain the original sub-domains in the original 47-item health literacy model from previous survey tools [6].

Table 6 illustrates the summary for instrument validity which refer to the Cronbach’s alpha values. All of the health literacy domains in the 18-item model indicate reliability levels (more than 0.7) across the three domains (healthcare, health promotion, and disease prevention).

The construct validity has also been reviewed by observing the Pearson correlation values of each item against the total scores of the measured variables. Based on previous literature, correlation values above 0.25 are good and signal high construct validity [17]. Correlation scores can be categorised into 3 different stages; low (0.10 to 0.29), average (0.30 to 0.49) and high (0.50 to 1.00) [18]. The result of this study has shown that the correlation value of each item with their overall domain value is good (0.414 to 0.734).

The HLS-M-Q18 was validated among 866 respondents. CFA is an important step to confirm the domains of health literacy and define its structure. The HLS-EU-Q47 instrument was designed to measure the multiple aspects of health literacy, namely accessing, understanding, evaluating and applying health information in the contexts of health care, disease prevention and health promotion. The initial model (HLS-EU-Q47) showed poor goodness-of-fit and was therefore modified to achieve acceptable fit indices.

In evaluating model fit, guidelines suggested by previous literature were adhered [14]. The researchers constructed three modification models and the results were satisfactory with an 18-item model. There were high correlations among the 18 items, ranging from 0.414 to 0.734 using Pearson Correlation.

The modification model, HLS-M-Q18, consisted of 18 items that retained 9 out of the 12 original domains of the HLS-EU-Q47. For health care, the model retained only three out of the four sub-domains: understanding, appraising and application of information. Three domains were retained for disease prevention and health promotion: access, understanding and appraising of information. The exclusion of three original domains of the HLS-EU-Q47 may have decreased the conceptual representation of the original model but has resulted in a valid and reliable measure with increased utility for the Malaysian population.

The HLS-M-Q18 showed satisfactory convergent and discriminant validity (CR > 0.7, AVE > 0.5, MSV < AVE and ASV < AVE), demonstrating that the items within the sub-domains were adequately related and that the sub-domains were sufficiently different from one another.

The instrument (HLS-M-Q18) was reliable, with high internal consistencies. All sub-scales achieved Cronbach’s alpha of 0.80 and above, and the overall instrument achieved 0.91. These results represented equivalence and consistency among the responses to items of HLS-M-Q18, suggesting that these items were suitable in measuring people’s health literacy in the Malaysian context. The internal consistency reliability was robust, with no floor/ceiling effects. However, the results are different as compared to the previous version of HLS-EU-Q16 [2, 17]. This indicates that the model hinges on the context of the country. The HLS-EU-Q16 did not achieve a good model fit in the Malaysian population. Therefore, the researchers used the HLS-EU-Q47 as a benchmark to shorten its version for the context in Malaysia. This is in line with many other studies that utilised the HLS-EU-Q47 as a benchmark for reducing the items for their respective countries [2, 7, 19].

Sampling for the survey was conducted using a quota sampling procedure that was based on population parameters such as ethnicity and place/ area of residence. Data was collected from three selected states in Malaysia. Due to the random sampling procedure utilised prioritising ethnic group and urban/ rural stratas, this study did not precisely reflect the current gender and age distribution in Malaysia. In this study, 65% of our respondents were women while the Malaysian population estimates women at 49% [20]. The age distribution in this study observed a high proportion of respondents between the ages of 25 to 42 years (51% of the sample) as compared to the Malaysian population which indicates that only 32.9% Malaysians are between the ages of 25 and 44 [20]. The use of the HLS-M-Q18 within larger Malaysian settings would provide further evidence on the validity and reliability of the instrument, particularly with the country’s ethnic, age, gender and social economic composition taken into consideration.

A second limitation of the study is that, as with most short-version instruments, the HLS-M-Q18 may not be able to measure each health literacy domain independently. Rather, it is most appropriate for the measurement of health literacy as a single, unified domain. As a consequence, the shortened instrument may not be suitable for all situations. The validity and reliability of instruments are commonly compromised in their shorter versions. It was therefore important that convergent and discriminant validity of the HLS-M-Q18 were met, as demonstrated in this study.

Previous research recommended that the HLS-EU-Q47 be used for a comprehensive measurement of health literacy [7]. Even so, the authors also suggest that cultural settings be considered when adapting the instrument for use, emphasising the importance of testing and validating in a population to ensure suitability. At the same time, a shorter version of the health literacy instrument is preferable due to its practicality of use, especially when used in combination with the measurement of other health variables (such as in the National Health Morbidity survey). With the development of the HLS-M-Q18, health literacy in Malaysia is able to be measured with minimal difficulty and a shorter response time.