SMS text messaging to measure working time: the design of a time use study among general practitioners

The use of one’s working time plays a pivotal role in health workforce planning aimed at maintaining, or restoring, a balance between the supply and demand for care provided by doctors. How many hours doctors actually work, related to their patients or not, is critical for assessing the supply that is available and needed in the future. This applies particularly to general practitioners (GPs) who function as gatekeepers in many health care systems and who play a central role in the access to care [1]. An important challenge, from a research perspective, is how to achieve a valid, reliable and accommodating method of measuring GPs’ working time. GP care is provided around the clock in most countries, so a measurement instrument to determine the working hours and use of time by GPs needs to be applied at any moment of a day or week. At the same time, many GPs complain about increasing administrative tasks, hence the possibility of participating in time consuming research is very limited, even though they realize how important this is.

There is currently an awareness of the availability, reliability and feasibility of methods to research the use of working time. But very little is known about methods or tools that can be applied to measure the working time among health care professionals such as GPs. A general rule is that these methods should match the setting it is applied to. This will depend on the target group and the type or level of activities being measured.

The dominant methods used to measure working time are observations, surveys and diaries. Observational research is considered to be a reliable research technique [2‐4], but it is difficult and costly to apply on a large scale. This is particularly true when studying large numbers of work settings such as small general practices where observations of multiple respondents are only possible to a limited extent. The survey method is less costly but suffers from important disadvantages. Inaccuracy can occur due to the impact of recall bias and the socially desirable incentive for respondents to overestimate their working hours and specifically activities that are generally considered as a burden, such as administrative tasks. Clearly, these limitations also apply to GPs the subject of our study. Furthermore, it appears increasingly difficult to obtain sufficiently high response rates in survey research [5]. The diary method is considered as a good alternative to the observational method and is assessed as a reasonably reliable research technique [6]. However, the important limitations of keeping a conventional paper-based diary are the time and effort required from respondents and researchers [7]. There is, however, a growing number of communication and mobile devices that can reach respondents and send them simple signals or questions to measure their activities in real time. This type of research for sampling work activities has already been applied among nurses in 1995. These nurses carried a “bleep device” which provided a signal at random moments, after which they had to document their activity [6].

During the past few years, new communication techniques such as personal digital assistants (PDAs) [8‐10], and smartphone apps [11] have been used increasingly as data collection tools. These tools yield more possibilities, but have their limitations as well. Devices such as PDAs require mobile internet availability and downloading of data afterwards which can result in possible data loss [8, 12]. The proper development, testing and design of smartphone apps yield relatively more expenses and are often dependent on the user having the latest version of the Android or iOS system [13]. However, Short Message Services (SMS) or text messaging lacks these compatibility or dependency problems as it is a relatively ‘old’ technology. It is a cheap and highly reliable type of mobile communication and is available on mobile devices of every kind. In almost all developed countries, nearly 100% of the population use a mobile phone and are able to send and receive SMS text messages. This makes SMS an attractive tool for measuring working time as it provides an easier and cheaper way to collect real time data from large populations. There are, however, few studies in which SMS have been used to measure the working time of certain populations. One of the few exceptions is a validity study of Brenner and colleagues [13] who applied SMS text messages to measure the use of time among university students. To our knowledge, there are no studies about the use of time in which SMS have been applied among health care professionals such as GPs.

We piloted an SMS-based measurement among 14 GPs to test if an SMS tool would be a proper instrument to measure their working hours in real time [14]. This achieved positive results. It was experienced as feasible by the respondents and delivered valid measurements. For example, a response was received on 96% of the activity messages sent and 66% of the messages sent was replied within 10 min. Furthermore, the respondents confirmed their answers on the activity messages after the week of SMS messages. Subsequently, we applied the design and SMS instrument on a larger scale in a project including more than 1000 Dutch GPs over a period of 14 months. In this paper, we describe the development and application of this real time measurement tool for conducting large-scale research into the use of working time. The research question that will be addressed is: “How could a real time measurement tool be developed; and what can we say about its feasibility and ease of implementation?”

The target group of the SMS-based measurement of the use of working time were all active and employed GPs in the Netherlands. In 2013 the population concerned 11,075 GPs [15]. Two types of samples were designed:

Considering this two-staged sampling approach, we aimed to include at least 1000 GPs in our study.

Our study sought to describe the deployment, feasibility and ease of implementation of an SMS-based time use method applied among Dutch GPs. The research was conducted over a period of 57 consecutive weeks in which 1051 individual GPs participated, an average of almost 20 participants per week. During the survey week, the participants received multiple messages on a daily basis which questioned them about their activities at random selected points in time (in blocks of three hours).

The study showed that by applying the SMS instrument of time use it is possible to collect valid and rich data in a user friendly manner among a large group of respondents – in this case Dutch GPs. The measurement tool was shown, in particular, to be a feasible instrument for collecting data about the use of working time among different groups of GPs and for a limited number of activities. The high response rate to all the SMS messages sent, and the positive reactions of GPs after participation, are good indicators of its success.

There are several factors which contributed to the success of this study. Firstly, a lot of effort was invested in the recruitment of the participants. More than 5000 invitation letters were sent with support from professional organisations such as the Dutch GP association. In addition, a significant number of GPs subscribed to the research not directly invited. Secondly, although only a relatively small allowance for GPs was offered to compensate for their SMS costs - a gift voucher of €10 - this was sufficient for their recruitment. Thirdly, the design of the SMS instrument was driven by the aim to reduce the burden on the participants as much as possible. GPs could easily reply on the SMS messages by simply entering a number and one multi choice letter. Additionally, participants could indicate when they were not working during a particular part of the day, avoiding their being interrupted unnecessarily with so-called activity messages. Finally, the SMS service provider developed an SMS application which ideally suited the aim of the research which was to measure the working time of GPs in a valid and user-friendly manner. This system managed to measure the working time of different GPs at all times during one year with a 99.9% uptime.

The use of the SMS service provider, who was responsible for sending and receiving the SMS messages and for processing the data, did involve considerable costs. In particular there were relatively high costs for the development of an SMS-application which suited the aim of our study. These start-up costs would be considerably lower, however, were this research to be repeated in collaboration with the same SMS service provider. Additionally, limited material costs were involved as almost everybody has a mobile phone at their disposal to send SMS messages [20] and the sending of SMS is becoming cheaper. A paper-based diary method would have involved considerably higher material costs for paper and postage. It is also very likely that the costs of the SMS instrument are lower compared to observations on large scale as these would have resulted in the higher costs and greater effort required for observing the GPs [3].

The design of the method was aimed at GPs in the Netherlands. Nonetheless, this instrument could be applied in other countries as well, because of the broad availability of mobile phones even in low- and middle income countries [8]. The method could, with some adaptations, also be applied to other target groups. For instance, just as with GPs, it is important to gain insight into activities, related to patients or not, of other health care professions which can support manpower planning [21]. Even professional groups not involved in health care such as bus drivers or construction workers are a possible target group, although here the response categories would be defined differently. It would be necessary to adapt other parts of the design, depending on the working time of which target group will be measured. For example, it may not be necessary to measure the use of time 24 h a day, seven days a week, when out of office hours are not relevant. Yet, at the same time, it would be possible to perform more measurements per respondent during a day and include a lower number of participants.

The design of the SMS-based study described is adequate to obtain data from some general activities. A single SMS provides enough memory for a simple question and a limited number of response categories, but the obtained information is more limited compared to many other methods. For example Sinsky et al. developed a time and motion study in which the allocation of time on 12 activities of physicians was recorded by observers. The observers coded among others what the physicians were doing, where they were doing it and with whom they were engaged [22]. Yet, with some adaptations to the design the SMS tool can also provide more detailed information about certain activities, for example when it concerns the time spent on specific indirect patient-related activities such as travelling to a patient or the registration of patient data. An option is to send two or more types of questions with a smart allocation of questions for different respondents. One could also think of two or more consecutive questions based on the response to the first question.

The SMS method appears to be a feasible method to collect real time data about the working hours of large populations and different subgroups, during one year. The method has potential for broader application in other countries and among other target groups, possibly with some small adaptations. Further research has to provide more insight into the question of to what extent the real time measurement tool could be applied reliably among a smaller number of respondents during a shorter period of time.