Background
Methods
Framework development
-
Map and identify the indicators already being measured at each call centre;
-
Map and identify the data collection systems in place at each call centre;
-
Identify country specific elements that need to be considered in a framework, (e.g. cultural norms and attitudes); and
-
Provide broad direction for the development of the Framework.
Pilot evaluation in Mexico
Results
Framework development
Number of call centres (N = 12) | ||
---|---|---|
Indicators currently utilised | Call numbers/tracking | 8 |
Service bookings/referral rates | 9 | |
Nature of call | 4 | |
Client feedback | 2 | |
Evaluation techniques currently utilised | Assess call centre operators | 5 |
Mystery client callers | 4 | |
Phone interviews | 2 | |
Client satisfaction | 4 | |
Call database analysis | 4 | |
Call centre staff feedback | 3 | |
Informal client feedback | 1 |
INDICATOR TYPE: [U]= Universal, [R]= Best practice- highly recommended, [O]= Best practice- optional | ||||
---|---|---|---|---|
Indicator | Recommended data source | **Optimal suggested range | Satisfactory suggested range | Bare minimum suggested range |
Access
| ||||
A.1 [U] The number of calls received have increased by x% since y year | Call records | 21-30% | 11-20% | 0-10% |
A.2 [R] The percentage (%) of calls converted into appointment bookings have increased by x% since y year | Call records | 20-30% | 10-20% | 0-10% |
A.3 [R] The percentage (%) of bookings converted into attended appointments have increased by x% since y year | Call records | 31-40% | 21-30% | 0-20% |
A.4 [O] The percentage (%) of clients in MSI clinics referred by call centre has increased by x% since y year | Clinic records | 21-30% | 11-20% | 0-10% |
A.5 [O] The number of referrals made to other services has increased by x% since y year | Data analysis | 10-15% | 5-10% | 0-5% |
Equity | ||||
A.6 [U] The percentage (%) of potential clients who are young people1 has increased by x% since y year. | Call records | 10-15% | 5-10% | 0-5% |
E.1 [R] The percentage (%) of potential clients who are from [particular target group of interest] has increased by x% since y year | Call records | 10-15% | 5-10% | 0-5% |
E.2 [R] The percentage (%) of potential clients who live below the poverty line has increased by x% since y year | Call records | 10-15% | 5-10% | 0-5% |
E.3 [O] The percentage (%) of new potential users of family planning2 has increased by x% since y year | Call records | 10-15% | 5-10% | 0-5% |
Quality: Outputs
| ||||
Q.1 [U] x% of “mystery” client callers are satisfied with MSI call centre service | Mystery caller report | 81-95% | 66-80% | 50-65% |
Q.2 [R] x% of callers are satisfied with MSI call centre service | Client survey/ interviews | 81-95% | 66-80% | 50-65% |
Q.3 [O] x% of callers report to be enabled and encouraged to participate effectively in their own care or treatment | Client survey/ interviews | 81-95% | 66-80% | 50-65% |
Q.4 [R] x% of service providers are satisfied with MSI service delivery due to call centre | Provider survey/ focus groups | 81-95% | 66-80% | 50-65% |
Quality: Process
| ||||
Q.5 [U] % of calls answered within 15 seconds | Call records | 81-90% | 71-80% | 60-70% |
Q.6 [U] The average call abandoned per year is less than x% (standard is 3%) | Call records | 1-5% | 5-10% | 10-15% |
Q.7 [R] The average level of ‘call quality’ of call centre staff has increased to x% since y year | Call Quality tool | 81-95% | 66-80% | 50-65% |
Q.8 [O] The percentage (%) of call centre staff that report that training and resources are adequate has increased to x% since y year | Staff survey/ interviews | 81-95% | 66-80% | 50-65% |
F.1 [U] The average cost per call has reduced by x% since year y | Call and HR records | 10-15% | 5-10% | 0-5% |
-
Access: every potential client can easily reach or obtain MSI services regardless of financial, geographical and/or cultural barriers.
-
Equity: every potential client has an equal opportunity to obtain MSI services and products regardless of their socio-economic status.
-
Quality: All MSI services and products meet high clinical standards and quality of care that is client focused and responsive to client needs.
-
Efficiency: every MSI programme produces the maximum possible output from a given set of inputs, thus being cost effective and sustainable.
-
Universal indicators are vital to the functioning of all MSI call centres, and hence should be measured by all call centres with small to large scope and capacity. These indicators are all necessary, feasible or within a high priority group, identified by MSI.
-
Best practice - highly recommended indicators are strongly encouraged to be measured.
-
Best practice - optional indicators are encouraged to be measured, although not necessary for all call centres given the variation in services provided, and range in resources and capacity available to conduct call centre evaluations.
Pilot evaluation in Mexico
Response rate
Access
Indicator | 2010-11 | 2011-12 | % change | |
---|---|---|---|---|
N Calls answered | A1 | 59505 | 76522 | +22 % |
N (%) Calls converted into bookings | A2 | 17050 (29 %) | 21064 (28 %) | −1 % |
N Total appointments booked | 17389 | 21233 | ||
N (%) Appointments made by clinic directly | 339 (2 %) | 169 (1 %) | ||
N (%) Of total appointments made by call centre | A4 | 17050 (98 %) | 21064 (99 %) | +1 % |
N (%) appointments attended | A3 | 12392 (71 %) | 15728 (74 %) | +3 % |
% of call answered within 15 seconds | Q5 | 94 % | 92 % | |
% Abandoned calls | Q6 | 5.7 % | 5.0 % |
Equity
Quality
Client interview: “The call operator gave me a lot of useful information and I am very satisfied with it. He made me feel confident and safe. He gave me really good advice for whatever questions I had.”
Client interview: “The appointment and wait time were adequate. The operator was attentive. The information was useful. For example, they explained how the method of contraception worked and what it meant for my body.”
Mystery client report: “They were very kind, attentive and interested in helping. With a warm and caring voice they told me it was my decision to make and no one else’s.”
Mystery client report: “They were sensitive, as I can’t get out of work easily and they gave me other hours to access the services.”