Background
Determining or assessing need is seen as "perhaps the most critical part of program planning" [
1] (p. 74). A primary goal in determining need is to find the gaps between what currently exists and what could be, what is desired, or what is an established standard [
2,
3]. Results from a need determination provide a focus for programs and for intervention strategies [
4]. If a person is at or above what could be, is desired, or the established standard, there is no need; if they are below, there is need. There are a multitude of methods to determine client and community need.
This paper reports on a process evaluation study that assesses the difference between how paraprofessional support specialists were trained to assess need and how they actually assessed client need in the field. The support specialists were working with a multi-state fetal alcohol syndrome prevention project in the US. This multi-state group is hereafter referred to as the consortium. Both the perspectives of the individuals who developed and implemented the training and the perspectives of the support specialists who worked with high risk pregnant women were taken into account. Differences and similarities among states and between trainers and support specialists regarding determining client need were assessed.
Needs assessment techniques were chosen by state versus collectively across the three states. Representatives from each state met together to discuss potential needs assessment strategies and techniques. After this meeting, staff members from individual states convened with an awareness of many needs assessment options and selected the techniques that best fit the needs of their state. Therefore, each state was free to select the tools and techniques that were most appropriate for their population and settings.
Although there are a number of articles that discuss how to conduct needs assessments around various health topics and articles that discuss the results of needs analyses, we were not able to locate any articles that compare the differences between how service providers are trained to conduct needs assessments and how those assessments are actually carried out in the field. The results of this paper have implications for the multitude of health and human service programs that train providers in methods for conducting individual-level needs assessments.
As in many other programs, needs assessments for this program were used to target specific intervention strategies to meet identified needs. The research presented in this paper provides valuable process evaluation information that can be used to gain a deeper understanding of the activities and results of the intervention strategies. Quantitative outcome evaluation data can help to answer the question "did something change that was statistically significant as a result of the intervention?" For example, did the intervention change pregnant women's drinking behaviors or levels of social support, depression, or family functioning? Qualitative process evaluation data can help program staff understand why changes did or did not occur as shown in the quantitative analyses [
5‐
7]. Using a hypothetical example, qualitative data can help staff understand why social support and depression changed but family functioning and drinking behaviors did not. It is fairly common for intervention strategies aimed at changing multiple outcomes to have varying levels of success across outcomes.
Previous analyses of this program have examined two important process evaluation questions: (1) Did people needing help for alcohol abuse receive more time from the support specialists[
8]; and (2) What were the characteristics of pregnant women most at-risk for alcohol consumption[
9]. A forthcoming paper will answer the important outcome evaluation question of whether or not the intervention reduced alcohol abuse by high-risk pregnant women.
Methods
Question construction
Questions were initially developed with the purpose of gathering need information that was not being garnered through other consortium-based data collection methods. Questions were reviewed by members of the consortium and minor revisions were made based on the review. The questions were open-ended and semi-structured, allowing the respondents to share information without the restrictions of closed-ended or forced choice questions[
10]. That they were semi-structured (i.e. the same open-ended questions were asked of each set of respondents) allowed for comparisons among those individuals interviewed[
11,
12].
Data collection
Two groups of individuals were interviewed from each of the three states. For reasons of confidentiality, in this paper we will refer to the states as State A, State B, and State C. The first group consisted of individuals who were involved in developing and delivering support specialist training. Anywhere from one to three individuals were interviewed from each state. The second group was the support specialists themselves. There were two support specialists interviewed in State A, four in State B, and two in State C. This includes all of the support specialists except one specialist in State C who was hired immediately prior to data collection. The support specialists are paraprofessionals who have prior work and/or life experience that would enable them to successfully work with high-risk pregnant women in their communities.
Interviews were conducted via telephone in the Fall of 2002 with additional information gathered through follow-up e-mail correspondence and from the support specialist training and field work documents. The use of two different types of qualitative data, interviews and documents, strengthens this study by providing data from multiple sources[
13].
To gather information on support specialist training and practice in States A and C, telephone conversations and e-mail correspondence was conducted between the first author and staff. The first author and staff developed and conducted training and were involved in providing needs assessment support to the support specialists in those states. The first author was deeply involved in developing and conducting the support specialist training in State B. Most of the training information for State B came from her experience. In addition, State B hired outside consultants to help with the training. The primary consultant and state staff provided additional information.
Interviews with support specialists in all three states were conducted by a graduate student at Montana State University who was completing a Master's degree in counseling and had extensive open-ended interviewing experience. She received additional interviewing training from the first author.
The trainers/developers were asked how they trained their support specialists to determine client need. Follow-up probes were used to elicit detailed information on the training. For example, they were asked how the training was carried out, if role playing was used, and the duration of the training process. Next, they were asked what written material in the training curriculum pertained to determining client need. Finally, they were asked about how need determination happens in the field. Documents were obtained whenever possible to provide additional information related to the questions.
Support specialists were asked to describe the training and/or information they were given to help them decide what the needs were of the women they work with. They were specifically asked this question in the context of the training/workshops they received to help them prepare for their job. Probes were used to gather additional information. Probes included asking about written materials they received and how the training was carried out (e.g., role play, lecture). Examples of need were given to clarify the type of information desired. Next, support specialists were asked how they determined need when they were working directly with women. Probes were used to clarify if this was done alone or with others and when this happened in the process of working with clients. Detailed examples of determining need were asked for.
Finally, support specialists were asked about need determination differences between what they received from training compared with what happens in the field. Questions were asked to determine how they felt that their training was helpful and useful and to ascertain additional methods of need determination other than those provided for in their training.
Data analysis
Hand written or typed notes were taken during each telephone interview. Hand-written notes were later typed into a computer using word processing software. Notes were expanded on and made more complete immediately after the interview. The "period after an interview or observation is a critical time of reflection and elaboration"[
13]. Existing documents were reviewed for relevant content.
Inductive analysis was conducted based on methods described by Strauss and Corbin[
14,
15], Patton[
11,
12], and Bogdan and Bicklen[
16]. In inductive analysis, themes arise from the data versus being predetermined before analyses begin[
17]. The transcript was read, themes and specific answers to questions were extracted and example quotes were identified. To increase the validity of the analysis, member checking was performed. In member checking, data interpretation and results are tested against the perception of the respondents[
18]. "Validity in qualitative research has to do with description and explanation, and whether or not a given explanation fits a given description. In other words, is the explanation credible?"[
19] (p. 216). Results and interpretation of data were sent to interviewees for verification. Changes in text were made when necessitated.
Discussion
There are many programmatic similarities across the three consortium states. For example, all three states used support specialists, all three states gathered a core set of data on client determinants and support specialist activities, and all three states used common intervention activities. Although the three states differed on their exact definition of a support specialist, all states saw these staff members as women who were familiar with their communities and who had an ability to work closely with high-risk pregnant women. One of the consortium activities that was developed separately by each state was how client need for the intervention group would be determined. To understand how support specialists were trained to determine need and how need was determined in the field, semi-structured interviews were conducted with state staff and support specialists in three states.
In State A, state staff and support specialists stated that client needs were identified primarily through the assessment instruments. The developers of the ASI stated that "upon completion there is generally the sense that the interviewer has a realistic appraisal of the patient's status in each of these areas"[
21] (p. 421). These areas are medical condition, employment, drug use, alcohol use, illegal activity, family relations, and psychiatric condition. In State A, a great amount of time and care were taken to develop support plans specific to needs that arose from the assessment instruments. These support plans were sent back to the support specialists quickly and state staff took time to follow up on these plans with the support specialists. The support specialists believed that this system was effective and useful for them. In addition, state staff felt that motivational interviewing and case management information from the training assisted the support specialists with determining need. One support specialist mentioned motivational interviewing as a helpful adjunct for determining need.
In State B, state staff mentioned that support specialists completed the Difference Game, a card-sort game developed to identify client need from the viewpoint of the client. In addition, motivational interviewing assisted support specialists in their work. The Difference Game was completed early in the process and the needs that arose from the game were reevaluated over time. All support specialists emphasized that they assessed need from the viewpoint of their client, though one specialist did not mention the difference game. They all stated that they received support from other staff members and supervisors and that this was an effective method for determining need.
In State C, state staff stated three things that help in need determination. First was hiring support specialists who were familiar with their community. Second was the information from the assessment instruments. Third were educational materials. Support specialists who were intimately familiar with the community they work in, who were aware of women's needs and of resources to meet those needs were seen as the most important mechanism for accurate needs assessments. Although all three states worked to hire women from the community who would be able to relate to the clients, only state staff from State C mentioned this as a part of the needs assessment. State staff sent reports of assessment instruments with comments on areas of need back to support specialists. The two support specialists described conducting need determination in the field in different ways. One support specialist relied on information provided by the assessment instruments. The other specialist relied on past experience, knowledge of the community, need determination tools from past work, and information from referral sources to determine need. Both specialists stated that their methods seemed to work well for them.
The projects in States A and C had the support specialists collect data for the assessment instruments. These two states relied on their state staff to provide need information to their support specialists based on results from the assessment instruments. It was also understood that support specialists gathered an understanding of client need by conducting the ASI interview. State A's system for feeding back need data was much more structured than State C's.
Unlike States A and C, data from assessment instruments was not collected by support specialists in State B. In State B, the main use of the assessment instruments was to track change in clients over time. Completed assessment instruments were sent to the evaluation team which was off-site from the state staff. Some support specialists in State B received information from the ASI and other assessments from their supervisors, though this was not uniform across the four specialists or uniform over time within the specialists.
In State A, the need identification and support plan were directly linked with the common activities that were developed for the consortium. The common activities were in turn directly linked with the logic model and assessment instruments. For example, there was an assessment to measure social support and common activities aimed at increasing social support.
In State B, some of the Difference Game cards mirrored the information obtained in the assessment instruments. For example, three of that cards state that "it would make a difference if I had" drug or alcohol treatment, someone to talk to about the things that worry me, a real friend. These cards would be linked back to the ASI and the social support assessment. Many of the items in the difference game focus on primary needs – for example, food, housing, sleep, clothes, safety. The underlying assumption for the State B group was that by letting women choose their own needs they would see that the support specialists cared about them and this would enable them to be open to discussing their alcohol and/or drug use. The other assumption was that only when basic needs such as food and shelter are met can higher needs be attended to [
3].
In State C, there was a combination of need coming from the client and need coming from the assessment instruments. State staff felt that they hired women who were knowledgeable about needs in their community and this was combined with information from the assessment instruments. The two support specialists in this state used different methods for determining client need.
These data are limited by a number of factors. First, information on training was gathered months after the training occurred and answers may have been affected by recall bias. However, this method was necessary in order to gather insight into the differences between need assessment information received in training and how needs were assessed with clients in the field. Second, in general, it is better to gather survey data via in-person interviews versus telephone interviews. Data is thought to be of higher quality and interviews can be longer with in-person interviews. Fortunately, the interviewers had easy access to the respondents and could conduct follow-up interviews if new questions arose while filling in notes after the interviews or if questions came up in the data analysis phase. Lastly, qualitative data analysis is always impacted by the experiences and viewpoints of the people conducting the data analysis. To lessen this impact and increase validity, respondents read the portions of this manuscript that related to the information they provided and offered suggestions and changes.
Competing interests
The author(s) declare that they have no competing interests.
Authors' contributions
SC&LS conducted the interviews. All authors contributed to the study design, data analysis and drafting of the manuscript. All authors read and approved the final manuscript.