FORMATIVE CLINICAL EVALUATION OF FIRST-YEAR STUDENTS IN FUNDAMENTAL NURSING SCIENCE

The outcome of any system of evaluation has a considerable impact on the daily lives of students in a variety of educational settings. Hence, it was attempted to illustrate the complexity of the problem •when evaluating the performance skills of first year students in the clinical environmenta of the general hospital. The choice of the research field originated from the increasing concern of the researcher about the reliability of the current evaluation practices in the formative assessment of first year student nurses. The impression gained is that nurse educators are more concerned with the end results of evaluation than with the teaching-learning process needed to reach this goal. Due to the many variables that can influence its results, the implementation of the evaluation process in clinical nursing is extremely complicated. In the course of studying the literature relevant to the research field, the researcher identified aspects that can be considered as of critical importance in the assessment of student performance in clinical nursing.


FORMATIVE CLINICAL EVALUATION OF FIRST-YEAR STUDENTS IN FUNDAMENTAL NURSING SCIENCE I . Wannenburg, W. J. Kotzé
"The purpose o f evaluation is not to prove but to improve" (M organ & Irby, 1978, p. 176) The outcome of any system of evaluation has a considerable impact on the daily lives of students in a variety of educational settings.Hence, it was attem pted to illustrate the com plexity of the problem • when evaluating the perform ance skills of first year students in the clinical environm enta of the general hospital.
The choice of the research field originated from the increasing concern of the researcher about the reliability of the current evaluation practices in the formative assessment of first year student nurses.The impression gained is that nurse educators are more concerned with the end results of evaluation than with the teaching-learning process needed to reach this goal.Due to the many variables that can influence its results, the im plem entation of the evaluation process in clinical nursing is extremely complicated.In the course of studying the literature relevant to the research field, the researcher identified aspects that can be considered as of critical im portance in the assessment of student perform ance in clinical nursing.

^i S S E N T I A L R E Q U IR E M E N TS FOR FO R M A TIV E EVALUATION Evaluation criteria
The following evaluation criteria have been ' identified as essential for both formative and summative evaluation: -criteria to measure the reliability of the measuring instrum ent -criteria to ensure safe patient care -criteria to facilitate the integration of theory and practice -criteria to assess student learning whilst mastering nursing skills

Variables
The variables with direct influence on the reliability of the outcome of evaluation have been found to be the following: -H um an behaviour, in principle is difficult to observe and evaluate, so that some degree of error should always be anticipated -Students' clinical experiences are obtained in a variety of clinical situations, so that a uniform system of individual student evaluations are complicated by factors such as a mix of patients, the dissimilarity of their illnesses, personalities and nursing needs -Evaluators and students, being part of a pluralistic society, will invariably interpret d ata according to their own culture, standards and values -Evaluation of a student's perform ance in the clinical environm ent is done by direct observation, which necessitates the setting of behavioural objectives to serve as criteria for measurement, irrespective of the measuring instrum ent utilized.

Safety standards
The inclusion of a patient in the teachinglearning milieu dem ands the inclusion of safety standards for the perform ance of nursing skills.
Integration of theory and practice Integration of theory and practice does not occur automatically.This may be facilitated by the nature of planning and utilization of learning content and opportunities in the learning environm ent, as well as by the integration of cognitive learning in the criteria of clinical measuring instruments.

Progress of student learning
Form ative evaluation focusses on the attainm ent of teaching and learning goals.Criteria, therefore, which reflects both development in skill learning and synthesis of knowledge, should be a requirem ent in all clinical perform ance measuring instruments.
Because Nursing Science is defined and accepted as a clinical health science, all the above-mentioned aspects are regarded to be of vital im portance in the training of student nurses.The assessment of perform ance in the clinical field must be a reliable reflection of a student's achievement, as the outcome of such evaluations will have a considerable influence on the student's position in the training program m e and h is/h er future career prospects.The data for the research was acquired by obtaining the relevant formative measuring instrum ents from all nursing schools after having received permission from the relevant controlling health authorities.
F or the assessment of the evaluation instrum ents a reliable approach had to be found.The researcher chose the approach of Battenfield as the appropriate one for this study because it encompasses all the criteria identified in the literature as of critical im portance in form ative clinical nursing evaluation.A checklist containing this au th o r's fourteen criteria was compiled.
In the first phase of the analysis the macro design of every evaluation instrum ent received from the research population was assessed by using these fourteen criteria.In the analysis each of the criteria was regarded as of equal importance.
Following this the instrum ents were assessed for their compliance with the requirem ents of the four m ajor critical formative criteria, namely: -to measure the reliability of the measuring instrum ent -to ensure safe patient care -to facilitate the integration of theory and practice -to assess student learning whilst m astering nursing skills.
The grouping of the fourteen criteria of Battenfield into the above four major critical areas was done by the researcher according to priorities identified in the literature studied.It was done in this way because no proof could be found of any other, more scientifically based, e.g. through statistical analyses, approach.Based on her experience in this regard the Curationis Vol. 12, Nos. 3 & 4, D ecem ber/D esem ber 1989 researcher recommends that a comprehensive study be done in which for example, factor analysis could be utilized to verify the most critical nursing areas for evaluation purposes.

F orm ulation of a hypothesis
The hypothesis form ulated for the research was as follows: "The form ative clinical measuring instrum ents utilized currently to assess first year student nurses in the clinical environm ent of the general hospital do not prom ote student learning".D ata analysis and recom m endations It must be emphasized that the whole research population came under discussion.The response from the relevant nursing schools was 86,3%.

N um ber of clinical measuring instrum ents in use
The total num ber of clinical measuring instrum ents obtained from the nursing schools was 254 and varied between 1 to 62 per school (com pare Figure 1).Only 36,8% of the nursing schools submitted ten or more clinical measuring instrum ents to assess perform ance skills.The criterionreferenced grading system was utilized by 73,7% of nursing schools.A checklist was the selected instrum ent utilized to assess the first year student nurse's perform ance skills in the clinical milieu.
A checklist basically records w hether a characteristic is present or absent or whether an action was taken or not, it requires a simple "yes-no" judgem ent.Checklists are useful when evaluating those perform ance skills that can be divided into a series of clearly defined, specific actions where appraisal is so rough that one is limited to only two options.

Recom m endations
The emphasis in form ative evaluation should be on guiding the student towards self-directed learning.By providing regular feedack, using clear, precise and measurable evaluation criteria, the student is encouraged to become more selfevaluative and actively involved in the learning process.The art of evaluation lies in not only stating that something is wrong, but indicating why it is wrong and, most im portant, which measures should be taken to correct the nursing actions.It is at this stage th at clearly stated objectives and measurable evaluation criteria support the evaluator in guiding the student constructively.A specific measuring instrum ent should be designed for every identified nursing skill in which the student has to prove her competency.This approach to formative evaluation will not only increase the reliability of the measuring instrum ent but also enhances learning.It can, therefore, be utilized by both the tu to r and the student to improve teaching and learning skills.

Incidence of individual evaluation criteria
The frequency of the evaluation criteria found in the 254 obtained measuring instrum ents were as follows (compare Figure 2).
Criterion 1: Frequency 1 Does the measuring instrum ent include clinical objectives th at reflect the curriculum and stage objectives?This criterion was only reflected in one of the measuring instrum ents, namely 74.This could possibly be a reason for the problems that exist with theory and practice integration in nursing education.The above finding and statem ent supports the concern found in the literature in this regard.

Recom m endation
It is suggested that programme, curriculum, classroom and stage objectives be included in the perform ance measuring instrum ents to facilitate the integration of theory and practice.
Criterion 2: Frequency 213 Does the measuring instrum ent contain items th at are attainable, clear and stated in behavioural terminology?This criterion was met by 83,9% of the presented measuring instrum ents, only 16% did not meet this criterion.

Recom m endation
It can be deduced from the above findings that most nursing schools are utilizing this criterion in the clinical measuring instruments.
Subjectively and vague expectations can be eliminated by stating precisely w hat is required, for example, "all" equipm ent , must be itemized, "correct" records must be specified, and so on.
Starting the statem ents with a verb and concentrating on perform ance actions will help the nurse educator to state test items that are attainable and easier to assess, for example: "Administers medications on tim e".
Criterion 3: Frequency 200 Does the measuring instrum ent contain items th at describe behaviours th at are measurable?This criterion is very closely associated with the previous criterion, because if the test item is not stated in behavioural term inology it cannot be measured, 78,7% of the m easuring instrum ents met this criterion.
The following was observed in the 21,3% of measuring instrum ents th at did not meet this criterion.It is, for example, impossible to measure abstract behaviour such as, "professional courtesy and com passion through verbal and non-verbal com m unication", "displays responsibility with regard to the level of understanding and language ability of the patient", and "dem onstrates knowledge and insight".

Recom m endation
The more specific the wording of the individual test items the more reliable the results.A verb th at is open to varied interpretations must be avoided.If it is used it must be accompanied by a description of the exact expected behaviour.Verbs such as know or understand are difficult to evaluate and should be excluded except if they are supplemented by action statements.Other examples of terms that should be avoided are those that are not observable, for example, enjoy, believe, grasp the significance of, respect, always and never (compare Battenfield, 1986, p. 19).
Criterion 4: Frequency 254 Does the measuring instrum ent appear • appropriate for the clinical setting in which it is used?
The highest frequency is obtained by this criterion which reflects the appropriateness of the designed measuring instrum ents for the hospital as clinical evaluation milieu.
No recom m endation is required for this criterion.
Criterion 5: Frequency 80 Does the measuring instrum ent relate classroom objectives to the perform ance skills?As indicated by the low frequency obtained, namely 31,5%, it is obvious that this criterion has not been achieved.

Recom m endation
It is recommended that classroom objectives be included in perform ance measuring instrum ents as it not only ^^n h a n c e s theory and practice integration ^^u t also excludes the danger of performing any psychom otor skill w ithout the necessary cognitive basis.(Battenfield, 1986, p. 27-28) Terminology such as "occasionally" and "seldom ", for example, should be supported with clear descriptions of the expected behaviour.
Criterion 8: Frequency 188 Does the m easuring instrum ent identify the critical elements, if any, that have to be met?This criterion was met by 74,0% of nursing schools.These nursing schools also clearly indicated in their instructions to the evaluator how these critical elements should be assessed in the final calculation for a pass-fail grade.However, 26,0% of nursing schools are not sure of the meaning of a critical element or how to distinguish it from steps th at have to be followed to learn a skill.

Recom m endation
With the addition of a patient into the teaching-learning situation the inclusion of safety standards for patient care should receive top priority.This is obtained by identifying the critical elements for each nursing skill.These statements do not tell how the task should be accomplished but focus on the behaviours that are essential for the patient's protection.These critical elements should when finalized, explicitly state the essential nursing behaviours that constitute acceptable practice and heed 100 percent accuracy to ensure safe patient care.The emphasis is not on the needs of the student but on the student's behaviour in relation to the patient's needs.
Critical elements are thus the observable and m andatory behaviours that collectively comprise the standard against which the student's com petence is measured.When one of them is violated or omitted the patient is actually or potentially endangered and nursing care is unsafe.The student therefore, fails the perform ance evaluation no m atter how well she has perform ed in any of the other criteria (Lenburg, 1979, p. 47-48).
Criterion 9: Frequency 184 Does the measuring instrum ent clearly describe the grading scale for determ ining a pass-fail grade?This criterion was achieved by 72,4% of measuring instruments.A final calculation for determ ining a pass-fail grade is obtained by integrating the m ark allocated for the theoretical and the perform ance evaluations.Confusion exists when, in the final evaluation, the theoretical knowledge is assessed by using a normative-referenced grading scale and the perform ance evaluation is assessed by using a criterionreferenced grading scale.Calculating this final pass-fail grade then becomes a very difficult task.
A nother problem encountered is com plicated weighting of numerical systems, for example: that of 3 test items to which 5 m arks have to be allocated.As the evaluator receives no instruction on how the m arks are to be divided between the items, the nature of the calculation is left to the discretion of the individual evaluator, resulting in subjectivity and thus, unreliability.
Recom m endation W hen form ulating a final judgem ent on the student's clinical perform ance the clinical measuring instrum ent should describe the grading scale for determining a pass-fail grade.It is suggested that the following be used as a guide: -identification of the total num ber of clinical objectives that must be obtained -compliance with critical elements -provision of a key for assigning numerical values and a form ula for calculating numerical ratings (compare Battenfield, 1986, p. 29).
Since com parative grading is not an issue in a criterion-referenced system, the student must pass, or dem onstrate competency in every required skill.The student cannot com pensate for failure on one skill by passing another.This problem is encountered when a numerical grading system is used.A student may, for example, obtain 80% for measuring a patient's blood pressure, 50% for bed bathing a patient and 20% for urinalysis.The average m ark for all these skills may then be calculated as 50%, but 50% of what?
Com plicated weighting and numerical systems should be avoided as this is confusing and reduces the reliability of the instrum ent.
Criterion 10: Frequency 93 Does the m easuring instrum ent provide a column for notation o f not applicable or not observed behaviour?Only 36,6% of m easuring instruments made provision for this criterion.

Recom m endation
A student cannot be judged on a clinical objective or on an individual test item not applicable to a situation or which the evaluator failed to observe.It is unfair and results in unreliable evaluation.
Criterion 11: Frequency 143 Does the measuring instrum ent provide the evaluator with a space for writing supporting statem ents and, if needed, a recommended plan for remediation?Only 56,3% of measuring instruments reflected this criterion.

Recom m endation
All measuring instrum ents should reflect this criterion as form ative evaluation focusses upon teaching-learning strategies.The evaluator, therefore, needs a space on the measuring instrum ent for writing supporting statem ents, comments and, if needed, a recommended plan for remediation.
Criterion 12: Frequency 0 Does the measuring instrum ent prom ote interrater reliability by providing a guide for writing supportive statem ents that are congruent with the evaluation rating scale?The incidence of this criterion in the measuring instrum ents was zero.

Recom m endation
It is suggested that this criterion be included, as the reliability of the measuring instrum ent will be enhanced by the writing of supporting statem ents, com ments and, if needed, a recommended plan for rem ediation (compare Battenfield, 1986, p. 38).
Criterion 13: Frequency 42 Does the measuring instrum ent provide students with a space for writing optional statements?The incidence of this criterion was rem arkably low, namely 16,5%.

Recom m endation
The student should be actively involved in the evaluation process and has the right to approve of, or disagree with what the evaluator has recorded, in writing on the measuring instrument.
Criterion 14: Frequency 53 Does the measuring instrum ent provide space for signature and date lines for both the evaluator and the student?This criterion was met by 20,9% of the measuring instruments.M ost measuring instrum ents made provision for the FIGURE 3: RELIABILITY signature and date lines for the evaluator but not for the student.This fact once again caused the researcher to question how actively the student is involved in her own learning process, or whether this inform ation remains in the possession of the evaluator only.

Recom m endation
If the student is to be actively involved in her own learning process she should also be actively involved in the evaluation process.The instrum ent should, to identify this, therefore also provide space for signature and date lines for student use.
Ratings obtained for reliability, safe patient care, facilitating theory and practice integration and progress in student learning When the instrum ents were assessed for compliance of the criteria with: reliability, safe patient care, facilitating integrating of theory and practice and progress in student learning, the following was found: A measure of the reliability as being measured by these five criteria, may be taken as: The second highest rating 59,8% was obtained for the criterion reliability.The reliability of measuring instrum ents can be increased by prom oting interrater relia bility, describing rating keys, form ulating behavioural objectives that are clear, precise and measurable, and constructing a measuring instrum ent for each nursing skill that has to be assessed.
The more specific the wording of the test items, rating items and instructions to evaluators, the more reliable the results.By minimizing the num ber of judgem ents required, for example by using "y es/n o " or "satisfactory/ unsatisfactory" response options, the more likely it will be that agreement am ong raters and, therefore, higher interrater reliability will be achieved.
The evaluator may want to elaborate on the observed behaviour by writing comments or supportive statements.These recordings may be to highlight positive behaviour or to clarify statements of negative ratings.
A list of descriptors to guide supple mentary docum entation of the observed behaviours can further enhance interrater reliability.For greater reliability adequate sampling of experiences, representing all desired behaviours, is preferred.W ith the variables inherent in clinical situations it is necessary to assess the student in a range of learning situations to ensure reliability of judgem ent.No student should be judged Guessing is inaccurate and subjective.To allocate a zero in an unfilled rating box in such an event is unfair to the student.Therefore, it is necessary to provide a means for notation of "not applicable" or "not observed" behaviour.
A measure of the safe patient care as being measured by these three criteria, may be taken as:

FIGURE 5: FACILITATING THEORY AND PRACTICE INTEGRATION
Safe patient care received the highest rating 69,8%, which indicates the concern of nurse educators for patient safety.It must in addition be emphasized that this is also the most im portant criterion in summative evaluation.Considering the low rating obtained by the other evaluation criteria, student learning is not the main concern in the form ative evaluation milieu.
A measure of the facilitation o f theory and practice integration as being measured by these two criteria, may be taken as: G reat concern has been expressed about the lack of integration between the theory and practice in nursing.The low incidence, 15,9%, in this study supports the concern expressed in the literature.The nurse educator plays an im portant role in facilitating the integration of theory and practice by structuring, organizing and presenting nursing curricula in such a m anner that retention and transfer of knowledge can occur in the student.This can be enhanced by including clinical objectives in the form ative measuring instruments that reflect the curriculum, stage and classroom objectives.
A measure of the progress in student learning as being measured by these four criteria, may be taken as:  The researcher failed to obtain the following inform ation from the nursing schools: -whether the assessment is docum ented in the student's presence -whether the student is supplied with a copy of the feedback -the role of the tutor in structuring, organizing and presenting nursing curricula in order to facilitate theory and practice integration -the method of dealing with a student's inadequate clinical performances -the num ber of perform ance evaluations done per student -the nursing skills which are regarded as critical for clinical nursing assessment in the first year of study.

FIN A L R E M A R K S
The researcher has come to the conclusion that the nursing education fraternity is to a large extent guilty of rushing through the process of evaluation -more concerned about its com pletion than its substance.M ost of the problem s encountered can be solved by thorough design of the measu ring instruments.It must be taken into consideration th at nursing schools are currently in the process of not only implementing a totally innovative basic nursing program m e, but also of developing clinical evaluation measuring instruments to appraise the outcom e of these programmes.Evaluation, as an essential part of any educational programme, needs to be continuous in a meaningful criterionreferenced instruction.If a student does not progress as expected, feedback should help to locate the source of reason for failure.It may be due to failure by the student, the tu to r may be at fault for using ineffective teaching methods or creating inadequate learning opportunities and thus causing problems related to teachinglearning strategies.
The lack of appropriate measuring instrum ents for evaluation results in clinical nursing perform ance assessments that are often subjective, vague, unreliable and meaningless.Until accurate and reliable assessment techniques to measure students' progress towards all learning objectives have been pursued, evaluation cannot be accepted and offered as a scientific process.
Evaluation acts as a tool to ensure that the standards of nursing care are m aintained and improved where necessary.The protection of the patient, student and tutor is, therefore, dependent upon the utilization of reliable assessment techniques and approaches in the clinical environment.
The nursing school is ultimately accountable to the South African Nursing Council for the standard of its curricula and the quality of nurse practitioners it supplies to society.The South African Nursing Council has the power to inspect these schools at regular intervals to ensure that students are trained at a professional and personal level and that the needs for health care of the consumer of the South African society are met.
In 1978 the N ational League for Nursing expressed concern regarding the subject of evaluation, "M aking decisions about anything in life is a complex and sometimes frustrating experience.Somewhere deep in the regions of our psyche lingers a fear that we might make the wrong decision.This is most true when we are dealing with decisions that affect other hum an beings.A nother unfortunate characteristic in decision making is that enough valid d ata are seldom available to assist in the process" (N ational League for Nursing, 1978, p. 63).
The researcher believes that the fear of subjective and unreliable outcomes in the utilization of clinical evaluation instrum ents will in future be eliminated i only to the extent that nurse educators succeed in introducing more scientifically based evaluation practices.

FIGURE 2 :
FIGURE 2: FREQUENCY OP USE OP EVALUATION CRITERIA sum o f the frequencies o f th e five c rite ria y iqo% n u m b er o f c rite ria in g ro u p

FIGURE
FIGURE 4: SAFE PATIENT CARE objective unless the evaluator has actually observed the behaviour.
sum o f th e frequencies o f the th ree criteria y 100% n u m b er o f c rite ria in g ro u p sum o f th e frequencies o f th e tw o c rite ria y ioq% n u m b er o f crite ria in g ro u p sum o f th e frequencies o f th e fo u r c rite ria ^ i 009j| n u m b er o f crite ria in g ro u p