ACADEMIC NURSE LEADERS ’ INTERPRETATION OF CONCEPTS AND PRIORITIES RELATED TO THE EXAMINATION OF SCIENTIFIC SHORT PAPERS , DISSERTATIONS AND THESES-PART 1

The purpose o f this study was to establish whether there is agreement among academic nurse leaders in their interpretation of aspects and criteria related to the examination o f scientific short papers, dissertations and theses. The Delphi technique was selected as the most appropriate method o f data-collection for this type o f study. The target population identified for this study consisted o f the Heads of Nursing Departments or their delegates from the 18 universities o f the four provinces o f the RSA, self-governing and independent states o f Southem Africa and Namibia, which offer nursing degrees. Three rounds o f questionnaires were sent to all heads o f the 18 identified nursing departments. Participation was poor (nine participated in first round, nine in second the third round, no further rounds o f questionnaires were sent due to the waning interest andpoporparticipation. Many divergent responses were received to most o f the aspects included in the study. This indicates that with a few exceptions there is little or no agreement among academic nurse leaders in their interpretation of aspects and criteria relating to the examination o f post-graduate scientific or academic treatises.


INTRODUCTION
T h e w ritin g o f s c ie n tific p a p e rs , dissertations and theses (collectively called scientific or academic treatises) is probably one of the most crucial stages in th e acad em ic d e v e lo p m en t of any student.These scientific treatises serve as d o c u m e n ta tio n of th e s tu d e n ts thoroughness and ability to undertake scholarly em p irical w ork, and are requirements for the award of advanced (postgraduate) degrees, namely the masters or doctoral degree.However, before such degrees are awarded, these scientific treatises have to pass through a n o th e r c ru c ia l s ta g e , th a t of examination.This is the last phase in the successful or unsuccessful conclusion of the treatise, and of great importance both to the University offering the degree and to the student submitting the treatise.The University offering the advanced degrees assumes responsibility to the p u b lic to e s ta b lis h a n d m a in ta in excellence and high standards in the e d u c a tio n o f its s tu d e n ts a n d responsibility to the students to protect them from possible unfair bias of a single exam iner.T h e re is th u s com m on agreement that the student's work should be examined not only by one person but by a panel of examiners.It is the task of this panel to determine among others whether the student's work is presented in a scientifically accepted manner in accordance with the accepted practice of the specific discipline in which the candidate wishes to be awarded a degree.The panel norm ally consists of the s u p e rv is o r/p ro m o te r a n d two independent examiners, of whom one is external to the U niversity which is awarding the degree.
Examiners normally receive a list of guidelines or criteria for examination from the particular University where the d e g re e is to b e a w a rd e d .T h ese guidelines vary in specificity from very broad to very detailed, yet focus on similar aspects but nevertheless, reports in the lite ra tu re suggest th at these guidelines do not necessarily provide the intended outcom e (A rrangies & Du Plessis 1992, Landm an 1992, Vorster 1992).It has been claimed by Vorster (1992;91) that the examination is often based on feeling or impression; that some examiners appear to accept only their own views and reject any other views, in clu d in g m e th o d s , s tr a te g ie s or perspectives that do not fit within their own p e rs o n a l fra m e w o rk .
T he examination therefore, instead of being objective and unbiased and consistent with universal norms agreed upon by the scientific community, tends to become a personal and subjective matter, with each examiner expressing his/her own opinion about standards.Practical experience in the handling of examiners' reports as well as complaints raised by colleagues lend support to the fact that Vorster's claims are as applicable to nurse examiners as they are to o th er disciplines.For example, when there is a difference of 19,29 or even 40 percent in the marks allocated by different examiners for the same work, or when one examiner fails a student outright because she does not agree with the theoretical perspective used by the student, or with the language and style of the student, or because there are too many typing errors, but makes no reference in her report to any scientific merit, while the other examiner/s award a good mark, sometimes even distinction, and report/s on the scientific merit, it is obvious that these examiners are using different criteria and have different priorities.They are not guided by universal criteria which is a matter for concern.
In their textbook on nursing research.Burns & Grove (1987) maintain that although a list of criteria can serve as a useful guide for critiquing, it nevertheless requires a sophistication in research knowledge not yet acquired by the inexperienced.Conducting a critique is not a basic skill and requires thorough preparation; undoubtedly it requires shared meanings in the interpretation of concepts and prioritising of criteria.As W ittg e n s te in (1961) p o in te d o u t, "meaning is use".When a person uses a word he does so according to what he b elieves it to m ean.T h e re is no guarantee that the m eaning that he imputes corresponds to the meaning intended by the writer or to the meaning imputed by another.This suggests that examiners who have not been exposed to similar or sufficient learning experiences with regard to examining are likely to in te r p r e t c o n c e p ts and p rio ritie s differently from one another and also differently from the way the scientific community uses them.It is however of the utm ost im portance for both the student and the university, that everyone who deals with postgraduate nursing projects shares common values and understandings, or in other word that there is a meeting of the minds on issues regarding postgraduate examination.
This study was designed to establish w hether th ere is agreem ent among a c a d e m ic n u rse le a d e rs in th e ir interpretation of aspects and criteria r e la te d to th e e x a m in atio n of post-graduate projects.Academic nurse leaders constitute the pool from which the majority of external nurse examiners are nominated, hence the focus on them.
Specific aims of the study were -to establish: • how aspects and criteria pertaining to m asters and doctoral studies are defined by academic nurse leaders; • w h ich e v a lu a tio n c rite r ia a re c o n sid ered im portant and which unimportant; • what kind of evaluation instruments are preferred; • whether there is consensus among participants with regards to meanings of aspects and criteria pertaining to masters and doctoral studies; • the extent to which participants' interpretations are in agreement with universal scientific norms described in the literature.

METHOD
The Delphi technique was selected as the m ost a p p ro p ria te m eth o d of d a ta collection for this study.The Delphi technique according to Roberts & Burke (1989: 359) is a specialised type of survey in v o lv in g s e v e ra l ro u n d s of q u e s tio n n a ir e s fo r d e v e lo p in g a consensus among a group of experts on th e to p ic of in te rest.The D elphi technique has also been used in nursing to assess priorities, make predictions and to measure the judgement of a group of experts (Burns & Grove 1987, Polit & Hungler 1991, Treece & Treece 1986

PROCEDURE
The Delphi technique involves several rounds of questionnaires.For this study three rounds of questionnaires were sent to all heads of nursing departments of 18 id en tified U niversities.To ensure anonymity the South African Nursing A s s o c ia tio n (S .A .N .A ) a c te d as mediator.Questionnaires were sent out and replies were received bu the SANA w hich w ere th en forw arded to the researcher.It was not possible therefore fo r th e re s e a rc h e r to identify any respondents.
In the first round the selected panel members were asked to express their viewpoint about five given aspects related to examination of post-graduate work.
Firstly members were requested to define and differentiate between a scientific short paper (skripsie), a dissertation (verhandeling) and a thesis (proefskrif) and to indicate their interpretation of the purpose of each of the three study projects.This aspect was included because of differences in requirements for these documents.If examiners are not aware of these differences they may apply identical evaluation criteria to all of them or one examiner may be more rigid in evaluating a masters dissertation than another with a doctoral thesis.
With the second aspect members were r e q u e s te d to in d ic a te w hat they u n d e rs ta n d by o rig in a l re s e a rc h , contribution to knowledge and ability to do in d e p e n d e n t r e s e a rc h .M ost Universities specify that masters and doctoral work must show evidence of the latter two, while doctoral theses must also comply with the first of the above criteria.Examiners therefore must include this in their evaluation and should be clear on the accepted usage of these terms.The third aspect was included to identify examiners' preferences with regard to evaluation instruments.Examples of three types were given, namely a detailed one which indicates weights for each criteria, a broad one without weighting which allows considerable flexibility, a rating scale for each of the criteria, and a fourth option was included for another type.If the latter option was selected, it had to be described.Members were also requested to provide reasons for their choice of a particular instrument.
The fourth area of focus was a Ust of commonly used evaluation criteria.Members were asked to delete those that they considered unimportant, add those that they thought were missing and to explain what they considered should be included under each criteria.
The fifth and last aspect dealt with passes and failures.Members were requested to explain when they would expect a dissertation or thesis to be passworthy, worthy of distinction, requiring major revision or doomed for failure.
The first round of questionnaires was mailed in June 1990.A covering letter was included inviting the members to participate, explaining the purpose of the study and requesting them to return the responses within 30 days if possible.Four months later only nine members had participated.As the questionnaires were returned anonymously it was not possible to identify non-respondents.The nine q u e stio n n a ire s received, formed the basis of the questionnaires for the second round.The responses to the questions were sorted and the combined information of all the panel members was used to make up a new questionnaire.As panel members had been asked to define term s or express th e ir view points, responses were very divergent.Some were very vague and broad and others very detailed.Although some of the responses contained information that was similar, with very few exceptions they also d iffe re d on c e rta in p o in ts .Therefore with the exception of identical responses, all responses from each panel member were included in the second questionnaire.The responses were not edited but included in their original form.
In the second round, panel members had to consider the o th er p a rtic ip a n ts' responses as they responded to the new questionnaires.They were asked to indicate their agreement or otherwise w ith e a c h r e s p o n s e lis te d in th e questionnaire on a seven-point rating scale (1-7) with 1 = SD (strongly disagree) and 7 = SA (strongly agree).The second round of questionnaires was again sent to all 18 heads of university nursing departments in Southern Africa, and in a covering letter they were all invited to participate.Panel members who had not participated in the first round were invited to add their responses to the list if they did not agree with any of the views expressed so far.However, again, only 9 panel members completed and sent in their questionnaires and when no further responses had been received by August 1991, planning for the third round started.Questionnaires from the second round were coded, tallied and categorised.The number of responses to each scale point (1-7) for each response item were calculated, as was the middle range in which at least 50 percent of re s p o n s e s fell.
In e sse n c e th e questionnaires for the th ird round remained the same as for the second round, except for additional feed back.The number of responses to each scale p o in t for each resp o n se item was in d ic a te d in b ra c k e ts above th e scale-point, while the middle range was indicated by means of a square bracket.This is illustrated in the example below.
"Critical ability consists o f making critical assessments and the candidate must show that he/she is able to question and analyse other findings, ideas and concepts and draw hislher own conclusions.
Where responses to an item were too diffuse no middle range was indicated.Panel members were once more invited to review the other member responses and to evaluate or compare their own o p in io n s w ith th o se of th e o th e r members.They were asked to indicate their agreement or otherwise with each response item on the seven-point scale, and to provide a com m ent if their response fell outside the indicated middle range.
By the end of May 1992 only seven of the third round questionnaires had been received, of which one was blank and therefore not usable.Because of the waning interest and because it was not considered cost-effective nor meaningful to continue with a fourth round no further questionnaire was designed.Each one of the questionnaires received was first analysed separately, item for item, to estabhsh both the degree of agreement or disagreement with each response item and the priority rating given to particular response item in re la tio n to specific a sp ec ts.T he information obtained was then compared across all six questionnaires.A total of 31 items each with at least two different viewpoints, but more frequently seven or eig h t w ere co m p o se d F in ally th e meanings attributed to commonly agreed u p o n ite m s w ere c o m p a re d w ith d e s c r ip tio n s in th e l it e r a tu r e to determine the degree of agreement.The d a ta did not lend itse lf to fu rth e r statistical analysis and thus a narrative description is given of the results.

RESULTS
The results of each of the investigated aspects relating to postgraduate studies are presented in some detail.

Scientific short paper (skripsie)
A number of definitions of a scientific short paper were formulated by panel members during the first round.These are reflected below: (1) A pilot study.
(2) Part of an honours degree.
(3) Part of a master's degree.
(4) A paper to be presented or published.
(5) An empirical research report which is limited in terms of literature and sample.
(6) A paper which is limited in terms of its field of study length of paper and scope.
( (8) A scientific essay/assignment on a specific subject which is of more limited scope than a dissertation but provides evidence that the student is conversant with methods of research.
(Selected by two respondents in final round).From the given definitions, it can be d e d u c e d th a t th e r e is no s h a re d agreement among panel members on the meaning of a scientific short paper.The term was v iew ed fro m d iffe re n t p e rs p e c tiv e s an d w ith in d iffe re n t contexts and definitions varied from vague and broad to detailed on specific.V ery few r e s p o n d e n ts in c lu d e d properties or ch aracteristics of the phenomenon or indicated the differences between a scientific short paper and a dissertation or thesis.This suggests that the focus for evaluation or examination would differ among different examiners.The seventh definition listed, which in the final round was selected by three of the respondents to best reflect their view can be considered the most explicit and flexible of them all.It is more or less consistent w ith a defin itio n in the literature by Arrangies and Du Plessis (1992: 34) but to differentiate it better clearly from other scientific treatises, the first line could be amended as follows: "A scientific paper o f more limited scope and length than the dissertation".
Little agreem ent was found among panel members in their interpretation of the purpose of a scientific short paper.Responses such as "Is part o f a stmctured master's degree; serves as a pilot study for a m a s te r 's p ro je c t; is p a rt o f the requirements for honours level; to provide evidence that the candidate is conversant with the scientific method, and the marks contribute to the total marks for a master's degree; to provide evidence that the candidate is conversant with the scientific method and can express himself according to the norms o f a scholarly person^"\-to expose a student to research while it only counts 50% o f the total mark*'^\ to inform persuade convince or stimulate^'^'.
The three last statements (a), (b) and (c) above were selected by two, one and three respondents respectively as being the most appropriate interpretation of the p u rp o se of a s c ie n tific p a p e r according to their view.

Dissertation
The definitions of a dissertation as d e p ic te d below w ere not q u ite as divergent or varied as those of the scien tific s h o rt p a p e r but vary in vagueness or specificity.
(1) A scientific formal and more Curationis, Vol. 16, No. 3,1993 detailed enquiry than undertaken for a scientific paper, into a specific subject or phenomenon.
(2) A written document presented for e x a m in a tio n w hich p ro v id e s evidence in terms of language, style, documentation and argumentation that the student is conversant with the methods of research.
( (4) A comparatively elementary research paper of defined scope and limited length ( + 120 p. A4 paper 1 spacing).The criteria are: technical competence, evidence of adequate scholarly research, critical ability, lucidity and coherence.It should have a theme which is implicit in the t itle , a n d w hich is s u s ta in e d throughout.A dissertation is not r e q u ir e d to be an o rig in a l contribution to knowledge, though it should contribute to an insight into or understanding of its subject.(Selected by 3 respondents in final round).
(5) A dissertation is limited to 80-l(X) p a g e s an d m ust be b a se d on empirical research data.
The fourth definition in the above list appears to be the most specific and also the most congruent with a definition by Arrangies and Du Plessis (1992: 34).
Perhaps one needs to add that it is a requirement for a master's degree or qualify it as a m aster's dissertation; universities differ in their nomenclature and some talk of master's theses and doctoral dissertations.
The panel members' interpretations of the purpose of a dissertation were rather vague.Responses such as "submitted for a master's degree; to contribute 50% towards a masters degree; to give a valid and appropriate answer to a research question; to expand the body o f knowledge" and "to enable a candidate to acquire a master's degree and to develop research competence" were received.During the second round, 6 of the nine 9 panel members and during the final found all 6 participating members opted for the last mentioned response.
It doubtful whether these descriptions are explicit enough to provide guidance to the examiner.In a chapter on "The nature o f dissertations and theses", Botha (1992:12) lists a number of abilities which should be demonstrated by the student in a dissertation.In a nutshell these are more or less consistent with the criteria stated in the definition and perhaps would be the most appropriate to explain the purpose of a dissertation.

Thesis
Panel member's definitions of a thesis contained far more commonalities than eith e r those of scientific essays or d issertatio n s.T here was also less disagreem ent among m em bers as to which one of the definitions was the most suitable.In the final round only two responses were selected by members as most appropriately reflecting their view and these are presented below: (1) A doctoral thesis is a considerably m ore dem anding research paper than the master's dissertation and it must be "an original contribution to knowledge".T here is no strict limitation on length, but 250-300 pages (A4 1 spacing) would usually suffice.It should cover a wider field or at least show a greater sense of perspective than does the master's dissertation.C andidates should provide evidence of an awareness of basic theoretic problems directly or indirectly relevant to their particular topic and should reveal an extensive as well as an intensive knowledge of their subject.(Selected by 5 respondents in final round).
(2) A thesis is a research report which shows that the candidate can make a c o n trib u tio n to the building of science.It must show evidence of abstract thinking of a high level and of the research process itself at a m o re co m p le x lev el th an the dissertation.A national sample and an exhaustive literature survey is required.
Although there are some similarities among the two definitions, the first one is much more explicit and provides much clearer guidehnes to an examiner.The second one is problem atic in that it stipulates that a national sample is required.As far as could be established no university has such a requirement.In fact it would be an unrealistic and rigid requirem ent which might be neither cost-effective or feasible.As with the dissertation, the information constituting the first definition is consistent with that gleaned from th e lite ra tu re .(S ee Arrangies & Du Plessis 1992:34, Perreira 1984: 35 and Botha 1992:12).
As with the scientific short paper and the d is s e r ta tio n , p a n e l m e m b e rs ' interpretations of the purpose of a thesis were very vague.Responses such as "submitted for a doctoral degree; to enable candidates to acquire a doctoral degree; to build or test theory; to participate in theory building or to provide evidence o f the quality of the candidates scholarship, as well as extensive and intensive knowledge o f his! her subject and to enable him/her to acquire a doctoral degree were given".The last mentioned response was selected by all nine panel members of the second ro u n d a n d all six m em b e rs who participated in the final round as the most a p p r o p r ia te .O nly one m em b er suggested that "to provide evidence of theory building should be added".This may be problematic though, depending of course on how wide or how narrowly theory building is defined.It is also not an e x p lic it r e q u ir e m e n t of th e u n iv e rs itie s .O n th e w hole th e explanation a c c e p te d by the panel members is not sufficiently explicit and does not even contain all the criteria listed in the definition.Universities require that a doctoral thesis should show evidence of an original contribution to the knowledge and insight of the subject as well as evidence of indepth study and of mastery of research methods and scientific merit.These are aspects an examiner needs to look for in doctoral examinations.

Interpretations of the concepts original research, contribution to knowledge and independent research
Original research A variety of interpretations of the term original research w ere received and included "the firstlprimary research on a subject; the individuals own unique research o f a problem ; creative and systematic investigation; the researcher asks a question that has not been asked before; comes up with new facts which were previously not known; a product o f a researcher's unique creative ability; not duplicating previous research and finally originality lies in c o n trib u tio n to knowledge o f insight into/understanding of field, freshness o f approach or value o f a reassessment; secondary sources can be used, but a mere synthesis or survey of others' findings is unacceptable; stress is laid on personal approach and insight, as well as initiative".
Two of these responses could cause confusion when examining, namely "the firstlprimary research on a subject" and "the researcher asks a question that has not been asked before".A student may have researched a subject on which much research has already been done, but may have looked at it from a differen t perspective or used a different method.However, because it is not the first research on the subject, would the examiner who regards this essential, reject the thesis?Would it be fair?The last-listed explanation is the most explicit one among those Usted and contains most of the facets that were listed by the others.During the second round of the study five of the nine participating panel members had selected it as the most suitable and during the final round it was selected by five of the six p a rtic ip a tin g panel members as appropriate.However adherence to this explanation may also cause confusion.According to Botha (1992: 12) the majority of universities in South Africa do not require that the m aster's dissertation be an original contribution to knowledge, but it should at le a st c o n tr ib u te to w a rd s understanding of a specific problem in the field of research.One can thus infer that most universities do not regard c o n trib u tio n to in sig h t in to th e understanding of a problem as original.In view of this, these words may have to be deleted from the explanation.
One respondent selected the response "the researcher asks a question that has not been asked or answered before" as the most suitable in te rp re ta tio n of original research, already referred to as a p ro b le m a tic re sp o n se .F rom th e literature it appears that other disciplines too, are experiencing problems with this term.In an analysis of examiners' reports Landman (1993: 83) found that only 26,7% of examiners reported on this aspect and from the reports it was not clear what criteria were used when p o sitiv e r e fe re n c e was m ade to originality.The reports appeared to suggest that a candidate was said to have been original, if he wrote something or did something which is not normally found in theses.

Contribution to knowledge
The following interpretations the term contribution to knowledge were received from the panel members participating in the first round."Findings contribute to to ta lly new know ledge on subject; system a tic investigation to expand knowledge in discipline; researcher links her research to a conceptual/theoretical framework; knowledge is presented in such a manner that it takes on new meaning".Two responses were selected by the 6 pcuticipating panel members of the final round, three each -as best reflecting their interpretation of this term.These are reflected below.
(1) Knowledge is theory -therefore a contribution to theory means that the researcher links her research to a conceptual/theoretical framework.This can be at any stage of theory development: concept identification, identification of relationships or testing of propositions, etc.
(  (1992: 37) the following examples would all indicate a contribution to science.
• when new or improved evidence is p r e s e n te d fo r s u p p o rtin g or disproving existing concepts, theories and models; • when a new or improved methodology for research is furnished with regard to both the subject of investigation and the paradigm of its understanding; • when the subject and the topic are a n a ly se d by new o r im p ro v e d procedures which are derived from new paradigms of understanding and new procedures of investigation; • when new or improved concepts or theories are postulated on the topic.As the terms had to be considered within the contexts of post-graduate studies and s u p e rv is o rs /p ro m o te rs a re alw ays appointed for such study it would be impossible to give no support to the student.
The following two interpretations were accepted by the six participating panel members as the most appropriate ones for reflecting their viewpoint.The first was selected by five participants and the last one by one.
(1) Being able to implement the research process and complete the research study with the assistance of a supervisor.
(2) A level of performance whereby a perso n is capable of taking the in itia tiv e and resp o n sib ility of undertaking acceptable research either alone or as leader of a team of researchers.
It is difficult to determine how examiners would judge "ability to do independent research" on th e b a sis of th e se interpretations.What criteria need to be c o n sid e re d ?N one a re in d ic a te d .Landman (1993: 84) experienced similar problems in his analysis of examiners reports.Only 46% of his sample of examiners of dissertations and 28,3% of examiners of theses included comments on this aspect in their reports and no criteria could be identified from the comments made.Landman therefore raises the question whether the external exam iner is really in a position to comment on this aspect.Is it not really only the supervisor/promoter who will know to what extent the student designed and conducted the study independently?He adds, however, that certain content specific to independent research should be evaluated namely balance, rational action and accountability.
) A scientific paper of limited scope and length.(-1-60 p. A4 paper 1 spacing).The criteria are technical competence, critical ability, lucidity and coherence.It should provide evidence th a t th e c a n d id a te is conversant with the scientific method b u t n o t n e c e s s a rily e m p iric a l research.It is usually part of the requirement for obtaining a masters d e g re e in a c c o r d a n c e w ith prescribed university regulations.(Selected by three respondents in final round).

( 9 )
A paper which is systematically compiled after collection of objective in fo rm a tio n a n d p ro p e rly d o c u m e n te d u sin g re c o g n is e d reference techniques.It is short or long enough to be presented at a meeting, symposium conference or to be published in a scientific or professional journal.It is crucial that it meets the objective for which it was originally undertaken.(Selected by one respondent in flnal round).
Landman therefore recom m ended th a t clear guidance should be provided on what is wanted, e.g. have new re la tio n s h ip s b e e n e s ta b lis h e d or new fa c ts b e e n d isc o v e re d ?V o rs te r (1992: 95) describes "original" as the student's ability to come up with new and fresh ideas about possible solutions for a problem.