Democratic evaluation of clinical clerkship in a medical school. A case study
10.14601/TUTOR-16460
We report a case study of democratic evaluation of a clinical clerkship for medical students. We used a mixed model combining interviews and questionnaires to students, teachers and nurses involved in 9 weekly clerkship rotations in 7 hospital wards. The main outcome variable in quantitative analysis was the score of perceived usefulness for professional development of 42 Observable Practice Activities (OPAs) that each student should perform during clerkship. Scores were higher when the OPA was performed more consistently, was performed in small groups, and when the teaching and the evaluation methods were perceived as valid. The overall satisfaction for the program of teacher was high and did not correlate with students’ perceptions. The burden for patients was perceived as slightly higher in nurses, and did not correlate with teachers’ perceptions. When returned to students, teachers and nurses, these data could contribute to improve the program. 
application/pdf
1971-8551
4838
2
it
Mario Messina
Tutor
medical students
Text.Serial.Journal
https://ojs.unito.it/index.php/tutor
15
2015-08-03
2019-07-30
2015-08-03
21-28
1
Mario Messina
2015/08/03
10.14601/TUTOR-16460
21
1971-8551
2
medical students
28
https://ojs.unito.it/index.php/tutor/article/download/4838/4421
15
tutor                                                                                                   Vol. 15, N. 2, 2015: 21-28
                                                                                                      DOI: 10.14601/Tutor-16460



           Democratic evaluation
           of clinical clerkship in a medical
           school. A case study
                   Piersante Sestini, Rossella Angotti, Claudia Commisso, Mario Messina
                   Department of Medicine, Surgery and Neurosciences, University of Siena. E-mail: piersante.sestini@unisi.it

        ABSTRACT
        We report a case study of democratic evaluation of a clinical clerkship for medical students. We used a
        mixed model combining interviews and questionnaires to students, teachers and nurses involved in 9 weekly
        clerkship rotations in 7 hospital wards. The main outcome variable in quantitative analysis was the score of
        perceived usefulness for professional development of 42 Observable Practice Activities (OPAs) that each
        student should perform during clerkship. Scores were higher when the OPA was performed more consis-
        tently, was performed in small groups, and when the teaching and the evaluation methods were perceived
        as valid. The overall satisfaction for the program of teacher was high and did not correlate with students’
        perceptions. The burden for patients was perceived as slightly higher in nurses, and did not correlate with
        teachers’ perceptions. When returned to students, teachers and nurses, these data could contribute to
        improve the program.

        Key words: Democratic evaluation, medical students, clinical clerkship

        RIASSUNTO
        Contrariamente ai tipi di valutazione basati su standard normativi pre-determinati dalla istituzione stessa
        (valutazione burocratica) o da un corpo professionale esterno (autocratica), la valutazione democratica è un
        tentativo aperto, indipendente, non normativo (in quanto non vincolante per gli organizzatori né i parteci-
        panti) di dare voce a tutti gli attori coinvolti nel programma valutato. Per questo è più probabile che fornisca
        una descrizione del programma più dal punto di vista dei partecipanti che da quello delle aspettative degli
        organizzatori, fornendo così ad entrambi i gruppi suggerimenti utili per svilupparlo e migliorarlo. Abbiamo
        utilizzato questo approccio per valutare l’impatto della introduzione in un tirocinio clinico del CdL in Me-
        dicina di una lista di attività cliniche che gli studenti dovevano effettuare, documentandole su un apposito
        libretto. Si è costituito un gruppo di valutazione indipendente autorizzato dal Comitato Didattico, costituito
        da un docente, una specializzanda, ed un gruppo di studenti. Abbiamo utilizzato una metodologia mista,
        basata su questionari, domande aperte e interviste. Quale principale variabile quantitativa abbiamo scelto la
        percezione dell’utilità di ognuna delle attività svolte per il proprio sviluppo professionale. Alcune attività non
        venivano svolte regolarmente. Sono risultate effettuate il 75%, con un massimo di 100% in due reparti ad un
        minimo del 44%. Limitando l’analisi a queste, la maggior parte delle attività è stata valutata positivamente.
        Fattori che sono risultati influenzare positivamente i punteggi delle singole attività sono la divisione in piccoli
        gruppi (6 o meno), la validità delle modalità di presentazione e di valutazione (se previsto), la consistenza di
        effettuazione fra le varie rotazioni. Questi dati, restituiti ai docenti e agli studenti, possono essere utili per
        migliorare il programma.

        Parole chiave: Valutazione democratica, studenti di medicina, tirocinio clinico


                                                                                                                                21
           © Firenze University Press      ISSN (print) 1971-7296
           http://www.fupress.com/tutor   ISSN (online) 1971-8551
  tutor Volume 15, Numero 2, agosto 2015

                   INTRODUCTION                                              each student should document the performance of
LAVORI ORIGINALI




                   Evaluation has a central role in the design and de-       3-10 pre-determined clinical activities (Observable
                   velopment of medical education [1]. Of the three          Practice Activities, 6) in each ward. These activi-
                   main types of evaluation originally described by Mc-      ties, as well as the teaching methods, were free-
                   Donalds [2, 3], however, (bureaucratic, autocratic        ly selected by the faculties of each discipline, and
                   and democratic), the latter, despite being largely        should have been classified as “Having seen”, “Hav-
                   acknowledged in the world of general education, is        ing done” or “Having learnt to do”. Each activity
                   seldom used [4] or even mentioned [5] in the med-         had to be recorded by the students and signed by
                   ical education literature. Unlike the other types         the teacher on a clerkship booklet.
                   of evaluation, which are based on pre-determined          In an attempt to evaluate the impact of this change
                   normative standards, stated either by the educa-          and to identify methods to promote more effec-
                   tional institution itself (bureaucratic) or by an ex-     tive practical and patient-oriented teaching, an in-
                   ternal professional body (autocratic), democratic         dependent study group was formed, composed by
                   evaluation is a non-normative, independent, open          a teacher, a postgraduate student and a group of
                   attempt to give voice to all the actors affected by       students, all affiliated to the local branch of orga-
                   the educational program. As such, it is more like-        nizations interested in the study and promotion of
                   ly to provide a picture of the program as is seen         medical education (the Italian Society for Medical
                   from the participants, than as is expected to be          Pedagogy, SIPeM, and the Italian Secretariat of Med-
                   from organizers, and could provide to both groups         ical Student, SISM). The group was acknowledged
                   useful hints for development and improvement [2,          by the Education Committee (the local organism
                   3]. Here we report our experience of democratic           ruling the course) and was allowed to operate inde-
                   evaluation of a change in the organization of the         pendently. The aim of the group was to evaluate the
                   clinical clerkship for 4th year medical students at the   factors associated with better perceived usefulness
                   University of Siena.                                      of clinical activities taught to 4th year medical stu-
                                                                             dents during their first structured clinical clerkship,
                                                                             giving voice to all the actors of the process: stu-
                   SETTING                                                   dents, teachers and nurses, the latter used as proxy
                   Unlike other western countries, medical education         for patients involved in clinical teaching.
                   in Italy is still heavily based on a strong tradition
                   of lectures and unstructured oral exams, while
                   teaching of practical medical skills is generally un-     METHODS
                   rewarded by academic authorities and marginalized         As a conceptual guide for evaluation, the group se-
                   in curricula.                                             lected the principles of democratic evaluation [7]
                   In the School of Medicine of the University of Siena,     as described in an authoritative textbook [8], sub-
                   the first three years are currently almost complete-      stituting however the existentialist philosophical
                   ly dedicated to biomedical and preclinical matters,       underpinning that characterize that author with a
                   so that the 4th year clinical clerkship constitutes the   critical rationalistic background valuing Popperian
                   first main occasion of practical clinical learning for    concepts like truth as correspondence of proposi-
                   medical students. It consists in ten weekly rotations     tions with facts, democratic education as a means
                   of groups of 18-20 students through 7 different           to promote personal development rather than
                   wards: internal medicine (2 weeks), general sur-          selection of excellence, piecemeal engineering,
                   gery (2 weeks), cardiovascular diseases, respiratory      unintended effects of rational actions, and identi-
                   medicine, occupational medicine, haematology and          fication of failure as a tool to promote knowledge
                   endocrinology. In 2013, the Education Committee           and improvement [9]. Accordingly, a mixed method
                   of the School of Medicine at the University of Medi-      approach was devised combining open comments,
                   cine, to promote a more effective practical learning,     semi-structured interviews, and closed answer,
                   issued the requirement that during each rotation          quantitative questions [10].



                     22
                                                                      Democratic evaluation of clinical clerkship



At he end of the clerkship, in December 2013,              in subgroups, the method of assessment (if any),
the students were asked to complete a question-            and whether it was performed consistently across
naire asking, for each activity, whether they felt         the different groups/weeks.
to have effectively accomplished it, whether they          There were 42 activities (see appendix): 12 were
considered the method by which it was taught or            proposed as “having seen”, 2 as “having done”,
presented to be valid, how they felt it was useful         and 28 as “having learnt to do”. We re-classified
for their professional development (on a 10 point          them 4 as lessons, 5 as ward rounds, 8 watching
scale), and, for activities classified as “having learnt   a procedure, 9 case discussion, 11 procedures on
to do”, whether they considered valid the method           patients and 5 on peers. There was considerable
by which they had been assessed. Open comments             overlapping between the types of activity (e.g. case
were also invited. A digital form of the question-         discussions or procedures on patients where some-
naire and an open forum were also offered on ded-          times performed during ward rounds), and consid-
icated website built with the open source learning         erable variability from week to week (e,g. lesson or
platform Moodle (www.moodle.org) for students              case discussions in place of ward rounds etc.) was
who could not complete the paper version.                  reported for 13 activities. Twenty-eight were per-
In addition, a group of students was charged to con-       formed in groups of 6 students or less. Most of the
tact each teacher for a semi-structured interview          activities were assessed only informally with ques-
about the activities in which they had participated,       tions to the group, only 5 were formally assessed
their experience and feelings about the clerkship,         using an OSCE.
and the impact on patients and clinical activities.        A total of 146 students and 33 teachers participat-
They were also asked to rate, on a 10 point scale,         ed in the clerkship. Questionnaires were filled by
their overall satisfaction, their satisfaction about       92 students (63%), and interviews/questionnaires
the students’ participation, and their perception of       were obtained by 28 teachers (85%) and 10 nurses,
the burden of the clerkship program on the ward            representative of all the clinical settings involved.
routine and on patients.                                   Questionnaire data were transcribed in digital form
Furthermore, at least one nurse was interviewed in         using the software Epi Info 7 (CDC, Atlanta, GA),
each ward or outpatient clinic involved in the clerk-      and analysed using Stata 12 for Windows (Stata
ship, asking for comments on the impact of the ac-         corp, College Station, TX).
tivities on the care process and on patients. They         Activities with a score of 1 or less and all other
were also asked whether nursed had been directly           scores of zero or missing were considered as not
involved in the educational activities and to rate,        having been performed.
on a 10 point scale, their assessment about the stu-       Statistical analysis was performed using generalized
dents behaviour and their perception of the burden         linear models using Gaussian, binary or logit distri-
of the clerkship program on the ward routine and           butions as appropriate, including the student group
on patients.                                               as a random variable. The primary outcome vari-
As many students reported a poor correspondence            able was the perceived usefulness for professional
between the original classification of many of the         development of each activity. A score for each dis-
learning activities (watch, do, learn to do) and their     cipline was computed using the average of all the
actual content, based on teachers interviews and on        activities performed during the rotations in that
a focus group with students from several different         ward, or of all the teachers or nursed belonging to
clerkship groups, the activities were re-classified        each ward. Except where specified, data from ac-
in 5 mutually exclusive categories: lessons, ward          tivities considered as not performed were excluded
rounds, watching clinical procedures, performing           from analysis. Unless stated otherwise, quantitative
clinical procedures on patients, discussing clinical       data is presented as mean and 95% Confidence In-
cases and/or interpretation of reports, perform-           tervals. [11]. All P values are two tailed.
ing clinical procedures on peers. Furthermore, for         Open comments and interview reports were sub-
each activity was recorded whether was performed           jected to content analysis and similar concepts



                                                                                                            23
  tutor Volume 15, Numero 2, agosto 2015

                   were grouped by family resemblance [12]. As most               times) and procedures on patients were the ones
LAVORI ORIGINALI




                   comments were rather terse and repetitive, this                missed more often (performed 64% of the times),
                   process was relatively straightforward.                        with all the others ranging between 76 and 78% of
                   Respecting the requirement of democratic evalua-               the times.
                   tion for confidentiality, all the activities and wards         When the activities were performed, mean student
                   are presented in anonymous form.                               scores varied greatly, from 4 to over 8, with 64%
                                                                                  of them reaching a mean score of 6 or more (Fig-
                                                                                  ure 1). A strong correlation was observed between
                   RESULTS                                                        the percentage of performance and the mean score
                   Many students reported that, during their rotation,            when they were performed, with less performed
                   some of the activities were actually not performed.            activities usually, but not always, scoring lower than
                   The rate of non-performance varied between ac-                 the ones performed more consistently (Figure 2).
                   tivities and between wards, with some wards con-               In multivariable analysis, the percentage of per-
                   sistently performing virtually all activities during           formance, considering to have effectively reached
                   each rotation and others failing to do so in different         the aim of the activity, the appropriateness of the
                   measure, with one missing to perform more than                 teaching method and the fact that was performed in
                   than 50% of the activities (Table 1).                          small groups (6 or less) were all significantly associ-
                   Within the same ward, some activities were missed              ated to better scores, with the appropriateness of
                   more often than others.                                        the method having the greater effect (a difference
                   Procedures between peers were the ones per-                    of 2.9 points, 2.4-3.4), followed by effectiveness
                   formed most consistently (virtually 100% of the                (+1.7, 1.2-2.2) and small groups (+0.3 points, 0.1-



                                  Wards                           OPAs expected                    % Performed
                                  Ward #1                                 368                            96%
                                  Ward #2                                 368                            68%
                                  Ward #3                                 368                            71%
                                  Ward #4                                 644                            99%
                                  Ward #5                                 276                           100%
                                  Ward #6                                 920                            44%
                                  Ward #7                                 920                            78%
                                  Total                                   3,864                          75%

                   Table I. Number of OPAs expected and percentage effectively performed during students’ rotations in 7 wards



                                  Factor                                              Mean difference in score
                                  Performed in small groups (6 or less)                      +0.3 (0.1 - 0.5)
                                  Performed consistently among rotations                     +1.9 (2.1 - 2.6)
                                  I have effectively seen/performed/learnt                   +1.7 (1.2 - 2.3)
                                  The teaching method was valid                              +2.9 (2.4 - 3.4)
                                  The evaluation method was valid                           +1.9 (1.5 - 2.3)

                   Table 2. Factors significantly associated with an increased score of perceived value for professional development of
                   OPAs. Mean differences with 95% CI in parenthesis




                     24
                                                                     Democratic evaluation of clinical clerkship



                                                           Open comments by students were not particu-
                                                         larly frequent. Most of them showed appreciation
                                                         for the organization, the respect of schedule, the
                                                         attention received by the teachers or the use of
                                                         smaller groups in the wards that did so more con-
                                                         sistently. Also, peer activities were often praised. A
                                                         few complained of specific teachers or experiences.
                                                         Two long comments argued against the use of peer
                                                         activities at this stage of training.
                                                         In teacher questionnaires, mean score for satisfac-
                                                         tion about the clerkship rotations were on average
                                                         higher than student’s (7.4, 6.8-7.9) and were not
                                                         significantly correlated with student scores for the
Figure 1. Scores of individual OPAs performed in
different wards (same ward numbers as in Table 1)        same ward rotation. Satisfaction for the quality of
                                                         student participation was also high (7.9, 6.8-7.5).
                                                         The score for added burden to clinical activities was
                                                         variable (4.6, 3.6-5.7). Forty per cent mentioned in-
                                                         creased time, a few complained about the morning
                                                         timetable, when ward activities are busiest. The
                                                         score for perceived burden on patients was low
                                                         (2.7, 1.7-3.6). Five teachers reported that patients
                                                         were happy to participate. Three complained about
                                                         space restriction when students were present in
                                                         the ward or in outpatient clinics. One mentioned
                                                         uneasiness about privacy.
                                                         All the nurses reported that nurses were not di-
                                                         rectly involved in the teaching activity. They were
                                                         generally satisfied of students’ behaviour (8.9, 7.6-
                                                         10). One complained about some students chowing
Figure 2. Correlation between the mean score of each     or using mobile phones in the ward. Except in one
OPA and the percent of times it was performed during     case, perception about burden on care activities
rotations                                                was lower than teachers (3.9, 1.8-5.9), while per-
                                                         ception about burden on patients was slightly high-
0.5). After adjusting for these factors, ward rounds,    er (3.5, 1.6-5.4), mostly due to the presence of too
lessons, and performing procedures on patients all       many unknown people in the patients’ rooms, and
                                                         was uncorrelated with the score given by teachers
scored similarly, watching procedures and case dis-
                                                         (doctors) of the same ward.
cussions scored slightly better (+0.3 to +0.4 points
more, on average, p<0,01), and procedures on
peers scored the most (+0.5 points, 0.3-07). These       DISCUSSION
differences, however significant, were small.            According to MacDonald, Democratic evaluation is
In activities classified as “learning to do”, in which   an independent information service to the whole
the perception of validity of the assessment meth-       community about the characteristics of an educa-
od was asked, an affirmative answer was associated       tional programme [2, 3]. Its main activity is the col-
to a mean increase in score of +1.9 points (1.5-2.3).    lection of definitions of, and reactions to, the pro-
Assessment by OSCE further significantly increased       gramme. The key concepts of democratic evaluation
the mean score by +0.4 (0.1-0.7) points.                 are ‘confidentiality’, ‘negotiation’, and ‘accessibility’.



                                                                                                              25
  tutor Volume 15, Numero 2, agosto 2015

                   The justificatory concept is ‘the right to know’. We      selected the students perceived educational value
LAVORI ORIGINALI




                   believe that we fulfilled these strict requirements:      for personal development of professionalism as the
                   although the evaluator team was not completely            more relevant outcome measure of our inquiry.
                   independent, as one teacher and several students          The use of quantitative methods, allowed us to ex-
                   were actually participating in the program, none of       amine the relations between students perceptions
                   us was part of the Education Committee or was             and different aspects of the program activities,
                   involved in the design, in the management and con-        providing useful information to the actors involved
                   trol of the activities. Furthermore, the group was        in the development and in the participation in the
                   fully autonomous in the design and development of         program. Most of these data are only of interest
                   the inquiry. Our only aim was to contribute to the        within the local environment, but some could also
                   students and to the Education Committee informa-          have external validity for other settings with a sim-
                   tions on the actual performance and on the effects        ilar cultural and structural conditions.
                   of the program. Accordingly, we concentrated on           First of all, we documented a difficulty by some
                   the perceptions of students, teachers and health          of the clinical wards to cope with the demanding
                   workers affected by the program rather than on            requirements of organization, reliability and repro-
                   technical achievements, which were outside of the         ducibility dictated by the sustained rate of students’
                   scope of our inquiry. We guaranteed complete con-         rotations. Providing a reliable and reproducible
                   fidentiality and anonymity to all the parties involved,   set of clinical experiences to different groups of
                   both during data gathering and in our report. We          students for almost 12 consecutive weeks, during
                   negotiated with the students the contents, meth-          hours of maximal clinical activities, is a formidable
                   ods and timing of the investigation, ensured a wide       task, so the lack of performance by some wards
                   range of openness of themes and issues through            comes without surprise. Nevertheless, having doc-
                   the generous use of interviews and open com-              umented this fact may help the Education Commit-
                   ments in the questionnaires. Finally, we made our         tees and/or the ward heads and staff to refine their
                   report fully accessible and open to comments on           performance by increasing organization and/or hu-
                   our website. Although the report contains some            man resources dedicated at the program, or reduc-
                   suggestions, as they have been contributed through        ing the number of expected activities to these that
                   the questionnaires or are implicit from the data, it      can be reliably performed. In either case, students
                   does not present recommendations, leaving to the          would take advantage of a better correspondence
                   Education Committee, teachers, and students full          between propositions and facts (that is, between
                   freedom and responsibility for identifying and tak-       what they could expect from what is written in the
                   ing actions, if they want to.                             program and what is actually performed).
                   Unlike the existentialist approach endorsed by the        Not surprisingly, activities that were not performed
                   main experts and practitioners of democratic eval-        were rated very poorly by the students. When we
                   uation [8], we adopted a critic rationalistic (Pop-       excluded these activities, we still found a correla-
                   perian) approach, which was more familiar to us,          tion between some characteristics of the clinical
                   to the faculty and to medical students. This led us       activities and their impact on the perception of ed-
                   naturally to the use of mixed methods of inquiry          ucational usefulness experienced by the students.
                   (10), combining quantitative methods (mostly Likert       Of course, in some cases this was influenced by the
                   scales in questionnaires) with qualitative analysis of    intrinsic interest of the discipline (say, cardiology is
                   open comments. We found these to be rather con-           probably more popular among students than occu-
                   cise and repetitive, so it was not difficult to analyse   pational diseases, although both are equally import-
                   them without the use of specific software and tech-       ant in their education). Clinical activities with peers
                   niques. Of course, a fully grounded theory or phe-        ranked particularly high among students, but they
                   nomenological study would have required different         carried the advantage of novelty (this was the first
                   methods and expertise for data gathering and anal-        time that they were inserted in the curriculum) and
                   ysis, but that was not the aim of our work. We            were only carried by two wards with high intrinsic



                     26
                                                                           Democratic evaluation of clinical clerkship



interest and that also ranked high for reliability of       In conclusion, with this case study limited to a spe-
organization. However, even when the types of ac-           cific course activity, we provide an example of the
tivities were more evenly distributed among differ-         feasibility and possible usefulness of democratic
ent rotations, some differences emerged: in partic-         evaluation in medical education. This observation
ular, smaller groups, perceived effectiveness of the        is in contrast with the scarcity of references to
teaching methods and the perceived validity of the          democratic evaluation in medical education [4, 5],
evaluation all were associated to higher scores of          suggesting that the barriers to the diffusion of this
professional value by the students. Evaluation using        type of evaluation in the education of health profes-
OSCE rather better than the other methods (usual-           sionals should be identified and its wider use should
ly informal questions to the group, in a typical ward       be promoted.
round style). However, OSCE was used by only one
ward, that also had an overall high score, and this
effect could not be adjusted for other factors in the       REFERENCES
analysis. These informations could be useful to the         [1]    Frye A.W., Hemmer P.A. Program evaluation mod-
organizers to split some of the activities in smaller              els and related theories: AMEE guide no. 67. Medical
groups, when feasible, or to develop better meth-                  Teacher, 2012, 34: 288-299.
ods for teaching and evaluation, and possibly for the       [2]    MacDonald B. Evaluation and the Control of Educa-
Education Committee for organizing a program for                   tion. In: Murphy R., Torrance H. (eds.). Evaluating Ed-
faculty development focusing on these issues.                      ucation: Issues and Methods: an Open University Reader.
One of the characteristics of democratic evaluation                New York: Harper & Row, 1987.
is to attempt to give voice to all the subjects affect-     [3]    MacDonald B.A Political classification of evaluation
ed by the program, therefore we also interviewed                   studies in education. In: Hammersley M. (ed.). Social
                                                                   Research: Philosophy, Politics and Practice. New York:
the teachers and at least one nurse for each ward.
                                                                   Sage Publications, 1993: 105-108.
Unfortunately, we had not the resources needed to
                                                            [4]    Guilbert J.J. Comparison of the opinion of students
interview the patients, so we used these two sourc-                and teachers concerning medical education pro-
es also as proxies for patients involvement. Teachers’             grammes in Switzerland. Med Educ. 1998, 32: 65-69.
satisfaction was usually high, both about the program       [5]    Laluna M.C., Ferraz C.A. The meanings of evaluation
and about the quality students participation, but was              practice in nursing education. Rev Lat Am Enfermagem.
not correlated with students scores, suggesting a                  2009, 17: 21-27.
lack of communication and understanding between             [6]    Warm E.J., Mathis B.R., Held J.D., Pai S. et al. Entrust-
teachers and students. Difficulties about the impact of            ment and mapping of observable practice activities
the presence of high number of students and on the                 for resident assessment. J Gen Intern Med. 2014, 29:
workload imposed by the many rotations were fre-                   1177-1182.
quently reported, but overall the impact of the pro-        [7]    MacDonald B., Walker, R. Case-study an the Social
                                                                   Philosophy of Educational Research. Cambridge Journal
gram on clinical activities appeared to be limited and
                                                                   of Education, 1975, 5: 2-11.
sustainable by most of them. None of the nurses was
                                                            [8]    Kushner S. Personalizing Evaluation. New York: Sage
directly involved in the teaching activity (reflecting a           Publications, 2000.
low level of inter-professional education in our institu-   [9]    Popper K.R. The Open Society and Its Enemies. Londo:,
tion). Overall, the impact of the program on patients              Routlege, 1945.
was considered to be limited. However, the lack of          [10]   Maudsley G. Mixing it but not mixed-up: mixed meth-
correlation between the perception of the impact of                ods research in medical education (a critical narrative
the program on patients between doctors and nurses                 review). Med Teach, 2011, 33: 92-104.
of the same ward raises doubts on the reliability of        [11]   Cumming G. Understanding The New Statistics: Effect
these sources as proxies for patients. Nevertheless,               Sizes, Confidence Intervals, and Meta-Analysis. London:
the combined views of teachers, nurses and students                Routlege, 2012.
could provide useful ground to promote a dialogue           [12]   Wittgenstein L. Philosophical Investigations. Oxford:
                                                                   Basil & Blackwell, 1953.
between these groups and to improve the program.



                                                                                                                      27
  tutor Volume 15, Numero 2, agosto 2015

                   APPENDIX. OBSERVABLE PRACTICE ACTIVITIES
LAVORI ORIGINALI




                   Perform EKG on peers
                   Assist to a cardiology visit
                   Perform an echocardiogram
                   Collect a cardiologic anamnesis
                   Collect an endocrinologic anamnesis
                   Interpretation of the main tests and imaging of hormonal funcion
                   Interpretation of laboratory data for diabetic and obese patients
                   Evaluation of calciotropic hormones
                   Assist to the activities of the occupational respiratory function laboratory
                   Collect an occupational anamnesis
                   Know the legal implications of occupational diseases
                   Participate in the ward round activities
                   Assist to the measurement of PFT
                   Assist to allergy skin tests
                   Interpretation of the PFT
                   Be able to use a pulse oxymeter
                   Perform spirometry on peers
                   Perform walking test on peers
                   Perform chest visit on peers
                   Give supporting indications for red blood cell and platelet transfusion
                   Read a normal blood smear
                   Physical examination of the spleen
                   Surgical suture
                   Procedure for hand washing and preparation of the outpatient surgical field
                   Watch rectal examination
                   Watch insertion of bladder catheter
                   Watch nasogastric intubation
                   Care of surgical wound
                   Physical examination of the abdomen
                   Diagnosis of the main diseases of the abdomen
                   Write the clinical record
                   Approach to patients and relatives at the hospital admission
                   Diagnosis of diseases presenting at the internal medicine ward
                   Physical examination of the chest and evaluation of the respiratory patient
                   Measure and interpretation of blood pressure
                   Investigations for the patient with respiratory problems
                   Investigations for the patient with cardiovascular problems
                   Physical examination of the heart in patients with cardiovascular problems
                   Syptoms and signs of heart failure
                   Measure and interpretation of central and peripheral cardiac frequency
                   Interpretation of chest pain
                   Measure and interpretation of respiratory rate




                     28
