Student feedback about the use of literature excerpts in Sparshanam, a Medical Humanities module

Medical humanities (MH) modules have been conducted for first year students at KIST Medical College, Lalitpur, Nepal for the last four years. Literature excerpts are widely used in MH programs in developed nations. In Nepal English language literature excerpts had been used previously in two modules. Problems noted were difficulty in comprehending the excerpts and relating them to the Nepalese scenario. The MH module for the 2011 intake was conducted from December 2011 to March 2012. The present study was conducted in the third week of March to obtain student perceptions about use of literature excerpts and suggestions for further improvement using a questionnaire. Literature excerpts used in the module dealt with Nepal and health-related topics. Sixty-eight of the 80 students (85%) participated in the study. The majority were male, self-financing and from urban areas. Respondents felt the excerpts introduced them to different aspects of the medical profession, prepared them for future practice, and underscored the importance of understanding the patients’ feelings. The literature excerpts with which they could identify the most and the least were noted. There were no differences in median enjoyment and effectiveness scores of the literature excerpts according to subgroups of respondents. The suggested benefits of using literature in medical education were similar to those reported previously. Most respondents were able to appreciate the English language excerpts. They felt that Nepali language excerpts and those by Nepali writers could also be included. The findings would be of interest to educators in other developing nations introducing MH modules.

Introduction
KIST Medical College (KISTMC) is a private medical school in the Lalitpur district of the Kathmandu valley affiliated to the Institute of Medicine, Tribhuvan University for the undergraduate medical (MBBS) course. The college admitted its first batch of students in November 2008 and four batches of students have been admitted since. A medical humanities (MH) module has been conducted for first year medical students right from the first batch 1 using small group, activity-based learning methods. Case scenarios, role plays, facilitator presentations and paintings are among the different modalities used to explore various aspects of MH 2,3 .
The first author had used literature excerpts during a voluntary module for medical students and faculty members at the Manipal College of Medical Sciences (MCOMS), Pokhara, Nepal 4 . Literature excerpts were also used during a module conducted for faculty members and medical and dental officers at KISTMC 5 . Problems were noted with the use of literature excerpts. The excerpts were in English and were mainly obtained from the book 'Ten years of the medicine and the arts' 6 published by the Association of American Medical Colleges. Participants felt the excerpts did not reflect the Nepalese scenario, the language used was difficult to understand and they could not relate to certain excerpts.
Literature has been widely used in MH programs especially in developed nations. At the Northwestern Medical School in Chicago, United States (US) literature is taught in courses from the first day of medical school through residency 7 . Literature has been used in the MH program at the University of Missouri-Kansas City School of Medicine in the US 8 and during a family medicine clerkship at the University of California, Irvine College of Medicine 9 . In Asia literature from both Western and Arabic contexts has been widely used at the Weill Cornell Medical College in Qatar 10 .
Literature has a number of advantages in the education of doctors. It develops skills in observation and interpretation, develops clinical imagination, and promotes clarity of expression 7 . Literature can foster tolerance for uncertainty inherent in clinical practice, and increase empathy towards patients. A review states that five broad goals are met by the study of literature in medicine. Literary accounts can teach medical students concrete and powerful lessons about the lives of sick people 11 . Physicians are able to better understand and empathize with narrative stories of patients, literature increases physicians' expertise in narrative ethics and literary theory can offer new perspectives on the work and genres of medicine.
The MH module at KISTMC is named as 'Sparshanam' meaning 'touch' in Sanskrit, an ancient South Asian language. The 80 students of the 2011 intake were divided into six groups each consisting of 14 or 15 students. The module was conducted for 90 minutes every Thursday during the early clinical exposure for first year students. Eight sessions were conducted from December 2011 to March 2012. The topics covered during the eight sessions were empathy, what it means to be sick in Nepal, the patient, the doctor, the doctor-patient relationship, the family, the medical student and a wrap up session. Literature excerpts were reintroduced in this module and were used during certain sessions. Keeping in mind previous feedback we used excerpts with relatively simple language and which were directly related to Nepal. The excerpts were by western authors and in the English language. Further information about the excerpts used is provided in the Methods section. The present study was conducted in the third week of March 2012 with the following objectives: a. Obtain feedback on the literature excerpts used and b. Get suggestions for further improvement.

Methods
The study was conducted among first year undergraduate medical (MBBS) students in the third week of March 2012 on conclusion of the Medical Humanities module. The aims and objectives of the study were explained to the students and they were invited to participate. Written informed consent was obtained from all participants. The study was approved by the Institutional Review Board of KIST Medical College.
Literature excerpts were used during certain sessions of the module. Among the books used were 'The snow leopard' by Peter Matthiessen 12 , 'Three men in a boat' by Jerome K Jerome 13 , 'Window on to Annapurna' by Joe Stephens 14 , 'Aama in America' by Broughton Coburn 15 and 'The tennis partner' by Abraham Verghese 16 . 'The snow leopard' is a book about the author's trek in the remote Dolpo district of Nepal in 1973. He was trekking with a naturalist who was studying mountain goats and the snow leopard. The excerpt used was towards the end of the book where the author was returning back and his Sherpa falls sick. He is treated by a Japanese doctor from a mountaineering expedition. 'Three men in a boat' is a humorous account of a boat journey undertaken by three friends on the Thames River. The excerpt deals with the author reading a medical textbook and beginning to imagine that he suffers from the different diseases enumerated. 'Window on to Annapurna' is a story of a woman and her husband staying for a year in a Magar village outside the town of Baglung in western Nepal and how they become a part of the life of the villagers. The excerpt used dealt with different faith healing practices which the author had witnessed. 'Aama in America' is the story of a Peace Corps worker who stays in a remote village and is adopted by an old Gurung lady as her foster son. Later he takes the old lady to America and shows her his country. The excerpt used describes the lady's fear that she might die in America away from her family and also describes Gurung funeral rites. 'The tennis partner' describes a young Australian student who joins the author's medical school and who is fighting a losing battle against drug abuse. The excerpts deal with doctors from India emigrating to America and about doctors and drug abuse. The student groups were asked to interpret the excerpt in the context of the topic of the day's session and present their interpretation to the house using flip charts.
Sixty-eight of the 80 students (85%) participated in the study. Student responses were collected using a questionnaire. Basic information about the respondents like gender, method of financing of medical education, medium of instruction at school, place of family residence and occupation of parents were noted. The questionnaire used is shown below. Free text comments were collected about participants' perception regarding the use of literature excerpts in Sparshanam, how literature excerpts helped in realizing the objectives of the module, participants' opinion regarding the use of excerpts mainly by western writers, which of the different excerpts they could identify with the most and the least. Opinion was also sought on what they would suggest to make the exercise more useful, whether they faced difficulties in putting the excerpts in a Nepalese context and if yes, how they overcame these. Similar responses were noted and the frequency of different responses worked out. Participants were asked to rate on a scale of 1 to 5 their enjoyment of the literature excerpts used and their perception regarding the effectiveness of the excerpts. The mean and median scores were calculated. One sample Kolmogorov-Smirnoff test was used to test the normality of distribution of the scores. Both were found not to be normally distributed and non-parametric tests (Mann Whitney test was used for variables with two subgroups and Kruskal Wallis test for the others) were used to compare the median scores among subgroups of respondents.

Results
Sixty-eight of the 80 students (85%) participated in the study. Table 1 shows the demographic characteristics of the student respondents.
The majority of students were male, self-financing and from urban areas. Only a few students had parents from health related fields. Table 2 shows the respondents' comments about the use of literature excerpts in the module along with the frequency of the comments. Other interesting comments in addition to those mentioned in the table were "excerpts introduced us to different ways of thinking, and gave us ideas about right and wrong". Ten students (14.7%) stated they had previous experience with use of literature for educational purposes. Among the books mentioned were 'Diary of Anne Frank' and 'Animal farm'. Regarding how the literature excerpts helped in realizing the objectives of the module, 11 respondents (16.2%) stated it helped by preparing doctors for future practice, 10 (14.7%) said it helped them understand patient's feelings, eight (11.8%) stated it helped them by highlighting the reality of various issues discussed during the module, while five respondents (7.3%) wrote it helped them in visualizing different situations covered during the MH module.
Only six respondents (8.8%) were aware of the use of literature in medical education elsewhere. They mainly mentioned the examples of medical schools in western nations without giving specific names. Forty-eight respondents (70.6%) felt the use of literature excerpts by western authors in the module were appropriate. A respondent mentioned, "Nepal is also rapidly developing. Being in a poor country does not mean we should be away from the modern drama of medical development. And apart from that, we are influenced predominantly by the western world". Among the reasons mentioned were the excerpts widened their horizons, they could learn from the western writers, the writers mainly described their experiences in Nepal, they showed cultural aspects of illness, students have been exposed to western literature since childhood, and this type of literature is more easily available. Twelve (17.6%) felt this was not appropriate. Among reasons mentioned were Nepali excerpts were also available, the local context was always better and the excerpts dealt with a different set of problems. Table 3 shows the literature excerpts with which respondents identified the most along with reasons. A student wrote, "Three men  . It was also suggested by a respondent that the same literature excerpt could be provided to all the groups.
The median score of respondents with regard to enjoyment of the literature excerpts used and perceived effectiveness of the excerpts used were 4 (maximum score 5). Table 4 shows the median scores according to demographic characteristics of respondents. There were no significant differences in scores according to respondents' characteristics. Respondents were asked whether they experienced any difficulty in putting the excerpts in the Nepalese context; 40 (58.8%) stated they faced no difficulties while 21 (30.9%) stated they had difficulties. Among the strategies used to overcome the difficulties were group work and consultation, using their imagination and putting themselves in the scenario, and taking help from the facilitators. Fifty-five respondents felt excerpts from works by Nepali authors and in the Nepali language should also be used. A student wrote, "Yes, it would be a great idea. The modules will be even more interesting and we can relate well. But excerpts from other nations should also be included. This will provide an overall vision on situations". They felt excerpts related to medicine and excerpts from rural areas should be used. The excerpts can be obtained from different sources, and a few respondents suggested specific books that can be used while some felt students can themselves supply the excerpts. A large majority [62 respondents (91.2%)] felt literature excerpts should be continued to be used in future modules. Table 5 shows the suggested advantages of literature in medical education. A student stated, "Medical Studies are often facts, facts and only facts. We lack situations where we can acquire knowledge critically. Thus this can help improve critical thinking skills". Respondents were asked whether they felt the use of English language excerpts in the module was appropriate. Fifty-one respondents (75%) felt they were appropriate. Among the reasons mentioned were English is the international language, the students were from English medium schools and were comfortable with English, the excerpts dealt with Nepal and English is the language of instruction. Eleven (16.2%) felt the excerpts were inappropriate. Cited reasons were they do not describe the Nepalese situation, Nepalese excerpts are available and English is not a commonly used language in Nepal. Forty-one respondents (60.3%) were easily able to understand the language used in the excerpts while 25 (36.8%) were not able to do so. Difficult words, tough language and low level of mastery of the English language were cited as reasons for difficulties.
The facilitators provided a brief introduction about the book and the author before distributing the excerpts. Fifty-eight respondents (85.3%) found this useful. The introduction often helped to put the excerpt in context, students knew more about the author and they could read the book if they were interested. Suggestions to further improve the use of literature excerpts were also obtained. Common among these were to also use Nepali language excerpts (16 respondents), use more excerpts (8), use excerpts with simpler language (6), use more excerpts from the medical field (5), provide photocopies of the excerpts before hand to the students (4), and provide more time for the activity (4 respondents).

Discussion
Participant feedback about the use of literature excerpts during the module was on the whole positive. They felt the excerpts used were not too difficult to understand and either reflected or could be extrapolated to the Nepalese scenario. They also wanted more scenar-ios by Nepalese authors and in the Nepal language to be used. They were strongly in favor of continuing the use of literature excerpts during future MH modules.
Most of the students are self-financing and only seven students were scholarship students. Scholarship students are selected on the basis of ranks obtained in an entrance examination conducted by the Ministry of Education, Government of Nepal. Compared to the self-financing students they are usually of lower socioeconomic background and from rural areas. These students have to serve for two years in rural areas after graduation and are becoming an important source of support to Nepal's health system 17 . KISTMC provides 10% of total seats on full tuition fees scholarship. In Nepal like in other parts of South Asia there are two media of instruction in schools. In vernacular medium schools all subjects are taught in the local language and English is taught as a second or third language. In English medium schools all subjects are taught in English and the native language is taught only as a second or third language. Most English medium schools are in the private sector and charge high tuition fees. Due to English being regarded as the international language and the language of science and education there is a high demand for English language education in the country. Most of our students were from English medium schools and from an urban background. They would have faced fewer problems in understanding and interpreting the English language excerpts. Medical education is expensive with most schools charging around 40,000 US dollars as tuition fees for the course.
The excerpts as stated were by foreign authors but were either about Nepal or dealt with health related topics. Students felt that these introduced them to different problems they may face in their future practice and also provided ideas about how to deal with these issues. Images of disease and death are common in literature and can serve to introduce students and doctors to experiences they may not have had themselves 18 . Narrative methods are widely used in medicine with patients telling their stories to doctors, who tell these stories to other doctors during case presentations and the patient is regarded as a book needing interpretation 19 . Study of literature can improve performance in the narrative aspects of medicine. We had used brief one-page excerpts in the module due to time and other constraints. Also literature was used as one of the modalities along with case scenarios, role-plays and paintings.
In California in the US a brief literature-based course was shown to contribute to greater student empathy and appreciation of the role of humanities in medical education 20 . At the University of Oxford in the United Kingdom, a special study module using literature was shown to have increased insights gained into patients and their experience of illness and empathy. Certain clinically important skills were also increased 21 . There have been many reviews highlighting the importance of MH and literature and its perceived advantages in undergraduate medical education. A recent review however, concludes that there is a shortage of studies describing long-term impact of MH on the development of medical proficiency 22 . The authors conclude that this may pose a threat to the continued development of MH as administrators and curriculum planners may demand objective evidence of its effectiveness. In Nepal at present to the best of our knowledge we are the only medical school offering a MH module and the first batch of students are now in their fourth year of study. Studying effectiveness in terms of professional behaviors will be important for continued growth of the discipline.
The books and the excerpts used have been already described. The excerpts were in English and most of the students were comfortable with the English language due to reasons previously mentioned. Two of the authors of this article (PRS and RMP) had previously explored the issue of English as the language of MH teaching 23 . In South Asia due to a variety of reasons among them the British colonial influence, English is the language of higher education and the medium of instruction in medical schools. Though Nepal was not ruled by a colonial power, it was strongly influenced by India especially in the field of education. English as the language of excerpts has both advantages and disadvantages. In many developing countries there are a number of languages spoken and it may be difficult to find a language acceptable to everyone 23 . In KISTMC nearly all our students are from Nepal and it may be possible to use Nepali language excerpts. This was not possible at MCOMS, Pokhara where there were students of different nationalities 4 . However, English language excerpts are easier to find and use compared to native language ones. In our previous modules at MCOMS 4 and the faculty module at KISTMC 5 participants had difficulty with the level of English in certain excerpts. These complaints were fewer this time as we had taken care to select excerpts with simpler language.
A few students had volunteered to provide Nepali language excerpts and we can consider this during future modules. The excerpts were short and certain participants had problems putting the excerpt in context. Opinion is divided among facilitators and others about whether to select the majority of excerpts from medicine and related areas. Some feel that this is important as students will be practicing doctors once they qualify while others feel medicine, illness and death are a part of life and cannot be divorced from the larger scenario. Providing photocopies of the excerpts one day before to all students can be considered.
A recent article has stated that MH could be considered as representative of western culture 24 . The authors state that MH programs in Asia promoted a quasi-Western approach to medical humanities (in history, philosophy, literature and art) and the diversity, sophistication and richness of different cultural traditions was uncomfortably marginalized. The authors further state that MH curricula concentrated on two genres of work. These were works in literature, arts and music valued for their insight and beauty and 'experiential' works written by both doctors and patients. The issue of indigenous cultural traditions being marginalized requires attention. In South Asia English is increasingly dominant as the language of education and culture. Western cultural values are strongly promoted through the mass media and the internet. How to involve more indigenous cultural expressions in MH modules can be a major challenge.
Our experiences with the use of literature excerpts would be of interest to medical educators planning to introduce MH modules in other developing nations. At present these modules are few in number but many medical schools are interested in introducing a certain amount of MH teaching in the curriculum. In many developing nations the medium of instruction is either English or other colonial languages. Also in most developing nations only students from a science background are eligible to study medicine. In South Asia twelve years of schooling with the subjects of Physics, Chemistry and Biology during the last two years is necessary. Medical students are less exposed to literature and the humanities and sometimes have a negative perception of these disciplines.
The strength of the study is the high response rate. Our study had limitations. Student perception regarding the use of literature excerpts was only studied using a questionnaire. Other methods were not used. The questions included were arrived at by consensus among the authors. The questionnaire was pretested for comprehension and understanding among five third year students. We however felt that a few respondents had difficulty in properly comprehending certain questions. There is a certain amount of repetition in the responses obtained.

Conclusions
The use of literature excerpts was appreciated by the students. Most felt English language excerpts were acceptable but felt Nepali language excerpts could also be considered. Students strongly felt that the use of literature excerpts should be continued in future MH modules. This use will provide us with more information regarding student understanding and acceptance of the same. The type of excerpts to be used and its length have to be decided based on consensus among facilitators. Our experiences would be of interest to educators planning to introduce MH modules in other developing nations.


A Needs Assessment of ODL Educators to Determine their Effective Use of Open Educational Resources

Nigeria has joined the global race of teaching and learning in a changing educational environment by adopting open and distance learning (ODL). Although it is a global trend, ODL poses some challenges at local levels, one of which is the untimely production of teaching materials currently affecting instructional delivery in Nigeria. The modern approach to ameliorating this challenge is the deployment of open educational resources (OER), and this practice is enabled by information and communication technology (ICT). Hence, today’s educators need OER tools and ICT skills to address the changing nature of education. This paper assessed the needs, readiness, and willingness of ODL professionals from two dual-mode universities in Nigeria to deploy OER in teaching and learning. Data were collected using structured questionnaire items. The major findings of the study’s survey indicated that educators have not really embedded OER in teaching and learning, but they are very eager to be trained in the rudiments of OER and wish to employ them thereafter. The results indicate there is an urgent need for professional development to include training in the rudiments of OER for educators.

Résumé de l'article
Nigeria has joined the global race of teaching and learning in a changing educational environment by adopting open and distance learning (ODL). Although it is a global trend, ODL poses some challenges at local levels, one of which is the untimely production of teaching materials currently affecting instructional delivery in Nigeria. The modern approach to ameliorating this challenge is the deployment of open educational resources (OER), and this practice is enabled by information and communication technology (ICT). Hence, today's educators need OER tools and ICT skills to address the changing nature of education. This paper assessed the needs, readiness, and willingness of ODL professionals from two dual-mode universities in Nigeria to deploy OER in teaching and learning. Data were collected using structured questionnaire items. The major findings of the study's survey indicated that educators have not really embedded OER in teaching and learning, but they are very eager to be trained in the rudiments of OER and wish to employ them thereafter. The results indicate there is an urgent need for professional development to include training in the rudiments of OER for educators.
Introduction "Everyone has the right to education" (United Nations, 1948, Article 26); this right was enshrined in the Universal Declaration of Human Rights over sixty years ago. Therefore, many countries are making a concerted effort to ensure that all people have the opportunity to be educated, a target in line with Education for All (EFA) and the Millennium Development Goals (MDG), which both placed emphasis on the issue. This effort is examplified in the modern adoption and deployment of open and distance education delivery systems in Nigeria to fulfill the nation's commitment to provide education for all, within the context of reaching the World Forum on Education for All (EFA) goals by 2015. These goals involve ensuring that the learning needs of all young people and adults are met through equitable access to appropriate learning and life skillls programs; improving all aspects of quality of education; and ensuring excellence for all so that recognized and measurable learning outcomes are achieved, especially in literacy, numeracy, and essential life skills (Federal Ministry of Education, 2002).
The substantial efforts several governments have made to achieve EFA and MDGs have led to a significant increase in the number of children attending primary and secondary schools in developing countries (Wright & Reju, 2012). Many African countries have invested heavily in education because it is widely accepted as a leading instrument for promoting economic growth (Bloom, Canning, & Chan, 2006, p. 1), and Nigeria is not an exception. Nevertheless, higher education in Nigeria has been under immense pressure to grow from a population that increasingly demands access. The National Open University of Nigeria (NOUN), the premier open and distance learning (ODL) university in the country, was established to widen access to all. But demand is still high, so ODL has come to be recognized in Nigeria as a viable alternative to the conventional school system which hitherto dominated the country's education sector. There are currently six schools in the country which may be regarded as dual-mode universities, with limited capacity to deliver degree programs using open and distance learning in addition to the conventional face-to-face mode (NUC, n.d.).
Researchers have observed that only 36% of those who want to enroll in secondary education programs in sub-Saharan Africa can find seats in schools (UNESCO, 2011). In Nigeria, this situation is even worse at the tertiary level of education, which is expected to provide the opportunity for those willing and able to further their studies. But providing education for all is a daunting task, considering the size of the country's population (about 150 million), and the compelling needs of the people (Okonkwo, 2012). The ever-growing demand for education in Nigeria cannot be met by the traditional means of face-to-face classroom instructional delivery alone. Also, in South Africa, 85,000 potential learners applied for one of the only 11,000 seats available at the University of Johannesburg (Polgreen, 2012). Allen (2010) opined that, "globally, of those 20 years old or younger people, 30 million are qualified to attend university, but there are no places for them. This number is likely to increase by 100 million in 2020." In their own report, Atkins, Brown, and Hammond (2007)  In some local areas in Anambra State of Nigeria, teachers are hired and paid by the parentteacher association (PTA) to teach in primary schools in order to make up for the shortage of the required number of teachers in the system. This challenge is not peculiar to Nigeria alone. The Niger Republic also hired "Volunteer" teachers (Lambert, 2004) who had no teaching experience and often lacked knowledge of subject matter they were teaching.
In the Republic of Congo, some teachers were not paid for several years (Prozonic, 2011).
These realities mean it is therefore not feasible for governments to continue to build, staff, and resource schools, universities, and teacher training facilities in order to meet the demand over the next 5, 10, and 20 years (Wright & Reju, 2012).
In order to fully realize the concepts of education for all and equitable access to educational oppportunities, experts are exploring other options. Notable opportunities are the increased use of distance education combined with information and communication technologies (ICTs), which have greatly influenced education and teaching practices in recent years, in addition to OER, which are being researched for use in both conventional and distance education settings.
It is clear that unless the assumptions that guide academics in open and distance learning are precisely defined, problems of "quality" and "equity" will haunt this mode of education (Das, 2010). There has been a remarkable increase in OER production since 2002, when it was first defined in a UNESCO workshop. There has also been a strong international debate on how to apply OER in actual practice, and UNESCO chaired a vivid discussion about this through its International Institute of Educational Planning (IIEP) (Das, 2010 longer what it used to be. In the changing arena of higher education today, the description of open and distance education has to include "arrangements to enable people to learn at the time, place and space which satisfies their circumstances and requirements" (Das, 2010).
Open education resources make learning available at the time of the learner's choice and at a place suited to his or her requirements. Thus, addressing the issue of openness in distance education contextually and pedagogically brings along with it the need to use digital technologies and ICT beyond borders. I envisage OER as a tool to enable viable outreach in higher education systems in general, and ODL in particular, incorporating innovative strategies in teaching and learning. I also anticipate that OER are capable of enriching learning much more than the materials that we have in the face-to-face institutions, which hitherto have been handicapped by a lack of resources. Despite the laudable vision of ODL globally, it poses some challenges to educators in Nigeria (Okonkwo & Ikpe, 2011). For instance, the writing and development of instructional materials, the backbone of instructional delivery, continues to be a major hindrance to NOUN's vision and mission (Okonkwo, 2012).

Conceptual Framework
Open education resources are teaching, learning, and research resources that reside in the public domain or creative common area and are freely available to anyone over the Web.
They are an important element of the learning infrastructure and range from podcasts and digital libraries to royalty-free textbooks and games. There have been many definitions of OER; I provide four here. Other scholars have also defined OER as teaching, learning, and research resources with intellectual property licenses that permit them to be reused, reworked, remixed, and redistributed (D' Antoni, 2009;Hilton, Wiley, Stein, & Johnson, 2009;Plotkin, 2010;Wiley, 2009). They observed that some conditions may be placed on the use of OER, such as the provision of attribution, but all OER are accessible to anyone. They are seen as having the potential to change the practice of learners, educators, and organizations in a profound way (McAndrew, 2010). These untapped resources have the potential to reduce costs, improve quality, and increase access to educational opportunities (Daniel, 2011;Plotkin, 2010;Wright & Reju, 2012 oer-quality.org/). (Leichty, 2012) Benefits of OER The potential benefits of OER for users already identified in literature (Das, 2010) are • independent discussion forum and news forum for programs; • online interaction facility among fellow students, faculty members, and even officials of the institution; • old examination questions posted online so that students can learn the patterns of questions that are asked; • recent activity reports on the members/users in the community; • online self-assessment tests, which a student can use to test himself or herself on a particular topic or issue; • provisions for uploading special writings by teachers/experts; • facility where students can view the list of online participants reviewing a particular topic so that they can communicate with each other through chatting.
Open and distance learning institutions and educational leaders must grasp the potential of OER by making the collective commitment to use this innovation in order to pursue the goal of education for all. This can be achieved by building on the OER success stories of the African Virtual University (AVU), OER Africa, SAIDE, the Virtual University for Small States of Commonwealth (VUSSC, http://www.vussc.info/), and other African and global OER initiatives. In addition to earlier identified benefits, Wright and Reju (2012, p. 189) opined that OER • have the potential to increase accessibility to quality educational materials at a lower cost to the user, thereby allowing governments and institutions to divert funds to other critical areas such as learner support, faculty professional development, research, and teacher recruitment and retention; • enable individuals to freely access information they can use to make decisions that affect their lives and ultimately the progress of societies; and • have the potential to allow for flexible, quality education through distance learning, and to assist most African countries to meet the increased demand for secondary and higher education. Kanwar, Kodhandaraman, and Umar (2010) (Kanwar et al., 2010). They advocated based on the lessons learned from the above experience that, henceforth, there were three important issues to address.
• It was not enough to develop a capacity and content, the creators must ensure a buy-in from local partners and have a clear implementation strategy; • A governance structure must be put in place to steer the project and monitor progress according to agreed-upon quality standards; • There must be increased capacity in online course development; content writers should develop courses that are needs-based and freely available to strengthen national educational and tertiary institutions.
The advocacy of Kanwar et al. (2010) can only be effective if teachers are first empowered with the necessary ICT skills. Giving basic ICT training will open more opportunities for them in the following ways: • increasing their modern teaching skills (including ICT), which enables instructors to use educational resources available beyond their borders to enhance their teaching abilities; • allowing them to reach a larger number of students by tapping into their expertise to improve literacy through frequent interaction with the students and helping them to solve their problems; • upgrading their knowledge and transferring this to students without having to wait for the new edition of the content to be printed and then distributed; • transforming them into content developers able to generate their own materials according to the specific requirements of the learners they serve by using local languages and examples that are easy for their students to grasp.
The institution will also benefit if teachers start producing educational materials using multimedia because they will be able to develop their own customized content; instructors will be free to modify and update content from time to time according to curriculum requirements with minimum cost; and the teaching quality of the institution will increase. Teachers with ICT skills can also put their content on the Internet and get it peer reviewed. This will ensure that more resources on the Internet are authentic and valid. It will also allow individuals who are not able to get a formal education to access learning resources.
The concept of open educational resources has become well known in Nigeria. Nevertheless, the extent of educators' use of OER and how they were used was not very impressive.

Objectives
The objectives of this study are 1. to identify the participants' knowledge of and experience with OER; 2. to identify the participants' purpose for using OER and the extent of this use; 3. to identify the participants' preferences in OER and how regularly they use OER tools; 4. to identify the participants' levels of agreement with relevant OER issues; and 5. to determine the participants' readiness to attend workshops or training on OER.

Methods and Procedures
The study used a survey which collected data with a structured questionnaire adapted from an unpublished RETRIDAL (2011)  302 questions with percentages only was set at 55%. The 55% cutoff point served as benchmark for acceptance of a participant's response to a given item because it is above average and therefore is meaningful. Thus, any response of 55% and above was accepted as favorable.
This condition applied to objective 1 and 5. The Likert-type scale was used for objectives 2 to 4. The responses for objective 2 and their weight were very regularly (3), regularly (2) Similarly, for objective 4 responses and weights were strongly agree (4), agree (3), disagree (2), strongly disagree (1), and not applicable (0). The boundaries of each response in the 4-point Likert scale were calculated by dividing the serial width (4) by the number of responses (5) and was found to be 0.8 (Topkaya, 2010). This value was used to interpret A score of 2.4 and above on the scale was taken as an indicator of participants' moderate agreement with identified OER issues, while 3.2 and above showed strong agreement. Any score below 3.4 was taken as an indicator of low agreement with OER issues.

Results
The data analyses and results are presented in the tables below. The tables show a summary of the research objectives dealing with the various sections of the study.   Table 1 reveals that the participants have used OER before (63.2%) and they are very familiar with OER (84.2%). Their level of familiarity is also moderately high (high, 10.5%) and (moderate, 47.4%). However, they have not really used the various formats of OER meaningfully, and this is obvious from their described experience of using OER, which was below the acceptable cutoff point, and was therefore rejected.     308 for active involvement in the demands of OER. This is observed in their responses, which ranged from 2.58 to 3.68, that is from moderate (item 2 mean = 3.00; item 3 mean = 2.58) to high mean scores (ranging from 3.32 to 3.68).

Conclusion
It is obvious that the capacities of our conventional institutions cannot ensure that the learning needs of our young people and adults will be met. Education for All (EFA) and the Millennium Development Goals (MDGs) have both placed emphasis on the importance of education to economic prosperity. These have brought about open and distance learning (ODL) in Nigeria, a method of instruction which has received a global acceptance. But ODL is highly dependent on self-directed instructional materials as the backbone for course delivery. So far, the realization of a complete ODL program in Nigeria has been greatly challenged by the untimely production of instructional materials (Okonkwo, 2012). Ameliorating this challenge necessitates continuing professional development for educators in ODL.
Indeed, OER and the emergence of ICT in education are playing key roles in repositioning educational provision in higher education, especially in ODL scenarios, since it has come to stay in Nigeria as a viable alternative to conventional systems of education. The ODL approach worldwide depends largely on the deployment of OER and the use of technology to thrive and succeed. Hence, effective and efficient implementation of ODL in Nigeria calls for the professional development of educators, who are the backbone of high-level academic institutions. These personnel are needed for the effective delivery of classes and have been introduced in response to strong social demands for access to higher education.
However, the results of this study indicate the following.
• The participants are familiar with OER but have not actually been using them sufficiently and effectively. Hence their experience using OER is below the acceptable cutoff point. Mere familiarity with and casual use of OER is not enough to meet the demands of teaching and learning in our changing environment. Adequate experience in using 309 OER is necessary to bring about the desired education demanded by an increasing portion of the population.
• The instructors in higher education institutions have yet to use OER for the purposes which they are supposed to serve in teaching and learning in a changing environment.
• The tertiary educators need urgent instruction in the rudiments of OER and even ICT to enable effective OER utilization in line with current global practices.
• The respondents agree that the issues of OER are meaningful and relevant.
• Above all, they had no reservations about taking full advantage of OER provision and indicated a strong interest in relevant workshops and training.
The paper therefore recommends that there should be training programs covering the rudiments of OER and the ICT skills needed for effective implementation of OER for all educators (both those serving in the conventional systems and those in the open and distance learning environment). This can be done with workshops and seminars for practicing professionals, and the program should be deliberately included in the curriculum for students in Nigerian teacher education institutions.


Relations between representational consistency , conceptual understanding of the force concept , and scientific reasoning

Previous physics education research has raised the question of ‘‘hidden variables’’ behind students’ success in learning certain concepts. In the context of the force concept, it has been suggested that students’ reasoning ability is one such variable. Strong positive correlations between students’ preinstruction scores for reasoning ability (measured by Lawson’s Classroom Test of Scientific Reasoning) and their learning of forces [measured by the Force Concept Inventory (FCI)] have been reported in high school and university introductory courses. However, there is no published research concerning the relation between students’ ability to interpret multiple representations consistently (i.e., representational consistency) and their learning of forces. To investigate this, we collected 131 high school students’ preand post-test data of the Representational Variant of the Force Concept Inventory (for representational consistency) and the FCI. The students’ Lawson pretest data were also collected. We found that the preinstruction level of students’ representational consistency correlated strongly with student learning gain of forces. The correlation (0.51) was almost equal to the correlation between Lawson prescore and learning gain of forces (0.52). Our results support earlier findingswhich suggest that scientific reasoning ability is a hiddenvariable behind the learning of forces. In addition, we suggest that students’ representational consistency may also be such a factor, and that this should be recognized in physics teaching.

I. INTRODUCTION
Assessing students' conceptual understanding has been a popular issue in physics education research (for a review, see [1] and references therein). In this field, perhaps the most widely used assessment instrument is the Force Concept Inventory (FCI) [2], intended for evaluating students' conceptual understanding of force. An important aspect of the research is evaluating the change in students' conceptual understanding during instruction. There are various ways of gauging the change, but one popular measure in physics education research is the average normalized learning gain hgi [3], which is defined as the ratio of the actual gain to the maximum possible gain: hgi ¼ ðpostscore%Þ À ðprescore%Þ 100% À ðprescore%Þ : The average normalized learning gain is used for measuring the change in a class of students (i.e., pre-and postscores are class averages), but the formula above has also been used for evaluating individual student's learning gain (see, for example, [4]). In the latter case, G is called a single student normalized gain, and the pre-and postscores in the formula are those of a single student.
The normalized gain is a useful measure as it allows the comparison of results with different preinstruction scores (a possible relation between prescore and gain is discussed later). For example, a normalized gain of 0.5 can be achieved with different combinations of test scores, e.g., 60% in the pre-and 80% in the post-test, or 80% in the preand 90% in the post-test. Hence, it has been used for comparing test results of student groups and thus the effectiveness of different teaching methods.
Hake [3] analyzed extensive FCI data from 62 introductory physics courses (n ¼ 6542) and showed that the average normalized learning gains were higher in interactive-engagement (IE) courses (0:48 AE 0:14; mean AE standard deviation) than in traditional courses (0:23 AE 0:04). There is no reason to doubt that normalized learning gain may depend on the instructional method used, but it has been suggested [5,6] that differences between the gains of student groups may not be due simply to instructional methods. Various hidden variables such as general intelligence, reasoning ability, and study habits may also influence the size of the learning gain a certain student population can achieve.
In Hake's study [3], no significant correlation was found between FCI prescores and average normalized FCI gain. However, Coletta and Phillips [6] reported a correlation (r ¼ 0:63) between the class prescore and class average normalized gain in 38 college and university interactiveengagement classes. They also reported a significant positive correlation between the FCI prescore and single student normalized FCI gain in three of four university courses where IE methods were used (r ¼ 0:33, n ¼ 285; r ¼ 0:30, n ¼ 96; r ¼ 0:15, n ¼ 1648).
Coletta, Phillips, and Steinert [4,7] and Coletta and Phillips [6] suggest that a student group's scientific reasoning level may explain why there is or is not a correlation between FCI prescore (FCI pre ) and single student normalized FCI gain (G FCI ) in some groups. Students with the strongest reasoning abilities may get both high FCI pre and high G FCI . Such students achieve higher G FCI in high school, so they have high FCI pre in university, and because of their high reasoning ability they also achieve high G FCI . This hypothesis was supported by the finding [7] in 98 university students that the Lawson prescore (L pre ) and the FCI pre correlated (r ¼ 0:53), and that the L pre and the G FCI also correlated (r ¼ 0:51). In this student group the correlation between FCI pre and G FCI was positive (r ¼ 0:33). On the other hand, Coletta and Phillips have proposed that perhaps these correlations do not exist among the highlevel reasoners who would score very high on the Lawson test. They reported no correlation between FCI pre and L pre (r ¼ 0:005) nor between FCI pre and G FCI (r ¼ 0:01) among the best reasoners of 65 university students (n ¼ 16; top quartile of Lawson scores) [6]. They considered that this could explain why the correlation between FCI pre and G FCI did not exist in one of the four universities studied (Harvard University), whose students they supposed to be such high-level reasoners.
Coletta and Phillips [7] reported a correlation between L pre and G FCI also among high school students (r ¼ 0:53, n ¼ 199). Such a correlation has also been found in many replication studies [8]. Coletta, Phillips, and Steinert have argued that achieving a high FCI gain can be easier in classes where the level of the students' scientific reasoning is also high. They have also created a program for identifying students who have low scientific reasoning ability which can also enhance their reasoning in order to help them to learn physics [8].
Previous research has shown that expert scientists are able to fluently use multiple representations when they are thinking and sharing ideas [9,10], and it is argued that one important goal of a physics education is to guide students to expertlike use of multiple representations for successful problem solving and a good conceptual understanding of physics [11,12]. Even the representational format (e.g., graph, vector, or motion map) in which a problem is posed can affect student performance [13][14][15][16]. Physics education research has shown that an instructional approach emphasizing multiple representations is helpful for students' use of multiple representations when the approach is strongly or weakly directed [17]. In the chemistry education context as well, it has been reported that students' learning from multiple representations can be supported by directive and nondirective help depending on their prior knowledge [18].
It is reasonable to assume that the ability to use multiple representations could play some role in students' conceptual gain in a physics course. Hence, our first aim was to clarify the relation between students' representational consistency and G FCI . By representational consistency we mean students' ability to interpret various representations (e.g., graphs, vectors, and motion maps) between isomorphic items in which content and context are essentially identical. For this we use the Representational Variant of the Force Concept Inventory (R-FCI) [16]. Our second aim was to investigate the relations between the students' FCI results and Lawson prescores. This was motivated by the interesting findings on the relations between FCI results and Lawson prescores among university and high school physics students in the U.S. [6,7]; specifically, we wanted to find out whether or not these findings can be replicated in a Finnish high school setting. Figure 1 summarizes the correlations investigated in this paper. We posed the following research questions: (1) Is there a relation between the preinstruction level of students' representational consistency (RC pre ) and single student normalized FCI gain (G FCI )? (2) To what extent can we confirm earlier findings concerning the relation between FCI prescore (FCI pre ) and Lawson prescore (L pre ) and their relation to G FCI ? The motivation to study representations in the context of force was due to two main reasons. Firstly, our research group has done research on the teaching and learning of the force concept for over ten years. Hence, we have special expertise in this particular domain. Secondly, students have some ideas regarding the force concept even before any formal schooling (unlike, for example, regarding special relativity). This is particularly relevant in our study as we were investigating the understanding of students taking their first, mandatory high school course on physics.

A. Research instruments 1. Force Concept Inventory
The Force Concept Inventory [2,19] is a multiple-choice test for assessing students' understanding of the force concept. It is probably the most widely used instrument for evaluating the effectiveness of instruction in physics education research [20]. It has gone through a lengthy process of validation and its reliability has been well established (for a review, see [21,22]). The 1995 version contains 30 items that cover the most basic concepts in Newtonian physics. Each item has five alternatives: one correct Newtonian alternative and four incorrect common sense alternatives. Most of the items are presented verbally, but some items also contain information in pictorial format.

Representational Variant of the Force Concept Inventory
(a) Description of the structure.-We have previously presented the structure, validation, and purpose of the R-FCI [16], which is based on nine items taken from the 1995 version of the FCI [19]. The original, verbal multiplechoice alternatives of the FCI items were redesigned using various representations (bar charts, graphs, vectors, motion maps). The purpose was to form isomorphic variants, keeping the physical concept and context of the items as similar as possible. For each of the nine FCI items, two new isomorphic variants were formulated in different representations. We use the term theme for the set of three isomorphic items consisting of an original FCI item and two isomorphic variants. Figure 2 presents corresponding multiple-choice alternatives of a theme depicted via different representations. There are nine themes in the R-FCI, so the test contains 27 items in total. The themes deal with Newton's laws and gravitation. For a more detailed description of the R-FCI, see our previous article [16].
(b) Analysis of R-FCI results.-The R-FCI score gives information about students' conceptual understanding of the force concept. We have found a strong correlation between R-FCI and FCI scores, although the R-FCI does not include all the dimensions of the force concept that the FCI covers [16]. Furthermore, the R-FCI results carry information about students' representational consistency, i.e., their ability to use different representations consistently between isomorphic items. To reach a deeper understanding of this, consistency analyses were conducted.
Representational consistency does not necessarily require scientific correctness in terms of physics. When exhibiting representational consistency, a student may answer all the items in a certain theme scientifically correctly. On the other hand, all the answers for the theme can be scientifically incorrect, and still the alternatives of the items correspond with regard to the representations (see Fig. 2 for an example of a scientifically incorrect but representationally consistent answer pattern in a theme). Thus, only the ability to interpret multiple representations is considered in the concept of representational consistency.
To determine the students' representational consistency their answers for a given theme were given points in the following way: two points, if they had chosen corresponding alternatives in all three items of the theme one point, if they had chosen corresponding alternatives in two of the three items of the theme zero points, if no corresponding alternatives in the items of the theme were selected In this paper we do not use information about consistency in single themes as we did in our previous study [16]. In contrast, we consider the average consistency in all themes. Thus, all numbers relating to the representational consistency presented in this study are percentages of maximal representational consistency of all themes.
The consistency analysis was solely based on quantitative data, that is, students' multiple-choice answers. These were typed in a spreadsheet which was used to implement the analysis according to coded categorization rules. Hence, there was no significant researcher effect on the consistency analysis and thus no requirement for an interrater reliability analysis.

Classroom test of scientific reasoning
Lawson's Classroom Test of Scientific Reasoning [23], or the Lawson test, is designed to assess the students' level of formal reasoning. The version [24] used in this study contains 24 multiple-choice items concerning the conservation of mass and volume, proportional reasoning, control of variables, probabilistic reasoning, correlational reasoning, and hypothetico-deductive reasoning (see Table IX in [25]). The validity of the original test version [23] has been established by several studies (see references in [26]).

B. Participants and data collection
Five groups of Finnish first-year high school students (n ¼ 131, aged 16) participated in this study ( Table I). The Phys1 groups consisted of regular students, and the Pre-IB groups consisted of students preparing for the International Baccalaureate program.
The students were taking their first, compulsory, high school physics course, which included a general introduction to physics, elementary kinematics, and Newton's laws. The Pre-IB students studied in English using an American textbook [27], whereas all the others studied in Finnish using a Finnish textbook [28]. Despite having different textbooks, the students had many common exercises addressing the use of multiple representations in kinematics and Newton's laws.
All participants took the R-FCI and FCI before and after the courses, but the Lawson test only before the courses. The Phys1 groups took all tests in Finnish. The Pre-IB group took their pretests in Finnish because their English was not good enough at the beginning of the course; however, as all teaching took place using the English language, their post-tests were in English. This may cause a concern about the effect of language on students' performance. To look for evidence of this possible effect, we compared the single student normalized FCI and R-FCI gain between Pre-IB and Phys1 groups: we did not find statistically significant differences (described in more detail below). In this regard, Pre-IB students' learning was very similar to that of Phys1 students, despite the change of language in the post-tests.
All the groups were taught by one of the authors (A. S.), using interactive-engagement teaching methods with various representations; this author has used these methods for many years (for details, see [22]). Furthermore, the teaching approach had a strong focus on treating forces as interactions; this approach has been very successful in fostering students' understanding of Newton's third law [29].
We did not separate Pre-IB and Phys1 students in the data analysis. Despite the described differences between the Pre-IB and Phys1 courses, the students had the same teacher, were exposed to the same instructional methods, and they were all participating in their first high school physics course. Certainly, it was possible that there were some differences between Pre-IB and regular students' academic skills (e.g., language skills) given that Pre-IB students selected to study under the International Baccalaureate program using the medium of English, which is not their native language. However, we did not find statistically significant differences between the student groups in the preinstruction results (L pre , FCI pre , R-FCI pre , RC pre ), or with regard to the single student normalized FCI or R-FCI gain when analysis of variance (ANOVA) and Kruskal-Wallis tests were conducted. Hence, for the purposes of this study we consider all students as one group.

A. Results for the whole group of students
The results of the different tests are given in Table II. The average normalized FCI gain (0.38) was in the ''medium-g region'' (between 0.3 and 0.7 [3]). The pretest results of the R-FCI revealed a big difference between the score (scientifically correct answers) and the representational consistency: despite the rather low pretest score (23%), students exhibited some representational consistency (64%). The R-FCI prescore was statistically significantly lower than the FCI prescore when the Wilcoxon signed-rank test was conducted (z ¼ 6:34, p < 0:001). In contrast, the postscore and single student normalized gain (0.50 for R-FCI and 0.40 for FCI) were higher for the R-FCI than for the FCI. The differences were statistically significant for the postscores (z ¼ 4:36, p < 0:039) and the single student normalized gains (z ¼ 7:21, p < 0:001). One possible reason for this may be that the items of the R-FCI used various representational formats, which can be difficult for students to handle at the beginning of their first high school course. The R-FCI gain indicates an increase in the conceptual understanding of forces and in representational consistency. The difference between the postscores could indicate that the FCI was more difficult for the students, which, in turn, might be due to the greater number and difficulty of items in the FCI.
For calculating correlations between different variables, Spearman's rank correlation coefficient () was used, because many of the variables studied did not distribute normally. Figure 3 shows correlations between G FCI , R-FCI pretest representational consistency (RC pre ), FCI prescore (FCI pre ), and Lawson prescore (L pre ).  There was a positive correlation ( ¼ 0:33, p < 0:001) between the FCI pre and G FCI , but it was clearly weaker than the correlation between G FCI and RC pre ( ¼ 0:51, p < 0:001) or the correlation between G FCI and the L pre ( ¼ 0:52, p < 0:001).
It should be noted that the R-FCI representational consistency and the R-FCI score are very different measures. We found that the R-FCI prescore (R-FCI pre ) correlated only weakly with G FCI ( ¼ 0:23, p ¼ 0:008), whereas the correlation of RC pre and G FCI was 0.51. There was also a strong correlation between R-FCI pre and FCI pre ( ¼ 0:79, p < 0:001), which indicates that the different tests were quite accurately measuring the same construct, i.e., the understanding of the force concept. In contrast, the correlation between RC pre and FCI pre was not so high ( ¼ 0:47, p < 0:001); it was almost the same as the correlation between RC pre and L pre ( ¼ 0:45, p < 0:001) and that between FCI pre and L pre ( ¼ 0:44, p < 0:001).
We found some interesting results concerning single student gain on representational consistency. In calculating this gain, two of the 131 students had to be excluded because their pretest representational consistency was 100%, and in such a case the calculation of normalized gain is impossible because the divisor would be zero (see the equation for normalized gain in the Introduction). There was no correlation between the pretest representational consistency and single student normalized gain on representational consistency ( ¼ À0:026, p ¼ 0:77, n ¼ 129), indicating that the students had learned to interpret multiple representations regardless of their preinstruction level of representational consistency. This gain also correlated very weakly with the R-FCI pre ( ¼ 0:11, p ¼ 0:20, n ¼ 129) and the FCI pre ( ¼ 0:18, p ¼ 0:041, n ¼ 129). Moreover, there was a weak positive correlation between FIG. 5. Scatter plot for the students' (n ¼ 131) Lawson prescore (L pre ) and the single student normalized FCI gain (G FCI ). Spearman's rank correlation is 0.52 (p < 0:001).  the L pre and representational consistency gain ( ¼ 0:28, p ¼ 0:001, n ¼ 129). The correlation between representational consistency gain and G FCI was strong and positive ( ¼ 0:44, p < 0:001, n ¼ 129). Figure 7 shows the G FCI averages in different quartiles. Quartiles were constructed in such a way that students were divided into four equal-sized groups according to a certain variable, for example, their RC pre . As regards the RC pre and L pre quartiles, we found that the G FCI average increased from the lowest to the highest quartile. In addition, in each of the four quartiles, the G FCI averages for representational consistency and Lawson score within a given quartile were nearly equal to each other. In contrast, when the FCI pre quartiles were considered, the G FCI average was even higher in the first quartile than in the second. It can be seen from Fig. 7 that the quartile distributions are consistent with the correlations in Figs. 4-6: representational consistency and the Lawson score correlated more strongly with the G FCI than did the FCI score.

B. Results in the subgroups
The analysis of correlations in the quartiles was problematic because of the small range of values of the variables studied in some quartiles. For example, when the RC pre quartiles were considered, it was difficult to calculate the correlation between RC pre and G FCI in a certain quartile because the RC pre may have had only two values in the quartile. Therefore (with one exception shown below), instead of quartiles we studied correlations when students were placed into the top and bottom half according to their L pre and RC pre .
As explained in Sec. III A, there was a positive correlation ( ¼ 0:33, p < 0:001) between FCI pre and G FCI . When students were placed into the top (T) or bottom (B) half according to their Lawson prescore (see Fig. 8), we found that this correlation did not exist in the bottom ( ¼ À0:017, p ¼ 0:90) but did in the top half ( ¼ 0:43, p < 0:001). Moreover, the correlation between FCI pre and L pre did not exist in the lower half ( ¼ 0:028, p ¼ 0:83), but was strong in the top half ( ¼ 0:50, p < 0:001).
Because our results seemed to contradict the earlier results [6] regarding the students in the highest Lawson quartile discussed in our Introduction, we also studied these correlations in the highest Lawson quartile (n¼30): the correlation between FCI pre and L pre was positive but nonsignificant ( ¼ 0:34, p¼0:069), as was the correlation between FCI pre and G FCI ( ¼ 0:33, p ¼ 0:072).
These correlations were very similar when the division was done according to the pretest representational consistency (see Fig. 9): the correlation between FCI pre and G FCI was not statistically significant and even negative in the bottom half (B, ¼ À0:22, p ¼ 0:091), but strong and positive in the top half (T, ¼ 0:45, p < 0:001). FCI pre and L pre did not correlate among students in the bottom half ( ¼ 0:15, p ¼ 0:25), but the correlation was strong in the top half ( ¼ 0:46, p < 0:001).

C. Reliability index
For internal consistency we calculated values of the Kuder-Richardson formula 20 (KR-20) for all the tests used (Table III). For a reliable group measurement, the KR-20 should be higher than 0.7, and for an individual measurement it should be over 0.8 [30].
We used the tests as an individual measurement because single student results were used. All test values were over 0.8 except for that of the FCI pretest (0.75). However, we accepted this value because it was near 0.8. In addition, the post-test value of the FCI was over 0.8. Because a reliability index is always sample dependent, it is possible that the FCI was quite difficult for the students at the beginning of their first physics course, and this produced the value under 0.8 for the pretest.

IV. DISCUSSION
Our first research question was to investigate the correlation between the R-FCI pretest representational consistency (RC pre ) and single student normalized FCI gain (G FCI ). The second research question was to examine the relations between the FCI and Lawson test results to confirm earlier findings [6,7]. In addition to the whole group of students, we also studied these relations among subgroups in order to discover whether the students' preinstruction level of representational consistency or scientific reasoning had an effect on the existence or absence of some relations.
We found that students' RC pre correlated strongly with G FCI ( ¼ 0:51, p < 0:001), which was bigger than the correlation between FCI prescore (FCI pre ) and G FCI ( ¼ 0:33, p < 0:001), but almost the same as the correlation between Lawson prescore (L pre ) and G FCI ( ¼ 0:52, p < 0:001). When students were placed into the top and bottom half according to their RC pre , the correlation between RC pre and G FCI disappeared in the subgroups. In that regard, the correlation seemed to be a property of the whole student group. Likewise, this correlation existed in both the bottom ( ¼ 0:49) and the top half ( ¼ 0:34) when students were split according to the L pre , although the correlation was slightly weaker among the top-half reasoners.
Interestingly, we found no correlation between students' pretest representational consistency and representational consistency gain ( ¼ À0:026), indicating that students can learn to interpret multiple representations regardless of their preinstruction level of representational consistency. Furthermore, students' preinstruction score on the Lawson test correlated weakly ( ¼ 0:28) with representational consistency gain.
We are not aware of previous reports concerning the relation between the ability to interpret multiple representations and the learning gain of a certain concept. We found a strong positive correlation between students' preinstruction level of representational consistency and their learning of forces. We cannot say that the relation is certainly causal. However, causality is not impossible, because an understanding of representations is required for the adequate use of scientific concepts. It is of course possible that there are also other influential factors, such as general intelligence and spatial ability which explain the ability to interpret multiple representations.
Coletta, Phillips, and Steinert [7] reported a strong positive correlation ( ¼ 0:53) between the FCI and Lawson prescores among the 98 American university students they examined. They assumed that the students with high reasoning abilities had achieved higher learning gains in high school, so they would have high pretest scores in university. This would explain the correlation between the FCI and Lawson prescores in university. In this study, this correlation also existed among students in their first high school course ( ¼ 0:44, p < 0:001), but it was weaker than that found in the aforementioned study. This seems  . 9. Spearman's rank correlation between single student normalized FCI gain (G FCI ) and the three pretest variables: representational consistency on the R-FCI (RC pre ), the FCI score (FCI pre ), and the Lawson score (L pre ). Students were placed into the top (T, n ¼ 69) or bottom (B, n ¼ 62) half according to their RC pre . Ã p < 0:05, ÃÃ p < 0:01, ÃÃÃ p < 0:001. reasonable, because the students in our study are unlikely to have achieved much conceptual understanding of force during their lower secondary school education. Their prescore on the FCI varied between 3% and 80%, and the average was 29%. It was slightly higher than the probabilistic score produced by guessing, which in this case would have been 20%. In addition, our high school data showed that the correlation between L pre and FCI pre was higher among better reasoners and among the more the representationally consistent students: the correlation was 0.50 in the top and 0.028 in the bottom half (L pre split) and 0.46 in the top and 0.15 in the bottom half (RC pre split).
We found a positive correlation ( ¼ 0:33, p < 0:001) among all students between FCI prescore and G FCI . This was quite the same as Coletta and Phillips had reported [6] concerning two of four university courses where IE methods were used (r ¼ 0:33, n ¼ 285; r ¼ 0:30, n ¼ 96; r ¼ 0:15, n ¼ 1648). The correlation was not found among students of Harvard University (r ¼ 0:037, n ¼ 670). Coletta and Phillips assumed that many of the Harvard University students had achieved a high level of scientific reasoning and would have scored very high on the Lawson test for that reason. They found that among 65 students from Loyola Marymount University, as regards the students (n ¼ 16) who scored highest on the Lawson test (top quartile), there was no correlation between FCI and Lawson prescores (r ¼ 0:005), nor between FCI prescore and G FCI (r ¼ 0:01). Among the top quartile in our data (n ¼ 30), these correlations existed ( ¼ 0:34, p ¼ 0:069; ¼ 0:33, p ¼ 0:072, respectively), but were not statistically significant. It must be noted that the participants in our study were first-year high school students, whereas those in the study by Coletta and Phillips were attending university. In our data Lawson prescore (85%) and G FCI (0.52) in the top quartile were lower than was the case in the top quartile of the study by Coletta and Phillips (93% and 0.59, respectively). There is a possibility that the scientific reasoning of the top quartile students in our data was not strong enough, so that these correlations would not have existed in their case. Anyway, in our high school data, the correlation between FCI pre and G FCI was stronger among top-half students when the students were placed into the top and bottom half according to the L pre and RC pre (see Figs. 8 and 9).
Coletta, Phillips, and Steinert [7] reported a strong correlation between students' preinstructional level of scientific reasoning ability and the single student normalized FCI gain among 98 university students (r ¼ 0:51) and 199 high school students (r ¼ 0:53). They have also reported that such a correlation has been found in many replication studies [8]. Further, they [7,8] have created a program for identifying students who have low scientific reasoning ability, and which can be used to enhance their reasoning in order to help them to learn physics. We were able to confirm the correlation between students' scientific reasoning ability and G FCI in our data ( ¼ 0:52, p < 0:001). Hence, we are convinced that weak physics students might particularly benefit from the explicit teaching of scientific reasoning skills.

V. VALIDITY AND LIMITATIONS
The data of this study were collected with quantitative multiple-choice tests that were straightforward to take, administer, and score without researcher bias. The reliability and validity of the study are affected by the reliability and validity of the test instruments. We discuss the validity of the tests in Sec. II A, and we consider them valid for high school students. For reliability, which is a prerequisite for any validity, we calculated KR-20 values, and these were acceptable for all the tests used in this study (Table III).
External validity (generalizability) is the major limitation of this study. The results cannot be generalized even to the population of all first-year high school students in Finland, because the data were collected in a particular high school and from students taking courses with a particular teacher.

VI. IMPLICATIONS
Our results concerning the strong relationship between students' representational consistency and their learning of forces are well in line with those of previous studies [11][12][13][14] supporting that careful consideration of multiple representations is important for learning and understanding physics concepts. One way to increase knowledge about multiple representations in physics teaching among Finnish high school teachers would be to offer resources for teaching multiple representations, such as researchbased materials and practices. There is a clear need for the aforementioned resources as physics textbooks in Finland do not often include many multiple representation exercises [31]. Furthermore, textbooks tend to have a central role in Finnish high school physics teaching. Another potentially effective field in which to highlight the importance of multiple representations could be in the training of preservice physics teachers.
Earlier research has shown that an instructional approach emphasizing multiple representations can be helpful to university students in their use of multiple representations [17,18]. Our study cannot fully take part in the discussion on instructional approach as only one teaching method was used and without comparison groups. However, in our other study [31], Finnish high school students (n ¼ 28) answered open-ended, paperand-pencil questions which we had designed to emphasize the use of multiple representations in the context of forces. The results lend some support that students' understanding of the force concept and multiple representations was increased.
The data of the present study were collected in one Finnish high school where the interactive-engagement (IE) teaching method and multiple representations were used. In the future, attempts should be made to replicate the results with different groups of students. Further studies should investigate what kind of correlation exists between preinstruction representational consistency and G FCI in high school when IE methods are not used, as well as when multiple representations are not used. Research is also needed to clarify whether a correlation between representational consistency on the R-FCI pretest and G FCI exists at the university level, where students probably have the competence to interpret the standard formats of representations used in the R-FCI.

ACKNOWLEDGMENTS
This study was supported by the Academy of Finland (Project No. 132316).