Outcome analyses of educational interventions: a case study of the Swedish “Boost of Technology” intervention

In Sweden, there have been multiple large scale interventions to support compulsory school teachers generally and within specific subjects. Due to the costs associated with such interventions it is critical that interim evaluation measures exist which can indicate potential success. Additionally, evaluation measures which can measure the actual impact of interventions relative to their intended aim are also needed as validation tools. The Swedish regional ‘Tekniklyftet’ or ‘Boost of Technology’ project which ran from 2011 to 2013 is presented here as a case study exploring evaluation measures for educational interventions in technology education. Three different evaluation approaches were used as measures of the intended outcomes of the intervention. These included (1) analysing the preconditions which exist in schools for teachers of Technology, (2) analysing the use of local long term technology education planning documents (school work plans) developed during the intervention, and (3) analysing the potential change over time in student performance in Technology based on national grades at the end of compulsory school. The findings gained from each approach indicate that the Boost of Technology project was a success. However, there were shortcomings associated with each approach. They are therefore discussed in the Swedish context with the intention to support future international stakeholders in the evaluation of interventions aspiring to develop technology education.

Introduction
Insufficient teacher education, in this case relative to quantity of its provision, is detrimental for any subject at primary and post-primary level. Lately, Science, Technology, Engineering and Mathematics (STEM) education in the whole of the Swedish school system is suffering from declining results in many areas. This is apparent from results of international studies such as from the Trends in International Mathematics and Science Study (TIMMS) (IEA 2018) and from the Programme for International Student Assessment (PISA) (OECD 2018) in which Sweden has shown a decline in performance since 1995 and 2000 respectively with the exception of the most recent round of results. While student achievement is suffering in these areas as illustrated by these studies, the importance of primary and postprimary STEM education is well-acknowledged in Sweden. One way this is evidenced is through the numerous initiatives aiming at increasing achievement and interest in STEM subjects. Studies have been undertaken by the government as well as private actors, an approach which is reflective of international efforts (e.g., Hernandez et al. 2014;Mammes 2004). However, for both educational and economic reasons, determining whether the outcomes of these initiatives are in line with their stated aims needs to be addressed.
In-depth knowledge of teaching and learning in the STEM topics is based on and depends on knowledge of the individual subjects individually and in combination. In this present study we exclusively focus on the subject of Technology, which in Sweden often is seen as the weakest link in the STEM chain. We specifically examine a regional educational intervention 'Tekniklyftet' (herein referred to by an English translation, 'Boost of Technology'), which was conducted in the Stockholm area between 2011 and 2013, is examined. This intervention aimed at improving the provision of Technology in Swedish compulsory schools. The project was funded by the European Social Fund (ESF) in which approximately 15.5 million SEK (≈ 1.5 million EUR) was spent and nearly 750 unique teachers participated to varying degrees. A number of evaluation studies were performed during the intervention (e.g., Gumaelius and Skogh 2015;Hartell et al. 2015) and two further studies were carried out subsequent to its completion (Johannesson 2014;Norström 2014). Johannesson (2014) published the final external evaluation report in which the possibilities and limitations, primarily regarding organisational issues, were highlighted. Norström (2014) analysed participating teacher's views and understanding of the concept of technology and found a considerable variation in how this concept was perceived. Norström's (2014) results also point to these teachers being unaccustomed to reflecting on the complexity of the concept of technology. Based on these studies and the final report, the funding agency (ESF), indicated that they consider the intervention to be successful. However, to the organizers as well as to the involved educators, it is not yet completely apparent whether the intervention was successful in terms of reaching the overall goal of improving technology education in the participating schools.

The need for the improvement of technology education in Sweden
In Sweden, Technology has been a mandatory subject in compulsory school for more than 30 years. However, the desired and necessary level of quality in the provision of the Technology subject at post-primary level in Sweden has not been reached (Skolinspektionen 2014;Teknikföretagen and CETIS 2012). The Swedish Technology curriculum is considered to be difficult to interpret, and the technological understanding and knowledge among those teaching the subject in Sweden varies greatly (Norström 2014). It appears as though the subject has not yet established a unique position within the Swedish educational curriculum.
In 2013, the Swedish School Inspectorate performed the first national technology education evaluation (Skolinspektionen 2014). The Inspectorate concluded that Technology often remains in the background to the benefit of the other STEM subjects, that teacher's instruction often lacks logical progression, and that teaching is not sufficiently aligned with the current curricula. The report also highlighted the lack of access to teaching materials, including books and equipment, and suggested a lack of collegial learning and planning with other teachers, factors which were also reported in a study conducted by Hartell and Svärdh (2012). In line with Skolinspektionen (2014), Hartell et al. (2015) also identified that not enough hours of instruction are allocated to the subject. Two of the main stakeholders promoting compulsory school technology education in Sweden-the Centre for School Technology Education (CETIS) and the Association of Swedish Engineering Industry (Teknikföretagen)-undertook a similar evaluation study (Teknikföretagen and CETIS 2012) in which they report on deficient technology education provision. Additionally, according to this study there is a strong relationship between the quality of teaching in Technology and the preconditions for teaching it, i.e., the level of qualification of teachers, the adequacy and quality of the premises, and the scheduling for the subject.
The subject's low status in Sweden (Skolinspektionen 2014;Teknikdelegationen 2010) and the fact that many Swedish teachers lack relevant training in teaching the subject (Teknikdelegationen 2009) undoubtedly creates problems regarding the quality of the teaching. Reports suggest as much as 70-93% of Technology teachers in Sweden are unqualified to teach the specific subject they are teaching (Skolinspektionen 2014;Teknikföretagen 2005). Teachers teaching Technology may have a teaching certificate, but the majority of them are trained in other subjects and not specifically in Technology. There has been, and still is, a large need for educational interventions targeting Technology teachers in Sweden, particularly at primary and post-primary level . The ESF funding of the Boost of Technology intervention should be considered both within and as a response to these circumstances.

The Boost of Technology project
To address these reported deficiencies in the Technology subject in Sweden, Vetenskapens hus (English translation: House of Science), a science educational centre owned by KTH Royal Institute of Technology and Stockholm University, and its partners, secured ESF funding to launch the Boost of Technology project in 2011. The activities at Vetenskapens hus are of varied nature, always aiming at increasing students' awareness in STEM subjects. Most often students are visiting with their class mates for a few hours, attending a laboratory program, but, activities also include events such as Researchers night or Physics in an Amusement park where thousands of students can be present at once, for more information see: www.veten skape nshus .se. Vetenskapens hus was the natural owner of this project as they serve as the link between university and school. The long-term goal was to contribute to a broad renewal of technology education in Swedish compulsory schools. Focusing on Technology teachers and their work situations, the project sought to strengthen teachers' skills and raise the national status of the subject by including teachers who teach Technology, but also other teachers, school administrators and representatives of the local school authorities. In line with Skolinspektionen (2010) and the OECD (2009), teachers' competence was seen as a critical factor in increasing students' interest in the subject and encouraging more young people to pursue technology education and careers. Initially, the main focus was on the final years of compulsory school (Years 7-9), however some schools included classes from preschool through Year 9. The improvement of technology education was defined as improvements towards teaching according to the national curriculum for Technology (Skolverket 2011a) in relation to content, resources, and pedagogy.
While many schools and teachers were involved to varying degrees in the Boost of Technology intervention, the measurable goal was to certify 20 invited pilot schools in the Stockholm region that agreed to develop technology education according to a set of goals. The improvement of technology education was defined as improvements towards teaching according to the national curriculum for Technology (Skolverket 2011a) in relation to content, resources, and pedagogy. To receive the certification, schools had to: 1. Account for at least one teacher having obtained 45 European Credit Transfer System (ECTS) credits in subject-specific training (i.e., in Technology and/or Technology pedagogy) to align with the level required by teacher education programmes in Sweden for teacher certification in specific subjects since 2011 (Sveriges Riksdag 2011a). 2. Account for all Technology teachers to have obtained at least 15 ECTS-credits in Technology and/or Technology pedagogy (unlike the previous criterion, this was decided internally within the Boost of Technology project). 3. Submit a local work plan for the subject of Technology. The development of this plan had to be interdisciplinary; in other words, teachers from different subjects participated in the process to raise the status of the subject at the school. Both teachers and school management had to agree on the work plan. 4. Develop collaboration with a nearby company/industry and a science centre to link technology education to reality.
Activities were led by Vetenskapens hus and involved university institutions, industry and science centre's representing different areas of society dependent on the nature and aspect of technology education being focussed on at a specific point in time. The project was considered as a pilot study. If it were proven to be efficient and effective, the aim was to undertake the project in other schools and areas in Sweden.

Educational interventions supporting technology education and their evaluation in Sweden
Interventions to support teachers are not uncommon in Sweden. The Boost of Technology intervention is one of multiple such efforts, which have been or are being conducted which either directly or indirectly support Technology teachers. Directly, a number of local and national educational interventions targeting schools have been undertaken since the turn of the millennium such as 'Teknik8an (competition in STEM for students in grade 8) (www.tekni katta n.se), 'Snilleblixtarna' (for teachers and students in K-6) (www.snill eblix tarna .se),'Tekniken lyfter' (English translations: 'Boost of Technology' or 'Technology Lift') (e.g., Lindahl and Jansson 2004), which ran from 1999 to 2003, and the currently ongoing 'Lärarlyftet' (English translations: 'Boost of Teachers' or 'Teacher Lift') (Skolverket 2018a). Indirectly, 'Matematiklyftet' (English translations: 'Boost of Mathematics' or 'Mathematics Lift') (Skolverket 2018b) which ran from 2012 to 2016 targeting Mathematics teachers may benefit Technology teachers due to the relationship between Mathematics and Technology.
Considering the extent of the investment, both financially and in terms of time, there is a clear need to be able to evaluate these interventions. Such evaluation is needed early and continuously to decide on the merit of continued investment, and at the end to determine the effect of the intervention as a whole. To study the effects of interventions is not a new area of research internationally or in Sweden. Lärarlyftet has been evaluated several times by the Swedish Agency of Education, mostly by interviewing involved teachers about how satisfied they are with the courses they have taken through the intervention. For example, Westlund (2012) found that teacher's self-efficacy increased during the studied intervention. Regarding Matematiklyftet, an interim report (Karlsson et al. 2013) was performed based on a mixed method approach (teachers and management interviews, observations of classroom activities and text analyses of formal documents). Additionally, the final report highlighted the success of the project (Österholm et al. 2016).
In line with these studies, there is now a need to reflect on and evaluate the Boost of Technology intervention. Many different approaches can be and have been used to determine whether technology education is of an acceptable quality. For example, Timperley (2011) shows that appropriate methods include measuring student achievement outcomes and attitudes. According to Teknikföretagen and CETIS (2012), there is a strong relationship between the conditions available to teach Technology and the way technology is taught. In their report, a 'formula' for technology teaching based on six criteria is suggested: 1. Each school should have at least one teacher certified in Technology. 2. Professional development should be available to all teachers teaching Technology. 3. Each school should have a subject specific plan (work plan) for Technology. 4. There should be time set aside dedicated for all teachers to plan and develop the teaching of Technology. 5. The subject of Technology should be included in all participating schools' curricula and it should be continuously taught across Years 1-9. 6. The subject of Technology should be included in all participating schools' curricula and it should be continuously taught across Years 1-9.
In conclusion, the importance of developing and highlighting effective interventions that aim to increase knowledge and interest in Technology and improve the quality of technology education cannot be underestimated.

Aims and research question
There are two complementary aims of this study. The first is to examine the outcomes of the Boost of Technology intervention relative to its aim of improving the provision of technology education in Sweden beyond the remit of the previously discussed associated studies and final report. The second, and arguably more important aim as it informs future practices, is to explore ways of gaining an insight into the potential future success of an intervention by examining the outcomes of the Boost of Technology intervention. The main objective is to identify possible ways to design usable evaluation methods that allow for an early insight into the efficacy of technology education interventions so as to inform the continuation of such projects. This study is particularly interested in the institutional level, i.e., how the schools have succeeded in changing education in line with the aims of the intervention. It is critical that the evaluation method is (1) relatively cost efficient and easy to collect relevant data, (2), possible to repeat, with generalizable results and (3) permits exploration of actual improvement in the schools' teaching-practice and resultant learning. As such, the research question guiding this study is: To what extent are the following approaches appropriate and relevant, relatively cost-efficient, and methodologically suitable, when evaluating a technology education intervention?
Approach 1: Exploring if and how the preconditions for teaching technology change during an intervention. Approach 2: Analysing the development of work plans devoted to the subject of Technology during an intervention. Approach 3: Examining changes in student performance as a result of an intervention.

Approach 1: exploring changes in the preconditions for teaching technology
In line with Teknikföretagen and CETIS (2012), the first selected approach was to analyse how the preconditions for teaching Technology have changed over time as a result of the Boost of Technology intervention. Three of the suggested criteria were considered including (1) the education level of Technology teachers, (2) how Technology as a subject was scheduled, and (3) what material standard the participating school offered for Technology. While this approach does not provide a direct measure of the ability of teachers or of students' results, it has been suggested as a criterion that is vital for good technology education provision (Teknikföretagen and CETIS 2012).

Method
Data was collected through the requirement of documents within the Boost of Technology project. The collected documents consisted of local work plans, local timetables including information of when (which school year) Technology was taught, budgets for all subjects, the number of certified Technology teachers per school year, and the books and other teaching materials used in teaching technology. This information was gathered where available at the start-up of the project, 2010-2011, and after finishing the project 2012-2013. Where necessary, phone call interviews with schools were conducted to clarify information from within this documentation.

Precondition 1: teachers level of qualification
To be a certified Technology school as a result of the Boost of Technology intervention, each school needed to educate at least one teacher to have 45 ECTS credits in the subjects of Technology and/or technology pedagogy. All other teachers delivering Technology in the participating schools had to have at least 15 ECTS credits.
Throughout the entire project, approximately 1000 teachers and school leaders participated in the different professional development activities facilitated through the Boost of Technology intervention. This level of participation suggests that the intervention achieved positive impacts on a large number of Technology teachers. During the project, teachers at 17 of 20 selected pilot schools attended training courses. The remaining three schools either already met the criteria regarding ECTS credits or had teachers who were unable to attend. On average, 1.85 teachers per school participated, and 3.45 courses per teacher were undertaken. Nine schools had fulfilled the ECTS certification requirements when the project finished and the remaining schools were working on their last component of competency training. During the Boost of Technology intervention, the number of accredited teachers (15 ECTS credits) in the participating schools increased by 28%, with this number expected to rise.

Precondition 2: technology as a subject within school schedules
According to the steering documents, students must receive 800 h of instruction 1 in science and technology during Years 1-9, and school leaders are free to distribute the allocated time among the different subjects to suit their local circumstances.
At the beginning of the Boost of Technology intervention, at least four schools did not mention Technology as a specific subject at all, but during the project they addressed and changed this practice. Figure 1 shows that at the end of the project, Technology was mentioned as a subject in its own in all local syllabuses in school Years 7-9. In school Years 1-6, it seems more natural to treat the subject as part of the Science and Technology block as nine of 20 schools had a syllabus combining those subjects. It should be noted that schools with missing formal information were likely to not offer Technology as an explicit subject as if they did the documentation would have been available.
CETIS (2014) recommends that 200 h should be explicitly devoted to the subject Technology in post-primary education with 100 of those hours being taught in Years 7-9. At the beginning of the project, due to a lack of documentation or clarity within documentation, it was difficult to obtain explicit information concerning how many hours were solely dedicated for Technology. Among the 14 schools that were able to provide information at the beginning and the end of the project, eight increased the time set aside for the subject. At the end of the project, large variations were seen in the amount of hours dedicated to Technology in Years 7-9 (Min = 54, Max = 144, M = 90.40, SD = 24.94).
Although most schools treated Technology as a subject in its own right with a time plan of its own, the work plans (discussed below) showed that most participating schools also approached it in an interdisciplinary way. This shows that an interdisciplinary approach does not necessarily mean that the subject receives less attention.
Following up on this criterion was relatively easy as all schools had information about class schedules available. However, it was difficult to determine that technology was actually taught in classes when combined with other subjects.

Precondition 3: What educational conditions are present in the schools for Technology?
Data were collected verbally (interviews with teachers and principals), in written form, (by documentation or reports from participating school's economical accounting), and from inspections of the school premises during school visits. Due to the nature of the subject, in that it frequently involves practical elements, the basic condition assessed was the premises or classroom for Technology. Figure 2 reveals a development from using general purpose classrooms for Technology at the beginning to using specially equipped Technology classrooms at the end of the intervention. This is seen as positive development and a result of the project. It should be noted that schools with missing formal information were likely to use general purpose classrooms for Technology as specially equipped rooms would have been detailed in school documentation.
To show the importance of appropriately equipped classrooms, Fig. 3 illustrates two examples of how material and equipment were stored and used in schools without specific Technology facilities at the beginning of the project.
Participating schools were also asked about the budget for teaching materials and equipment. At the beginning of the project, 14 schools submitted their budgets, and all schools provided this information at the end of the project. At least nine schools had increased their budgets for Technology, while 11 schools at least had specific budgets for Technology at the end of the project. This is also seen as positive development and a result of the project.
Of the schools that did not have a budget specifically for Technology at the end of the project, most included it in that for Science subjects or had a common budget for teacher teams teaching all subjects to a specific group of students. Some schools did not have an allocated budget for Technology, and of those which did, some spent 10 SEK/pupil (equivalent to 1 EUR) annually, and others spent 500 SEK/pupil (equivalent to 50 EUR) annually.

Approach 2: analysing the development of work plans devoted to technology
In Sweden teachers are advised to work in line with the General guidelines provided by the Swedish National Agency for Education (Skolverket 2011b). The General guidelines are recommendations for how schools and kindergartens should apply rules and regulations. These recommendations must be followed unless a school can ensure that the regulatory requirements are met in a different way. According to the General guidelines, a school's mission is to create opportunities for all students to develop as far as possible in relation to the national targets. This requires well-planned and structured teaching based on the Education Act (Sveriges Riksdag 2011b) and curriculum and syllabi (Skolverket 2011a). Teachers should also plan to evaluate students' knowledge and their own practice, and for communicating with students, parents, head teacher and other teachers (Skolverket 2011b).
In order to become at certified Technology school within the Boost of Technology project, each participating school had to develop a strategic, local steering document (work plan) showing how Technology would be taught throughout the entire compulsory school curriculum (Years 1-9). Since the new Education Act was applied in 2011 (Sveriges Riksdag 2011b), legislation for documentation of local steering documents has not yet been created. However, many schools and municipalities use similar documents created at local level (sometimes referred to as local work plans or local pedagogical planning) for each subject in school. It is often argued that this kind of documentation is good for quality assurance (Eriksson 1994;Teknikföretagen and CETIS 2012). Analysing schools work plans for Technology is the second approach investigated here in terms of determining a cost-efficient and effective approach to evaluating educational interventions.

Method
At the start of the project only three out of the 20 schools could submit local work plan, reflecting previous research findings (Bjelksäter 2011;Skolinspektionen 2014;Teknikföretagen and CETIS 2012). At the end of the project local work plans (approved by the local school administration and management) from 19 schools had been submitted. The 19 work plans were analysed according to a scheme developed by the research team. The two members of the research team who conducted the analysis worked in collaboration to achieve consensus for each judgment. The seven categories which the work plans were evaluated under were determined both from important qualities mentioned within steering documents and by the Boost of Technology research team. They included: a. Objectives of teaching and how they are linked to the curriculum. b. Evaluation and assessment. c. Distribution of selected teaching elements between years. d. Collaboration between the Technology subject and other school subjects. e. The four perspectives given in the national curriculum (the school values); (1) the historical, (2) the international, (3) the ethical, and (4) the sustainable development perspectives. f. Gender.
The work plans were evaluated under seven categories by being graded on a 5-point scale. However, at the time of analysis, it was determined that some schools required scoring in half points to more accurately reflect variances in quality. It can therefore be considered that a 9-point scale was utilised. However verbal descriptors were not created for the extra points.
The scale applied for the analysis was: 1. Aspect is not mentioned in the document. 2. Aspect is mentioned, but no description of it is included in the document. 3. Aspect is mentioned with a brief description, but no commentary is provided as to what or how it will be taught. 4. Aspect is mentioned with a brief description, and what and how it will be taught are partially described. 5. Aspect is mentioned with a brief description, and what and how it will be taught are fully described.

Findings
Every school was given an overall assessment of their work plan, which corresponds to the total number of points in this evaluation. The highest score a school could achieve was 35 points (i.e., five points for all 7 categories). The average score for the participating schools was 23.7 points. The highest score achieved was 32.5 points and the lowest score was nine points. Eight schools achieved over 25 points. A full breakdown of the results from the analysis of the schools work plans is provided in Table 1. Further examination of the mean result for each category (Fig. 4) illustrates which categories schools considered most and least when planning for the provision of Technology. Schools performed best in relation to their distribution of Technology across school years (category c) and in linking their objectives to the curriculum (category a). In relation to evaluation and assessment (category b), the average score was 3.00. Only three schools specifically mentioned how and what should be assessed, although most schools did include some form of assessment criteria, most often directly copied from sample material written by the Swedish Agency of Education. Category d, which describes collaboration between the Technology subject and other subjects in schools, had an average score of 3.50. This suggests that many schools included the teaching of other subjects in projects/ learning activities aimed at developing Technology education. Category e, the relationship between the work plans and the perspectives in the national curriculum, had the lowest average score of 2.89. Categories f and g (gender issues and entrepreneurship respectively), are important school values that should permeate all teaching in all subjects (Skolverket 2011b). The average score of approximately 3 for both aspects can therefore be seen as low. Six schools (not the same schools for both categories) out of 19 described these aspects in a clear and relevant way.
While all work plans referred to the policy documents (categories a and e), which is considered a positive result, this does not imply translation into practice. Many of the work plans contained quotes from the curriculum without any specification regarding how the aspects in question should be enacted (choice of activities and/or aims and learning goals). In the cases where actual suggestions regarding themes and activities did occur, the learning objectives (i.e., what the students should/must achieve) were rarely described. As mentioned, the work plans demonstrated that the participating schools were able to distribute the provision of Technology between different years. While this could be interpreted as an easy task, considering that many of the participating schools did not have Technology represented in each year prior to the start of the project, this is seen as proof of improvement in the provision of the subject.

Approach 3: exploring changes in student performance at a group level
The third approach explored in terms of an evaluation method for educational interventions is to investigate the potential change in participating Year 9 students' grades in compulsory school over time, during the time of the Boost of Technology intervention. The national Swedish compulsory school examinations in Year 9 are considered a high-stakes assessment as they serve as a matriculation system for entry into high school. Therefore, these grades function as proof of how students' succeed in each subject. However, caution is advised regarding this approach in Sweden as the Swedish grading system has been shown to possess many flaws (Gustafsson et al. 2014). It has been found, for example, that both school assessment systems and individual teacher assessments are not reliable among different Swedish schools (Nusche et al. 2011), meaning that students received different grades depending on what school they entered. Even though there are national criteria in the form of standardised learning outcomes specified for each subject (Skolverket 2011a), it is difficult for individual teachers to grade their students based on their performance. In some subjects national tests can help teachers to define what level their students perform at, but in Technology no such tests are available. The grades are the only formal documentation collected nationally in Sweden regarding performance in Technology.

Method
The grades from Year 9 for all students at the participating schools were retrieved from the National Agency database (SIRIS), where the pertinent available data that was collected and analysed included the number of students examined and the mean performance in Technology for each school. Data was analysed for three cohorts of Year 9 students (n total = 4634). The 2010 cohort (n 2010 = 1610) was selected as this was the year before the intervention started, the 2013 cohort (n 2013 = 1438) was selected as this was the year the intervention ended, and the 2016 cohort (n 2016 = 1586), who were examined 3 years after the project finished, was selected to explore the potential longitudinal effects of the project in these schools. In 2010 merit points were awarded according to a 3-point scale, were the highest grade corresponded to 20 points, the middle grade to 15 points, and the lowest grade to 10 points. Subsequently a new grading system was implemented (Skolverket 2011a) meaning for the 2013 and 2016 cohorts a 5-point scale for merit points was used. The highest grade still corresponded to 20 points, followed by 17.5, 15, 12.5 and finally 10 points. Figure 5 illustrates the mean performance across the 20 pilot schools for each of the three cohorts and reveals a positive trend with a mean difference of 1.87 merit points (9.35%) between 2010 and 2016. There was a statistically significant difference between cohorts as determined by a one-way ANOVA (F(2,4631) = 215.507, p < .000, η p 2 = .085) and a Tukey HSD post hoc test revealed statistically significant mean differences in performance (p < .000) between each pair of cohorts. This trend is also apparent in the percentage of students who achieve a passing grade in Technology across each of the years with 74.50% passing in 2010, 93.56% passing in 2013, and 95.77% passing in 2016.

Findings
While the trend is positive and there are statistically significant mean differences across each year, the effects cannot be purely attributed to the Boost of Technology intervention due to potential confounding variables. Comparing the performance differences in the pilot schools with other schools to more clearly determine the effect of the intervention is not possible for two reasons. Firstly, during the intervention some teachers and principals in the pilot schools moved to other schools to teach Technology and secondly, as discussed approximately 750 teachers engaged with the project to some degree. While the pilot schools interacted with the project the most, there is no guaranteed pure sample for comparison. Despite this, the positive trend is considered to be at least partially a result of the Boost of Technology intervention and these results are seen as a good implication of the project.

Discussion
As previously outlined, there were two complementary aims for this research. The first was to examine the outcomes of the Boost of Technology intervention relative to its aim of improving the provision of technology education in Sweden. The findings for each of the three approaches discussed in this paper, examining preconditions, work plans, and student performance, suggest that the intervention was successful in this regard. Furthermore, when combined with the results of the previously published studies on the intervention (Gumaelius and Skogh 2015;Johannesson 2014), more certainty is given relative to this aim.
However, there were shortcomings with each approach described in this paper which are important to be noted as they most likely affected the final results. First, teachers were found to move their positions to nearby schools. Second, school principals were also found to move to other positions. Finally, very few schools had the same school management in the beginning as in the end of the Boost of Technology project. These circumstances affected the coherency and efficiency in data collection, may have negatively impacted potential change, and invalidated the use of nearby schools as reference schools as they were most likely to change as well.
The second aim of this research was to explore ways of gaining an insight into the potential future success of an intervention by examining the outcomes of the Boost of Technology intervention which are (1) relatively cost and time efficient, (2) possible to repeat, giving generable results and (3) permitting of exploration into actual improvement in the schools' teaching practice. Each of the three approaches from this paper will now be discussed and compared relative to these criteria.

Time and cost efficiency
When comparing cost efficiency it is assumed that labour costs derived from data collection account for the majority of the total costs. In terms of time consumed, consideration is given to whether the material sought from schools was produced for the sake of the investigation, or whether this material was already produced for other purposes within the school.
For the first approach, examining the preconditions within a school relative to teaching Technology, the time needed for data collection was longer and more difficult than expected, which impacted the associated costs. School management needed to be contacted between one and four times (approximately twice on average) over a time period of several months, as the data was not compiled by habit. At the same time, the data was achievable by all schools. Therefore, the introduction of a compulsory scheme or template for compiling such data yearly could easily be achieved in future interventions. For the second approach, examining school work plans, it was found that the time required for collecting data was entirely dependent on whether the school had used work plans before this investigation or not. As a result, the labour cost for collecting the material at the end of the project was relatively short in comparison to at the beginning of the project. Introducing the use of work plans created substantial work for the teachers who did not already do this, but as a method of collaboration between teachers it was quickly accepted positively within schools. It seems surprising that such planning documents were uncommon, however their absence was also observed by Bjelksäter (2011) who found work plans or similar documentation to be rare. Therefore this approach is not considered to be cost-efficient, especially if the use of work plans was not already included in the schools' regular activities. For the third approach, examining student performance grades, the data was collected easily through a national database, and is therefore considered to be the most cost efficient and time efficient method regarding data collection.

Repeatability and generalizability of results
Examining school preconditions for teaching Technology are considered to be both repeatable and generalizable if the parameters asked for in terms of preconditions are clear and decided upon beforehand. Most of the parameters asked for in this project were available to some extent, but as previously stated they were difficult to obtain. Differences between schools were observed, for example in relation to subject budgets, which meant that this precondition was difficult to compare and thus not generalizable. Examining subject work plans is determined to be the least repeatable as there are no national or regional standards for working with such documents. However, the Boost of Technology intervention showed that when standardised templates were provided to teachers and schools, the results could be generalizable. Analysing student performance relative to the grades in Year 9 was found to be the approach which was easiest to handle, most repeatable, and it provided generalizable results. This may seem intuitive as assessment is often a national tool for quality assurance and provides a standardised structure and criteria that are equal across all schools throughout the system. To date, almost all efforts for establishing a comparative measure of school quality in Sweden have focussed upon the grading system. The biggest issue for this approach in the Swedish context is the variance in performance measures between schools. While this may not be an international limitation, in Sweden at least more work is needed in terms of validating the tasks teachers use with considering Year 9 performance in future studies which consider such grades as a variable.

Measurement of actual improvement
The last and perhaps most interesting issue to discuss is whether these three approaches can actually measure real and purposeful changes within schools. Examining school preconditions for teaching a subject does not provide any insight into meaningful changes in teaching practice as it is only a measure of the improvement in the prerequisites for improved education. Having better preconditions does afford a teacher better opportunities in a classroom for teaching, but it does not necessarily mean these will be used, will be used effectively, or account for teacher's general pedagogical capacity. However it is important that teachers have the preconditions they need in order to teach effectively and previous reports further confirms this (Teknikföretagen and CETIS 2012). Additionally previous research (Hartell 2015) has found that preconditions and affordance for teachers' assessment practices must be enhanced in order to bridge teaching and learning, so supporting teachers with this is of great importance. Therefore, while not a measure of actual changes in teaching, it is considered necessary to ensure that appropriate prerequisites are in place for teaching Technology. Examining subject work plans or other similar planning documents also does not directly measure real changes in educational practices as it cannot be guaranteed that enacted practice will align with planned activities. However, this approach does give insight into intended classroom activity, and into teachers' competence and ability to structure their teaching activities. Therefore, it is arguable that analysing work plans or other similar planning documents is a more useful and sophisticated tool than studying changes in general preconditions, and that assessing work plans could be more relevant as a predictor of potential learning. Examining performances grades appears to be the most direct measure of determining actual changes in the provision of Technology as they reflect student learning. However, this is not always the case. This uncertainty may have arisen due to the fact that the Swedish grading system is difficult to interpret (Norström, 2014) and due to the variability in teachers assessment despite the standardised national criteria. In Sweden it is difficult to measure a change specifically in the Technology subject, as this subject is often not taught explicitly and instead is often taught sporadically in conjunction with Science subjects. The result of this means that a good grade in Technology does not necessary mean that the student is performing well in technology, due to its conflation with Science. Furthermore, this approach does not allow for quick access to evidence during an intervention to determine if efforts are having a positive effect as such examinations are not necessary regularly, or carry such high stakes.

Conclusion
This study was a follow up study for the Boost of Technology intervention, which aimed at improving technology education in Swedish compulsory schools. During this project three different approaches for measuring the effect of the educational intervention where used, analysed and compared in order to find the most appropriate evaluation approach. When triangulating the results, it is clear that these aligned as similar effects could be measured for all three approaches. This suggests that the intervention was successful and had a positive impact on the provision of Technology within the included schools. However, with respect to identifying a single 'best' approach, there remains some difficulty. Table 2 provides a summary of each approach, evaluated under the three criteria previously established.
Examining preconditions for teaching Technology is seen as the most valuable approach if a fast and reliable approach is desired. Evaluating work plans is believed to be the approach that could best predict what is intended to happen in practice, and therefore if they are found to mirror enacted practice they provide the best insight into change in educational provision. Finally, performance grades provide a good measure of change as they can reveal an impact on learning. However, the examinations must be purposeful and analysis of performance is restricted based on the examination schedule. Considering the results of all approaches align, this suggests that enhancing the preconditions and work plans can impact teaching and learning positively. Therefore examining preconditions and work plans may act as a good interim predictor of the success of an educational intervention, but further work with a more controlled intervention is needed to confirm this, and examining student performance can act as a good predictor depending on the duration of an intervention or as a good validation tool at the end. Providing grades are reliable, this appears to give the best indication of learning and impact Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creat iveco mmons .org/licen ses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.


s2fieldsofstudy:
Education
Computer Science

text:
Current The extent to which universities have embraced technology in service delivery: A comparative study of NUST (Zimbabwe) and University of Venda (RSA)

Information and communications technologies (ICT) have greatly impacted the education sector over the years leading to electronic service delivery. Universities all over the world are embracing ICTs especially the Internet, making use of web 2.0 applications, that is, social networks and moving their services from manual based to electronic platforms. The comparative study was done to find out to what extent the National University of Science and Technology (NUST) (Zimbabwe) and the University of Venda (RSA) have embraced ICTs in the conduct of business and how this has impacted on their service delivery. A combination of web content analysis and a comparative study of the web presence of the two universities was done which revealed that the universities are present on some web 2.0 applications though not fully exploiting all the functions. It is recommended that these universities adopt online service delivery to enhance and develop lives of the university community.

Introduction
The advent of computers, leading to the initiation and growth of Information and communications technologies (ICT) especially the Internet, has had an impact on all aspects of life, universities included. This has seen transactions and business shifting from traditional manual basis to electronic transactions, and in the process this has led to what is now known as the global village. Communication has been made easier and people located in different continents or locations can communicate, do business and even complete studies online without physically going to a university campus. Furthermore, university life has been made easier, instead of spending their time queuing to apply for admission, register, get their results and even get books from the library, students can do all these transactions online in minutes. ICTs can be considered as the technology that facilitates the processing, transfer and exchange of information and communication services (Zinyeka, 2004). In this paper, ICTs specifically refer to networked computers, that is, the Internet and or the World Wide Web. The use of the Internet has enhanced service provision of academic institutions. University admission application, registration, campus residence booking, display of examination results and timetables, library services like electronic resources, catalogues and online bookings for books are some services that are now provided online.
The Internet is a mechanism for information dissemination and a medium for collaborative interaction between individuals and their computers without any regard for geographic limitation of space (Leiner et al., 2000;Singh, 2002). According to Evans (1996), content created on the Internet ranges from simple e-mail messages to sophisticated 'documents' (sites) incorporating sounds, images and words. Higher institutions of learning have succumbed to the winds of change and adopted ICTs and this has brought about virtual realities, ease of communication between these institutions and the students and or other stakeholders. Unlike back in the days when students and even prospective students had to make their way to the university's physical location; now, in the technology-centred world, they can interact with the universities from remote locations through the Internet. Furthermore, Obeng (2004) postulates that a lot of services in universities are now provided online, and the use of ICT can to a greater extend reduce the pressure on university admission process and thereby make it more accessible to many. Obeng (2004) also notes that universities are becoming more and more innovative and creative in the use of the Internet to provide services and reach out to a larger community base than they did before the Internet came into being. Another application of ICT is that ideas could be shared online and lecturers could enhance their skills remarkably through the creation of online communities. However, it must be emphasised that the presence of ICT in the education sector is not sufficient, there is also a great need for development of good and relevant quality content (Devi, Rizwaan & Chander, 2001).The impetus is on different departments, units and sections within each and every university to strategically exploit the Internet in order for the services to be flexible, convenient and efficient. Kumar and Kaur (n.d.) highlight that the current information revolution and increasing impact of ICTs has modernised the process, learning and research in most universities. Many universities have adopted ICTs and dedicated resources to the application of ICTs, yet few have really looked at the extent to which ICT have augmented or diminished service delivery. To Tsokota and von Holmes (2013), the power of ICT lies in the unprecedented growth of its world-wide network and the ability to connect computers and several individuals without any of the barriers of geographical space. This paper studies the web presence and use of the Internet in service provision in general by two universities, one in Bulawayo, Zimbabwe and the other in Thohoyandou, South Africa.

Methodology
The research was comparative in nature and also applying qualitative web content analysis, assessing the use of ICTs and or the web by the National University of Science and Technology (NUST) and the University of Venda (UNIVEN). Qualitative content analysis is similar to textual analysis in that it is primarily interpretive in nature, and often does not utilise statistics for data analysis. The researchers visited websites of both universities, analysed the information or content, looked at inferences and similarities. The aim was to identify the extent to which these universities make use of the web to provide services, and to reach out to the community and in the process to enhance service delivery.

Background of UNIVEN and NUST
The UNIVEN, as stated on the university's website, situated in Thohoyandou in Vhembe District of Limpopo Province, opened in 1982 to cater for the then Homeland of Venda. UNIVEN is the only university in the Vhembe District, which covers an area of 21,000 km and is a home to approximately two million people. The NUST is located in the second capital of Zimbabwe, Bulawayo. On the 8th of April 1991, NUST opened for the very first time with 270 students.
The study was comparative in nature, looking at the extent to which each university exploited different Internet applications in providing services to its different clients and generally leading communities in adopting such technologies. These universities have websites and are making use of Web 2.0 applications and other Internet applications which the researchers visited, reviewed and analysed to make a comparison.

Applications used by NUST and UNIVEN
Web 2.0 or 'social media' include blogs, social network Sites (Facebook), online video sharing/posting (YouTube), online photo sharing/posting (Flickr), and microblogging sites (Twitter) inter alia (Gibson, 2002). Staff and students are embracing Web 2.0 technologies because they are so easy to download, install, experiment with and use, in order to quickly engage with colleagues and share the latest information (Kelly, 2010). Trending social media platforms include Facebook, Twitter, Instagram, YouTube, Pinterest, LinkedIn, Tumblr, GooglePlus, FourSquare and Flickr. NUST and UNIVEN are active on Facebook, Twitter, Flicker, YouTube and Flickr. A visit to these applications revealed a presence of these universities in some of these web application technologies and a summary of the findings is presented below.

YouTube
Universities are becoming more and more innovative in exploiting the Internet and reaching to a lot of students and people. YouTube is one such a platform that these universities are making use of. YouTube is a video sharing website that has gained popularity in recent years, and there is a growing trend for making educational videos available as well. Many leading universities are creating 'channels' (organised video collections) for their exclusive use (Cooperative Extension Technology Services, 2002). Universities create channels on YouTube and upload lectures, tutorials, workshops, seminars, graduation ceremonies and other special events that visitors can view or download. A search conducted on YouTtube revealed that NUST and UNIVEN have some of their graduation ceremonies, performances by university choirs, grand openings and other events like the launch of the NUST institutional repository. However lectures and/or presentations and other presentations related to research, academic content for both universities were not present on YouTube at the time of conducting this study. Furthermore, a look at the websites of the universities under study led to useful social media links. NUST has a YouTube channel that it created and it had 33 subscribers and 1,200 followers at the time of conducting the study. UNIVEN's YouTube channel however was generated automatically by YouTube's video discovery system and therefore this means it was not generated by the university for purposes of marketing, outreach or advertising its products and services. Therefore, both these universities need to exploit YouTube in uploading audio-visual material that students and other researchers can make use of. UNIVEN also needs to create a YouTube channel of its own.

Facebook
Facebook is a social network platform that provides interaction usually on an informal basis. UNIVEN and NUST are making use of Facebook in communicating with their students and stakeholders. NUST had 7,361 'likes' and 208 people 'talking about this' whereas UNIVEN had 3,244 'likes' at the time of conducting this study. A look at the 'About' section of both universities reflected that UNIVEN has more information as compared to the NUST 'About' section. UNIVEN sheds light on products they offer, that is, degree programmes, whereas NUST does not provide such information. The UNIVEN 'About' section also includes a description of the university, awards, and some informative detail about the university and its history. These Facebook pages are valuable and used by these universities to make announcements, celebrate achievements they make and other important information that includes links to download application forms inter alia. Also included are deadlines for application and registration.

Presence on LinkedIn
LinkedIn standardises information entered by users into predefined 'Profile Headline', 'Summary', 'Education', 'Company', among other things categories. Schaffer (2009) notes that in addition to this huge database of information, the platform also provides an awesome search tool to allow one to pinpoint the person they are looking for depending on a number of very specific factors. UNIVEN has a link to its LinkedIn profile, which at the time of conducting this research had five members. NUST had neither link to LinkedIn, on its website nor did it have a LinkedIn profile that it created on its own, at the time of conducting this study, the only LinkedIn profile for NUST was created by LinkedIn and is not endorsed by or affiliated with the university. NUST therefore needed to create a LinkedIn profile and UNIVEN needs to grow the number of its members. LinkedIn goes a long way in connecting professionals, and thus, will go a long way in connecting the professionals in these universities and other professionals based elsewhere.

Twitter
Most universities have recognised the importance of Twitter and one study concluded that 84% of universities were on Twitter, and some schools like Harvard have amassed thousands of tweets and hundreds of thousands of followers (Genius Recruiter, 2012). UNIVEN and NUST are making use of twitter as evidenced by their presence on this platform. At the time of conducting this study, statistics of tweets, following and followers for the two universities was as follows: Compared to UNIVEN, NUST seemed to be trending in the number of tweets, following and followers. Therefore, it can be concluded that NUST was more active on twitter than UNIVEN.

Flickr
Flickr is a popular photo-sharing and hosting service which supports an active and engaged community, where people share and explore each other's photos and you can share and host hundreds of your own pictures on Flickr without paying a dime (Lowensohn, 2007). NUST and UNIVEN both have links to their Flickr accounts which are flooded with photographs related to activities taking place in these universities.
There are a number of ways, apart from Web 2.0 applications, in which universities can make use of the Internet to reach out to more people, and even educate and empower remote communities, one such means is by providing online learning.

Distance online learning (online degrees)
In a bid to expand their clientele base and influence, universities have embraced the notion of online learning. With online learning, students can apply, register, conduct lectures, submit assignments, write examinations, get examination results and graduate without physically going to the university. Watson and Kalmon (2005) states that online learning implies education in which instruction and content are delivered primarily over the Internet. The term does not include printedbased correspondence education, broadcast television or radio, videocassettes, and stand-alone educational software programs that do not have a significant Internet-based instructional component (U.S. Department of Education Office of Planning, Evaluation, and Policy Development Policy and Program Studies Service, 2010). The Colorado State University (n.d.) highlights that classes designated as online are delivered to students worldwide via the Internet and require that student have access to an adequately equipped desktop computer or laptop. A number of universities are currently offering online degrees and in the process of reaching out and influencing a lot of geographically dispersed people. Online learning can be especially useful to universities that want to increase their intake, but lack the infrastructure to house a lot of students. Such a facility also aides in providing education to prospective students who are based far from the campus and especially those who are working and do not have the time to attend lectures on campus. NUST and UNIVEN do not offer any online degree programmes. At the time of conducting this study, NUST and UNIVEN had no online learning facility for international students. This therefore means that prospective students have to find their way to their campuses otherwise they cannot get qualifications from these universities.
The NUST website has a link to its online facility for its students who however are not on distance learning or online degree programmes, but for those who are either conventional, parallel or on block release. UNIVEN's Director of the Centre for Higher Education Teaching and Learning, highlighted in December 2013 that the university was introducing e-learning. The e-learning facility is functional at NUST and at UNIVEN and it is being rolled out. At the time this study was conducted, NUST was at an advanced stage as compared to UNIVEN, in line with e-learning.

A study of website content of UNIVEN and NUST
Websites are very important in the 21st century, especially in the global village as they contribute to the visibility of organisations. The researchers also visited and surveyed websites of NUST and UNIVEN to see the content, and view some of the services offered by the universities as reflected on their websites. Areas of foci included information for international students, e-recruitment, online registration and application, library services and display of examination results online. These and other services will be discussed below.

International students' application
The homepages for both universities do not have a lot of information with regards to international students' application. UNIVEN has a page dedicated to international relations which also highlights on attending to foreign students, but both websites are not informative when it comes to international students' application, admissions, information on visas and other requirements. NUST envisions being a world-class centre of excellence in teaching, research, innovation and entrepreneurship for sustainable development. Therefore, there must be a reflection of that in their website, where international student enrolment must be reflected on the website. With some universities, a homepage is usually dedicated to providing information on international students' registration, fees and testimonials from other international students studying in the particular university. However with the two universities under study, that is not the case. One has to search to find links that reflect on international students fees. The public satisfaction in finding the application form, filling it out, submitting it online, and receiving a response, confirmation or feedback is fundamental to the success of the university's e-recruitment (Chiung, Lee & Humphrey, 2010). Both universities have to improve especially with regards to information provision for international students. Liu (2008) highlights that one of the problems most academic institutions have is that their websites lay out information mainly for domestic students. Chiung et al. (2010) highlight that providing general information about the institution is fundamental in e-recruitment and today universities act much like corporations while marketing such products as academic programmes, majors and services. As stated earlier on, universities all over the world have provisions for e-recruitment, where prospective students can apply online using electronic forms guided by automatic responses. Prospective employees can also be recruited electronically, where there are electronic forms and automatic responses to guide the whole application process. This service is of great importance as it eases and smoothens the application process. At the time of conducting this research, UNIVEN and NUST did not have such facilities and that would mean that all persons wishing to apply would have to download forms, complete them and post them back to these universities. Prospective employees could only submit their application forms in a paper form. NUST and UNIVEN both needed to improve on e-recruitment as they move towards efficient and standard service delivery in the information society. Students and other visitors to the university' s website might be in need of external supervisors especially in line with doctorate (PhD) studies and Master of Philosophy (MPhil) studies. A look at the Moi university website showed that in the downloads section it has downloadable curriculum vitae (CV) for academic staff. Prospective students wishing to find supervisors can make use of such CVs in making their choices. Even upon getting supervisors and or external examiners, they can download their CVs rather than always consulting the supervisor requesting for their CV. Such a facility was not provided for by both UNIVEN and NUST.

Promotion of research output
Both NUST and UNIVEN reflect on research, however, during the time of conducting this research, the UNIVEN website had links to different faculties' research output by different academics in each and every faculty. This is of importance as it makes research output at the university visible and visitors of the website will get to credit lecturers and other academics even before interacting with them. Research is of importance and a core function of universities, and thus, such information is of importance to the university first, stakeholders and other concerned parties. A look at the NUST faculty or departmental links did not produce lists of publications. NUST however has NuSpace, an institutional repository that is managed by the library and makes research papers, journal articles, conference papers, accessible free of charge. However, a list of research output by departments as in the case of UNIVEN can go a long way in guiding users who have no time logging onto the NUST library page, then to the Nuspace. In most instances, visitors will not even bother to find out what Nuspace is all about. Most academics at NUST do not submit their works to NuSpace, and thus, a faculty or department list will go a long way in making research output visible.

Accommodation and residence
NUST and UNIVEN enroll students from different parts of their respective countries, and therefore, one would expect information in line with accommodation on and off campus. The UNINEV website highlights on the availability of residence application forms online, whereas this facility is not available on the NUST website. In the technology world, prospective students and other parties concerned must not be kept in the dark about some of these key necessities like accommodation. Some students would be coming far from places that are remote, without any relatives close by and they must be made aware of accommodation arrangements, bookings and availability even prior to their arrival at the physical location of the university campus.

Notice on living expenses in and around campus
A look at a number of developed universities' websites revealed that they include details of living expenses in their respective cities. This goes a long way in assisting students budget and gauge, if they can afford to survive for the period of their study. Such estimates and information were not made available on websites of both these universities under study. Students usually register and upon arriving at the universities discover that living expenses are very high, and this contributes to dropouts or even students failing to cope and dropping out or even underperforming. Such relevant information is not made available in websites of both universities.

Services for staff members
The Internet has brought about convenience to the workplace in remarkable ways.The download section of the Moi university website has important downloads for staff members. These downloads include all leave forms, claim forms, medical forms and accounts related ones. Instead of members of staff going all the way to human resources or departmental offices to get these forms, they can download them online. ICTs must make life easier and avoid monotonous and unnecessary procedures. A look at UNIVEN and NUST website revealed that both universities do not have such downloadable forms that members of staff can utilise. The human resources departments of these universities can learn from the Moi university human resources department which has taken advantage of ICTs to automate some of its services.

Library services
Libraries are using modern ICT to automate their core functions, implement efficient and effective library cooperation and resource sharing through networks (Tiwari & Sahoo, n.d.). Back in the days, students had to be physically present in a library to interact with finding aids, make bookings for books that were usually scarce. ICTs have brought about changes to that as students can consult the library and interact with resources therein from remote locations. This has seen in the rise in subscriptions by university libraries and usage of online journals and e-resources University libraries for NUST and UNIVEN have library catalogues online and links to journals they subscribe to. With this development, students, academics and researchers can access a lot of journal articles and study material. Libraries for both universities are present online and are developing services to meet different clients and stakeholders. The libraries are not as virtual as other developed countries' libraries are, but they provide sources through e-resources and e-journals online. The libraries have catalogues which users can interact with online before. Furthermore, NUST has the Nuspace institutional repository, which accommodates publications or research done by the NUST family, and its digital library which has links to dissertations and theses, NUST publications inter alia.

Conclusion
The research sought to find out web presence of NUST, a university in Bulawayo, Zimbabwe, and UNIVEN, another university in Venda South Africa. The websites of these universities were visited and searches on the Internet were done to find out their presence on different Internet platforms and their presence in social networks. The results showed that these universities were adopting electronic service provision such as e-learning, provision of downloadable application forms, online registration inter alia. UNIVEN and NUST are also participating in social networks, though not fully exploiting these platforms to further their mandates of educating and imparting knowledge on their communities. The study concluded that more and more departments within these departments, for example, the human resources department must also exploit such technologies in a bid to make life easier for prospective employees and its employees. Information for international students including online application was lacking in both universities, and therefore, they needed to improve on that note. A look at websites of both universities also showed that recreational activities like sports were not represented enough and had no homepages nor images dedicated to them.


Technical and tactical readiness of the team "Helios" (Kharkiv) in the 26th Ukrainian soccer championship in the first league

Purpose : to determine the model characteristics of the technical and tactical readiness of the team that participated in the Ukrainian championship of the first league in order to further improve and correct the training process. Material & Methods : the data were obtained using the method of expert evaluation, the calculation was carried out using the methods of mathematical statistics. Results: analyzed the average values of the registered values for 19 games. Analyzed various technical and tactical actions and their differences in the first and second halves, as well as individual indicators of the game players and the team "Helios" (Kharkiv). Conclusion: quantitative and qualitative (rejection rate) indicators were obtained both for team technical and tactical actions, and separately for each technical and tactical method for each game period.

introduction
A great contribution to the construction of model characteristics of competitive activity in football was made by domestic experts [7; 9; 12]. Football experts analyze technical and tactical readiness from young football players [2; 6; 8] to football veterans [11]. Some experts are exploring certain indicators that, in their opinion, lead to a positive result of the game. Among them are: one-touch ball transfers in various zones of the football field [13], ball delivery in the opponent's penalty area [10], etc. There are some studies of the competitive activities of highly skilled football players (Ukrainian Football Premier League), which were conducted with the same the same team for a long time [15]. However, there are practically no longitudinal studies of technical and tactical preparedness of the teams of the First League of Ukrainian football. [3][4][5].
In our time, when developing quantitative indicators characteristic of a given level of sportsmanship, various approaches can be distinguished [14]. We used the approach associated with the study of a significant set of athletes of different qualifications, establishing the relationship between the level of sportsmanship and the dynamics of changes in a particular indicator.
purpose of the study: to determine the model characteristics of the technical and tactical readiness of the team that participated in the Ukrainian championship of the first league in order to further improve and correct the training process.

Material and Methods of the research
Studies were conducted using the method of expert evaluation. As experts were involved 5 football professionals. Among them: one master of sports in football, one -candidate master of sports, the rest were players of professional football teams.
All the experts in the past worked with professional and amateur football teams as coaches. From among experts: two professors, one candidate of pedagogical sciences, associate professor; two candidates of sciences in physical education, associate professors of the Department of Football and Hockey of the Kharkov State Academy of Physical Culture. If during the registration of the competitive activity of the Helios team in Kharkov there were debatable questions, they were decided by a majority vote. During the pedagogical observations, the methodology itself assumed mutual control over the indicators of competitive activity, which made it possible to obtain more objective data. So, one of the specialists counted the total number of transfer, and the other, at the same time, slandered the dictaphone which specific players (player number) and which was transfer in the direction and distance.
During the development of model characteristics, the mean -Х, was calculated, the sample mean error (the error of representativeness) was ± m. The significance of differences was determined using Student's t-criterion according to the generally accepted method [1].
Scientifically-methodical group at the team "Helios", Kharkov, 19 games were registered in their field. In the matches of 26 championships of Ukraine in the first league 18 teams participated. Players "Helios" had 34 games, got 16 wins, 10 -draws, 8 -defeats, the goal ratio was 31-22,58 points scored.

results of the research
This study presents averages that were obtained for 19 home games in the 26 championship of Ukraine in football among the teams of the first league. Thus, the total number of technical and tactical actions was 644,98±27,91, while the reject rate when performing these actions was 35,86±2,01%. During the first half, the players of the Helios team performed SlobozhanSkyi herald of Science and Sport 338,15±17,45 technical-tactical actions with a rejection rate of 37,48±2,99%. In the second period, the players did 301,4±14,64 technical and tactical actions, with a rejection rate of 40,47±3,03%. Significant differences in these indicators we have not identified (Table 1). Significantly less (t=3,2; p<0,01) of interceptions in the second period were performed by players of the Helios team in Kharkiv.
We also calculated the greatest number of technical and tactical actions of all players of the "Helios" team per match (the activity coefficient was also calculated). Thus, the greatest number of technical and tactical actions of the players was 82,0±3,26, while the reject rate in these players was 33,48±2,04%. The average total number of pass that were performed by the players of "Helios" for 19 games amounted to 348,0±14,16 ( Players of the team "Helios" on average performed per game back and across passes -111,42±7,08, with a rejection rate of 18,18±1,41%. The percentage of the number of passes in the total number of technical and tactical actions was 53,22±1,06%.  We also registered the attacking and defensive actions of the "Helios" team and the opposing teams that the Kharkiv team met with (Table 3). Not only quantitative indicators were calculated, but also qualitative ones (efficiency indicators,%). Attacking actions are divided into successful, foiled and all.
Were also registered (among all attacks) the number and effectiveness of penetrating attacks.
There were no significant differences in the indicators of attacking and defensive actions between the first and second periods.

conclusions / discussion
The   works of specialists, there is a desire to find in indicators of technical and tactical readiness those indicators that significantly affect the positive outcome of the match. Among them are the works of A. A. Pertsukhov on the game with one touch in different zones of the football field, V. V. Mulyk and co-authors of the delivery of the ball to the opponent's penalty area. However, they do not consider the dynamics of changes in technical-tactical readiness indices in longterm studies of the same contingent. V. M. Shamardin is close to our research, which he conducted with the team of the highest league "Dnepr" in Dnepropetrovsk, however, we chose a contingent of subjects different, that is, the players of the first league of Ukrainian football.
The results of our study show that the dynamics of changes in the technical and tactical preparedness of the team players influenced a positive result in a particular game and ultimately a place in the standings.
The studies were conducted with the aim of correcting the training process of the "Helios" team in Kharkov, selecting methods and means during training and achieving a high level of technical and tactical preparedness of the team players.
further research will be aimed at obtaining model characteristics of the technical and tactical readiness of the "Helios" team in Kharkov and their comparison with the teams of the first league of Ukrainian football and those teams that participate in the games of the Ukrainian Premier League on football.
conflict of interests. The authors declare that no conflict of interest. financing sources. This article didn't get the financial support from the state, public or commercial organization.


The teacher's role in increasing employability skills of the drawing building engineering vocational high school students

The process of preparing students on vocational education that includes the development of skills, competency, understanding, attitudes, work habits and the appreciation needed by workers to enable them to work successfully in their fields should be of interest to educators in vocational education institutions. Early observations of science and the emphasis on authentic assessment suggested that the 2013 curriculum for vocational high school is a renewal or innovation of the previous curriculum. The research is done through descriptive-qualitative re-search method. The population in this study is all teachers of vocational subject vocational competence of building drawing techniques in Bantul Regency. The technique of collecting data through an interview, observation, and documentation, data is analyzed to get a description about research object. The results showed that the selection of strategies and learning approaches should be studied closely, especially in relation to their relevance to learning missions, learning objectives, students' potential and characteristics, and environ-mental resources. Vocational teachers have at least four important roles in developing students' employability skills, namely the role of teachers as demonstrators, as mediators, as evaluators and as competent individuals. Implementation of the 2013 curriculum learning on the competence of building drawing techniques from several previous studies can be concluded to be implemented adequately.

INTRODUCTION
The current role of vocational education, including Vocational High School (SMK), is not just limited to provide workers, it must also be able to prepare students to become selfreliant and responsible societies. Preparation is the adoption of a holistic approach from the aspects of knowledge, skills, and attitudes towards the development of employability skills. Forming alumni into independent and responsible community members with educational programs that emphasize value development, ethics, and behavior. SMK should be able to increase the competence of students at a higher level. The training helps the trainee to capitalize his interests and abilities to the highest possible degree (Prosser & Quigley, 1957).
In line with the statute of Law number 20 of 2003 on the national education system which states "vocational education is secondary education that prepares students primarily to work in a particular field". According to Thompson (1973), the American Vocational Association has defined vocational education as education designed to develop skills, abilities, understandings, attitudes, work habits, and appreciations needed by workers to enter and make progress in employment on a useful and productive basis. The process of preparing students for the vocational education that includes the development of skills, competency, understanding, attitudes, work habits and the appreciation needed by workers to enable them to work successfully in their field must be the role of educators in vocational education institutions.
According to Law number 14 of 2005 regarding Teachers and Lecturers Article 4 asserts that teachers as learning agents serve to improve the quality of national education. The function and role that is very strategic in the development of education are in the teacher's capacity, therefore need to be developed as a dignified profession. Pedagogical competence is one of the requirements for teachers to teach well. The government and by various parties who care about learning in school has done of Improved quality of learning programs, include: (1) teacher upgrading; (2) teacher education qualification; (3) curriculum renewal; (4) implementation of new learning model or method; and (5) research on students' difficulties and mistakes in learning frequently done by teachers such as classroom action research.
The curriculum 2013 is also dedicated to link and balance soft-skills and hard-skills and accommodating the principles of contemporary learning (Hadiprayitno, 2016). Some of the principles of innovative learning in the Curriculum 2013 which is a paradigm shift from the previous curriculum are: (1) teachers are not the only source of learning so learners learn from various learning sources; (2) students are facilitated to find out rather than be in-formed; (3) the learning process uses a scientific approach; (4) learning that emphasizes divergent answers that have multi-dimensional truths; (5) utilization of information and communication technology to improve the efficiency and effectiveness of learning, as stated in Chapter I of the MOEC regulation number 65 of 2013.
Efforts to build and revitalize vocational education are also contained in Presidential Instruction number 9 of 2016 (Setneg, 2016) and on September 6, 2017, President Joko Widodo has signed Presidential Decree number 87 of 2017 on Strengthening Character Education (Setneg, 2017). The Presidential Decree for Strengthening Character Education is an education movement under the responsibility of educational unit to strengthen the character of learners through harmonization of the heart, taste, though, and sport with involvement and cooperation between education unit, family, and society as part of the National Movement of the Mental Revolution.
This Presidential Decree has the objectives: (1) to equip students as the gold generation of Indonesia in 2045 with Pancasila spirit and good character education in order

15
to face the dynamics of future changes; (2) to develop a national education platform that lays character education as the main spirit in education for the learner with the support of public engagement through formal, informal, and informal education with respect to Indonesian cultural diversity; and (3) The Presidential Regulation also aims to revitalize and strengthen the potential and competence of educators, education personnel, students, community, family environment in implementing Strengthening Character Education.
The results of research from Abbas, Kadir, & Azmie (2013) shows that 75% of job success is determined by soft skills and only 25% is determined by hard skills. According to Lankard (1990), educators at vocational schools should adapt to change and information, in developing and educating students especially in vocational schools. Knowledge of employability skills becomes important owned by SMK teacher so that SMK graduate students also have this ability. Employability skills as a skill that al-lows one to get a job or to keep working, including personal skills, interpersonal skills, attitudes, habits and behaviors (Overtoom, 2000).
Responding to the demands of employability skills for SMK graduates, regulation of the Minister of Education and Culture number 23 of 2016 asserts that the Curriculum 2013 re-quires the use of authentic assessment (MOEC, 2016). Paradigmatically authentic assessment requires the realization of authentic instruction (authentic instruction) and authentic learning (authentic learning). Gulikers, Bastiaens, & Kirschner, (2004) defines authentic judgments as judgments requiring learners to use the competencies of knowledge, skills and behaviors they master and apply in solving their professional life problems later, where the level of authenticity of a judgment will depend on the level of resemblance to the situation it will face in the real world. Thus, authentic assessment is very relevant to the characteristics of learning in SMK that must prepare graduates to be able to work in the real world of community life in a professional manner in the field of expertise.
The characteristics of education to be undertaken in SMK is to equip the learners with a variety of cognitive skills and vocational-technical skills and equip it with various soft skills (attitude, soft skills, employability skills, or generic skills) required in work (Sudjimat, 2009). The results of the research (Hadiprayitno, 2016) show the response of teachers of building engineering program of SMKN in Yogyakarta to the principles of learning Curriculum 2013 is "Agreed" with approval level of 76, 25%. This means that teachers believe that the principles of learning according to the Curriculum 2013 will be implemented to produce more competent graduates in building engineering skills.

RESEARCH METHOD
Research method on employability skills development analysis on the competence of building engineering skill of SMK in Bantul district. This research is conducted through descriptive-qualitative research method, which is a method that observes, analyzes and describes the phenomenon that occurs in learning on building drawing competence in Bantul then explores the data of each element, activities include: planning, organizing, actuating, con-trolling and factors which have contributed to the coaching of teachers' professional abilities. Researcher collect data in the form of interviews, documentation, and observations, the data and information obtained are then organized and analyzed in order to get a description of the object of research. The data will be analyzed more deeply to form a natural scientific conclusion can be accepted by various circles, especially in this case is Vocational High School with the competence of Building Materials Engineering skills.

RESULTS AND DISCUSSION Results
The Result of research are (1) profile of the working period of the teachers of drawing building engineering at SMK was shown in Table 1. The teachers' education level Bachelor as many as 7 people, Master as many as 5 people.
(2) the number of students SMK N 1 Sedayu, class X = 59, class XI = 54, class XII = 31 for students SMK 1 Display class X = 57 classes XI = 68 Class XII = 67 average graduates last three years more than 50% work according to drawing building engineering competency (3) Teachers as a demonstrator. The interview result showed that as a teacher, he or she needed self-development as an effort to improve ability and competence herself. The learning process approaches according to the demands of the curriculum. (4) Teachers as mediators and facilitators. He or she needed to learn and practice of development of science and information to understand and master them. Mastery of instructional media is needed as a complementary tool and learning resources for students. Teacher selfdevelopment is done to improve the communication skills, there are activities undertaken by teachers to encourage good social behavior, develop personal interaction style, and foster positive relationships with students. Including the development of employability skills students. (5) The teachers as an evaluator. Various evaluation or evaluation activities are intended to know whether the objectives that have been formulated are achieved or not, and whether the material taught is sufficient. The form of objective assessment that is mandated by the curriculum encourages the teacher to be able to photograph the students' overall competence in both technical skills and employability skills (basic skills / Thinking skills / personal skills) of the students. (6) Teachers as Competent Persons, teachers always become good role models for students, a measure of norms of behavior in the eyes of students. Teachers fully master the competence that is taught both knowledge and attitude skill.

Discussion
Labor users often complain that workers from graduates of educational institutions who do not possess good employability skills generally cannot stand the world of work are dishonest, get bored, unable to work together, and cannot communicate orally and write well. School as one of the instruments and tools of the state as an organization used to achieve educational goals. The role of vocational school prepares students to work in a particular field. The above data shows the role of SMK in encouraging economic growth by continuing to push the unemployment rate.
The purpose of vocational education cannot be separated from the role of teachers in running the curriculum set by the government. Teachers have a role to facilitate the active learning process that can generate interest and ability of students in optimizing their potential. It is important for teachers to have competencies and act effectively as one of the keys to the success of a learning process. Teachers and instructors in the 21st century are teachers and instructors who have learning skills and innovative skills in the field of study and pedagogy/andragogy field (Sudira, 2015). Teacher quality refers to all teacher-related characteristics that produce favorable educational outcomes (Cochran-Smith, 2005).
The preparation of students to have the technical skills and skills that are generic (employability skills) stem from the quality of the implementation of the learning program (Wu, 2005). The results from several studies indicate interacting factors in the learning process including learning system. Educators must have the capacity and capability in understanding the learning model, thus providing opportunities for students to develop their maximum learning creativity and train their independence. Education with learning how to learn every student should be encouraged to have the ability and understanding of how to learn and be able to continually explore information for strengthening self-capacity (Hardika, 2014). The selection of strategies and learning approaches should be carefully examined, especially in relation to their relevance to learning missions, learning objectives, students' potential and characteristics, and environmental resources.
The opinion of Rasul, Rauf, & Nor (2014) concludes that the major skills required by the industry and firm are strong interpersonal skills, such as communication skills; problem-solving skills; teamwork and entrepreneur skills; and project and work process oriented. Principles of learning in the implementation of Curriculum 2013 for SMK drawing building engineering competency has accommodated the demands of innovation skills and employability. The scientific approach to learning and the emphasis on authentic assessment recommended in the Curriculum 2013 for Vocational Education is a renewal or

18
innovation of the previous curriculum. Furthermore, the 2013 curriculum is intended to balance soft-skills and hard-skills and accommodate the principles of present learning, so that teachers have an important role in developing basic skills, thinking skills and guarding the development of personal skills of students.
The authentic self-assessment model can be developed to assess the cognitive, affective, and psychomotor aspects. Problem solving and self-management are part of the cognitive and affective structure of the learning process. Meanwhile, a psychomotor structure is shown by the ability to make database application conceptually and logically. This model has met the basic principles of appraisal, namely the principle of validity, the principle of reliability, focused on competence, comprehensive, objectivity principles, and educational principles. Based on the results of the trial can be concluded that the ability of problem solving and self-managing ability of students in both categories, as well as competence achievement to design database applications conceptually and logical including high category.

CONCLUSION
The results showed that the selection of strategies and learning approaches should be studied closely, especially in relation to their relevance to learning missions, learning objectives, students' potential and characteristics, and environmental resources. Vocational teachers have at least four important roles in developing students' employability skills, namely the role of teachers as demonstrators, as mediators, as evaluators and as competent individuals. The implementation of the 2013 curriculum learning on the competence of building drawing techniques from several previous studies can be concluded to be implemented adequately.