title: Report on the Evaluation of the Community Action Plan to Assist Tourism 1993-95 - Council Decision 92/421 EEC

eurovoc_concepts:
EU policy
action programme
project evaluation
tourism

text: COMMISSION OF THE EUROPEAN COMMUNITIES Tir & <r -6- Brussels, 30. 04. 1996 COM(96) 166 final R E P O RT ON T HE E V A L U A T I ON OF T HE C O M M U N I TY A C T I ON P L AN TO A S S I ST T O U R I SM 1993-95 C O U N C IL D E C I S I ON 92/421EEC (presented by the Commission) I INTRODUCTION 1. The three year Community Action Plan to Assist Tourism, agreed on 13 July 1992, recognised the importance of tourism and set out a framework to enable the Community to specifically assist its development1. The Action Plan to Assist Tourism (hereafter 'the Plan') broadly stated that Community action should: strengthen the horizontal approach to tourism in Community and national policies; implement specific measures; and coordinate the measures undertaken by other Commission departments. The Plan recognised the Community's role in improving the quality and competitiveness of tourism and that a joint approach to tourism by the Community and the Member States could promote the wider development of the Community in both economic and social terms. 2. The purpose of this document is to report on the evaluation of the Plan. This is pursuant to Article 6 which states that the Commission shall undertake such an evaluation and report to the Council and the European Parliament. The date specified in Article 6 for the production of this report was 30. 6. 95. While this timetable has not been met the delay has allowed for a more comprehensive evaluation of the Plan than would have been the case had the original timetable been followed. Even taking account of this delay however the external evaluator was only able to fully evaluate the results of the first two years of the Plan, which represents 55% of total expenditure commitments made under the Plan. Actions financed in the second year of the Plan were in general only completed at the end of 1995 while most actions financed in 1995 were started at the end of that year. 3. This document responds to an independent evaluation of the Plan undertaken by external consultants. Their final report was accepted by the Commission on 26. 2. 96 and has the following structure: Summary of Findings and Key Recommendations Section 1 Introduction Section 2 Programme model of the Plan Section 3 Technical evaluation of actions Section 4 Administrative and operational aspects Section 5 Assessment by contractors Appendices 4. The Commission responds here to the Consultants' Report using the framework developed by the Consultants in the first part of their Report entitled 'Summary of Findings and Key Recommendations'. All the points discussed in detail in the body of the Consultants' Report are contained in that Summary, and are set out in the following order: The Contents of the Programme (Paragraphs 2-6 / pg. i) The Role and Market Positioning of the Tourism Unit ( Paragraphs 7&8 / pg ii) 1 Council Decision 92/421/EEC, OJ L231 of 13. 8. 92 Compliance with Article 4 of the Action Plan and Achievement of Objectives (9-12) Technical Evaluation (13-33) Administrative and Operational Aspects (34 - 41) Key Recommendations (42 - 60) 5. 6. Sections II to VII of this document therefore deal sequentially with all the issues raised by the Consultants in their 'Summary of Findings and Key Recommendations'. Where reference numbers are given below these relate to paragraphs contained in the Consultants' Summary although more detailed comment can also be found in the later sections of their Report. Three annexes complement this Report. Annex 1 gives supplementary information on the Plan and its administration; Annex 2 gives a breakdown of the expenditure commitments made under the Plan; and Annex 3 is the Consultants' Report. The Consultants were not called upon to specifically investigate the allegations regarding the conduct of affairs in the Tourism Unit in the period prior to March 1994. Enquiries by the Commission, the Court of Auditors and other authorities are still proceeding. The Consultants were, however, asked to bear in mind the effect which these allegations and enquiries had on the overall effectiveness of the implementation of the Plan. II THE CONTENTS OF THE PROGRAMME 7. The Commission welcomes the evaluation undertaken by the Consultants and agrees with the substance of many of the issues identified. This is particularly the case in respect of the Consultants' analysis of the contents of the programme. Paragraphs 2 to 5 outline weaknesses in the structure of the Decision which caused difficulties throughout the lifetime of the Plan. These include the following: i) ii) iii) iv) v) vi) fragmented and incoherent Measures (set out in the Annex of the Plan); confusion between ends (or objectives) and means (the measures); lack of clarity on how some of the actions would contribute to wider objectives; incomplete or overlapping measures; over ambition given the available resources, lack of continuity over the 3 year period. 8. The Commission broadly agrees with these criticisms. The Commission views the Plan however as the outcome of a long discussion which resulted in a number of competing priorities. Tourism policy as such does not exist and a specific tourism 'strategy' at EU level is at a formative stage. The differing views and priorities of the Member States; the complex nature of the different interests associated with tourism; the lack of previous specific action at EU level; the subsidiarity principle; and the requirement for unanimity within the Council all served to shape the Plan. The Commission accepts that the measures would have benefited from further analysis by both Commission and Management Committee before implementation. This proved difficult due to its relatively short-term nature and the constraints imposed by budgetary rules applicable during the first year (differentiated appropriations). The Commission will take account of all the aspects noted by the Consultants in the drafting of any new programme. In particular there will be: • an unambiguous and logical link between objectives and measures; • • a reduced number of clear and focused measures; realism given available resources; • clear added value to existing actions taken by both Member States and other Commission services; • coordination with other Community actions; • respect for the principle of subsidiarity; • a logical work programme to ensure continuity of actions in T HE ROLE AND " M A R K ET POSITIONING" OF THE TOURISM UNIT 9. Paragraphs 7 and 8 of the Consultants' Report refer to the role and mandate of the Tourism Unit. There was, in the view of the consultant, a lack of market positioning. The Commission is in broad agreement that this remit should be defined with greater focus and clarity in the future but would also point out that role and mandate is dependent upon the content of any Decision which is agreed. Certainly the experience gained with both the Plan and the consultation on the Green Paper on the role of tourism in the Union2 has assisted with the definitions of any future role and function. A number of specific options are given in the Consultants' Report on the type of role which could be undertaken by the Tourism Unit. The Commission will ensure that the Tourism Unit concentrates mainly on the specific role of ensuring greater coordination of all Community actions affecting tourism. This will be achieved by the: • further development of internal coordination mechanisms; • active cooperation and partnerships with Member States and the tourism industry; • regular identification o£ and consultation on, specific issues of concern. COM (95) 97 of 4. 4. 9 5 IV COMPLIANCE WITH ARTICLE 4 AND ACHIEVEMENT OF OBJECTIVES 10. Paragraphs 9 to 12 of the Consultants' Summary state that the objectives (contained in Article 4 of the Plan) are at a high level and sometimes vague. The Consultants identify that the 11 measures ("Policy Areas") were never clearly linked to these objectives which were both high level and too broad. In addition it is apparent that intermediate objectives and performance indicators were either missing or were poorly defined. The Commission is in agreement with this analysis and accepts that the nature of the Plan made it difficult to implement. The Consultants recognise that the attainment of these high level objectives, given both resources and framework, was not realistic. 11. The Consultants note however that particular initiatives were successful, such as work on statistics and coordination, and that these successes made a contribution to the attainment of the intermediate objectives expressed in the Plan. The 'Programme Analysis' undertaken by the Consultants also reveals that objectives were confused with mechanisms, particularly in the case of Transnational Measures' which featured as a measure in the Annex. This analysis revealed the value of a continuous evaluation of the objectives. Such an evaluation includes analysis of the objectives in terms of their coherence and links to performance indicators and targets. This analysis will be undertaken by the Commission on an ongoing basis. Based on the lessons learnt the objectives of any new programme will be set out in a hierarchy as follows: • Long term objective. • Medium term objectives • Short term objectives linked to tasks • Specified performance indicators enabling progress towards the objectives to be measured • Targets to measure effective completion of objectives V TECHNICAL EVALUATION 12. Paragraphs 13 to 33 of the Consultants' Summary show the results of the technical evaluation. This included an assessment of all projects funded in 1993 and '94 and 10 case studies. The main points noted by the Consultants on the technical side are that: The quality of projects was low in 1993 but improved significantly in 1994; i) ii) The impact of actions was limited in space, scope and time; iii) The work on statistics was successful; iv) Studies were usually well executed although lacked clear analysis of needs; v) Built in post-project sustainability was often not well considered and some projects appear to have been largely designed to access EC Funds. vi) The commercial application of some projects does not appear to have been well thought through; vii) Publications relating to tourists with disabilities have produced excellent results; viii) Pilot projects on culture, technology and training have produced varied results, ix) The work on coordination made a positive contribution to reaching certain intermediate objectives, SMEs were not well enough considered in some actions, such as technology. x) These 10 findings are considered separately below. 13. The Commission is confident that the methodology employed by the Consultants has resulted in an accurate technical assessment. The Commission recognises that project quality was low in 1993 and welcomes the fact the quality of projects improved significantly between 1993 and 1994. It is also to be recalled that the Consultants have judged that a further improvement is likely with the 1995 work. The Commission believes however that there are a number of presentational points in the Consultants' Report which may not always result in the most accurate view being given. For example the data presented in Table 3. 1 (pg. 13) does not correspond with comments made in 65 (pg. 17). Here it is stated that the quality of projects is 'poor to average' while Table 3. 1 indicates that this is not the case and the number of good or very good quality projects is significant, reaching 44% in 1994. 14. The Commission notes the Consultants' view that the impact of actions was limited in space, scope and time and accepts the principle that a smaller number of larger actions is desirable. However the Consultants point out that the number of commitments made were too many in relation to the size of the Unit and this statement largely fails to take into account that many commitments were related to the same subject using standard contracts where the project management is reduced due to their similar nature. For example 40 of the 75 commitments in 1995 related to statistics and many of these were made under identical conditions to each Member State. 15. The Consultants also comment in 16 that actions taken were too small to make a significant impact It is not however suggested at what level of funding the actions would become more 'significant'. This comment is then partially contradicted with a recognition that some small actions have been successful and cost-effective. The Commission views this statement as one which is rather simplistic (given the many variables which can impact on project effectiveness) and believes that certain types of small action can be very significant due to the multiplier effect. 16. The Commission notes the Consultants' judgment that the work on statistics has been successful. This work, undertaken via a partnership with the industry, resulted in the agreement of a Directive on the gathering of comparable data at Member State level across the EEA3. The fulfillment of this widely recognised need has been a significant achievement. Further work on this topic will continue over the next years. 3 Council Directive 95/57/EC of 23. 11. 95 (OJ L 291 of 6. 12. 95) 17. The Consultants note that studies were usually well executed although lacked a clear analysis of needs. (Here 'studies' are defined as research work which is 100% funded by the Commission). The Commission accepts that this was the case in some circumstances and that this was partly a because of shortcomings with the Plan itself already noted. Additionally the Consultants' Report acknowledges that studies have been agreed or proposed by the Management Committee and usually. discussed with the sector. The procedure for proposing and selecting studies has been modified in such a way that after 1994 only studies which were based on a clear assessment of requirements and outputs have been progressed. 18. The Consultants note that there was a lack of built in post-project sustainability with some projects. The Commission would make the following points. Projects of the type requested are usually demonstrative, aiming to encourage others to do similar work. They should however be suitable to become longer term actions; Innovative projects are more likely to have a higher failure rate than other forms of intervention due to the nature of the action; Many of the impacts of the projects will occur in the medium term and have not been picked up in the evaluation, which examined directly measurable outputs; Many of the impacts are dependent on effective publicity which will be continued. This said the Commission is clearly concerned to ensure that the impact of its financial support is as effective as possible. The 1995 series of projects has taken on this issue and revised selection procedures have improved the quality of projects selected. The Consultants note that reference publications can become out of date rapidly and the Commission agrees that such publications should have sustainability built in. The Consultants also judge that some projects appeared to have been designed to "access" EC funds. This refers to a small number of projects selected in 1993 which were subsequently shown to be of poor quality. The Commission agrees with this and would state that no final payments have yet been made on these projects pending further assessment. 19. The Consultants note that the commercial application of some actions do not appear to have been fully thought through. Earlier in paragraph 18 the Consultants divide actions into those which promote 'European' concepts and those which are 'commercially interesting'. While the Commission agrees that commercialisation can be desirable it does not hold the view that this was the main goal of the Plan. Indeed networks and other initiatives which promote information exchange and greater knowledge are vital and cannot always be made immediately commercially feasible. Aspects such as accurate European-wide statistics and specific research are preconditions to ensure the industry's success. There is also a clear role for the public sector in certain areas, such as the environment, where the use of subsidy to ensure the conservation of public goods is widely accepted. 20. The Commission notes that the publications relating to tourists with disabilities have produced excellent results. The Commission would also add that a variety of other initiatives, such as the tourism and environment prize, and the coordination function with other Services have also created a significant and positive impact. 21. The Consultants state that the pilot projects on culture, technology and training have produced varied results. These fields of activity were those with the most projects and represented considerable expenditure, the details of which are set out in Annex 2. The Commission would question the Consultants' view stated in the body of the Report that the cultural projects have reduced or diverted demand from other areas. This statement cannot be made with great accuracy at present and may fail to take into account overall and niche market growth. The Consultants note that certain technology projects have had a positive impact in accelerating the application of new technology. The Consultants comment on the 'one-off training actions undertaken in Central & Eastern Europe (27) and accept that they may have been effective but were considered rather in isolation. Nevertheless the Commission considers this type of training to be of benefit and holds the view that adequate coverage in other similar regions is as important as follow-up in the region where training has taken place. The Commission accepts however that there remain issues to be examined relating to dissemination, post project follow-up and longer-term impacts. Further dissemination of results, continuing with partnerships where appropriate will create publicity and encourage follow-up action. 22. An area positively evaluated is that of coordination of other Commission policies with tourism objectives (see Paragraph 11). This work is seen by many as the most important aspect of the Plan. The results of this wide ranging task have been varied but specific initiatives such as the reports on matters affecting Tourism4 and a number of working partnerships with other services have been particularly successful. An example is the coordinating group on tourism for people with special needs, which served as a link between the Plan and the Helios II programme. An area where the Commission still needs to increase the direct influence of the Tourism Unit is on other established Community programmes, which often have a substantial tourism component. In the interests of improved coordination the Commission is now holding more specific and thematic meetings with the sector where policy implications can be discussed with the services responsible and independent experts. 23. The Consultants note in 33 that one issue which was not adequately addressed was the relevance to SMEs of the technology projects and other actions. While recognising that tourism SMEs have not been the subject of a specific call for proposals, the Commission would recall that some work, such as that on 'Yield managment in Small and Medium sized Enterprises in the Tourism Industry', did seek to examine how the application of technology could be applied by SMEs. 24. One major action on Promotion in Third Countries was launched in 1993 at a cost of 1 MECU. The action consisted of the development of an image for Europe and a publicity campaign. The concept was to present various common European themes under one 'umbrella'. The project suffered from a lack of coordination with other campaigns run presently by the European Travel Commission, (ETC) a representative organisation of various National Tourist Organisations (NTOs). While the technical quality of the work was COM (94) 74 of 6. 4. 94 & COM (96) 29 of 5. 2. 96 high, the value for money as measured by direct responses was low. This was largely due to lack of sponsorship, which had been anticipated in the original proposal, but which was not forthcoming. Second, in 1994 a market research project was jointly funded. This consisted of research on the demands of the major and emerging markets outside Europe and was recognised by the Consultants as a useful addition to current knowledge in this area. Third, in August 1995 a call for proposals was published relating to a promotion action in Japan. It had been hoped to transfer the necessary funds from the general budget line of the Plan or finance the action from 1996 funds. In the event neither of these options was possible and the campaign was not implemented, although the required sponsorship funds had already been guaranteed. All the recommendations made by the Consultants on promotion will be taken into account in future actions. In summary the Commission: • welcomes the recognition that significant improvements in project quality were made over the period of the Plan; • accepts that a smaller number of larger projects is a sound principle; • accepts that greater account should be taken in any future activities of the analysis of study needs, project sustainability and commercialisation; • will take account of the points made on the technology, training and cultural projects in order to ensure that they provide greater impact and added value within any further specific actions; • continues to pursue work on the dissemination of project material; • has already, or intends to, recover sums paid to certain projects. VI ADMINISTRATIVE AND OPERATIONAL ASPECTS 25. In March 1994, following an inquiry into affairs in the Unit by DG XXIII and a subsequent internal audit by the responsible Directorate General, the Head of the Tourism Unit was transferred to another service and subsequently suspended. In July 1994 a Temporary Agent was also suspended and both these members of staff were later dismissed. Following further work by the Commission's anti-fraud service these matters were later referred to authorities in a number of Member States which are now conducting an investigation into allegations of fraud. The results of these actions had great significance, both in terms of the administration of the remainder of the Plan and for the quality of some of the work which began in 1993. 26. Paragraphs 34 to 41 of the Consultants' Summary show the findings of the administrative evaluation. From this the following is apparent: i) Administrative arrangements and procedures were weak at the start of the Plan but have significantly improved; ii) The administrative load is unfavourable in relation to direct tasks such as project monitoring etc. ; iii) DGXXIII communication lines are lengthy and slow; iv) There has been high staff turnover, a lack of tourism expertise, poor staff training and understaffing of the financial administration; v) The procedures used to implement the Plan were too burdensome given resources; vi) Financial management was poor during the first part but has improved. The Commission accepts these findings and would add that many of these issues have now been addressed. It is also the case that the events during the first part of the Plan created considerable operational difficulties for the rest of the period. 27. As regards weak administrative arrangements and procedures the Consultants note that actions have been taken by the Commission to ensure improvement. This included reviews and strengthening of: Specifications of calls for tender and proposals Standard contracts and declaration forms; Selection procedures both for studies and pilot projects; Monitoring systems Post-project evaluation systems Payment control systems 28. These changes have resulted in a much improved operational system. In addition some of the purely administrative load has been reduced and a further review is in process which will enable desk officers to devote more time to direct project management. These measures include: Production of an external reference guide for project directors; Production of internal guidance for Desk officers within the Unit; Updating and improving the register of external consultants; Establishment of databases and other computerised systems for archiving etc. 29. The Consultants note in 37 that DGXXIII communication lines are lengthy and slow. This issue has been addressed by the recent approval of a new organisational structure which will incorporate a number of major changes. This includes the strengthening of the financial administration with a separate Unit, at the level of the Directorate General, concerned with budget and internal control. Another new Unit will be responsible for legal affairs and contracts. There will also be structural changes within the Tourism Unit itself. 30. In 38 the Consultants identified problems of high staff turnover, a lack of tourism expertise, poor staff training and understaffing of the financial administration. Some of these problems were caused or compounded by events during the first part of the Plan. The Unit has improved staffing, with a higher proportion of officials who are complemented by a number of tourism related national experts. The training needs of staff are examined on a regular basis. 31. As regards the burdensome procedures used to implement the Plan the Unit is reviewing options for efficiently adapting the administrative work in order to further improve quality of outputs and cost-effectiveness while ensuring compliance with standard Commission procedures. It is evident from the above actions taken by the Commission that the financial management which was poor during the first part of the Plan has now been substantially improved. 10 In summary changes made to administrative aspects include: • strengthened procedures relating to all aspects of project selection; • improved monitoring, control and evaluation systems; • new organisational structure which includes new Units responsible for budget control and contractual matters; • an increased number of officials working with tourism national experts. V II KEY RECOMMENDATIONS 32. The key recommendations are contained from paragraphs 42 to 60. The Consultants note that action has already been taken on many of their recommendations. As the Commission has already responded to many of these points this Section only comments on those areas which have not yet been covered, namely: the need for greater partnership and meetings based on specific issues; dissemination of results via the Committees in existence; identification of needs by independent experts; the greater use of performance indicators; greater autonomy in certain operational areas; the annual cycle of the Plan; i) ii) iii) iv) v) vi) vii) priorities for 1996; viii) guidelines for a future tourism programme. 33 The issue of partnership with external organisations is one which the Commission has already developed further in 1995. The Commission accepts however that existing arrangements with the industry are not yet ideal and will work with the various interests concerned to ensure that relationships are improved. Close partnership with the tourism industry will be a key element of any new programme. 34. Likewise some of the ways to increase cooperation mentioned in the Consultants' Report have already been progressed. The Commission agrees that a key value-added element 11 comes with the dissemination of results. It will examine with the various committees how best this is undertaken. The committees will continue to advise the Unit on the identification of needs and ensure that EU funding does not duplicate existing activities. Other experts, in the form of smaller technical working groups, will continue to be consulted. The Commission is in full agreement with the greater use of performance indicators and targets and will build these into any new programme. 35. As far as greater autonomy in certain operational areas for the Unit is concerned this issue will be examined when the new organigramme is in place. The Commission is in broad agreement that purely operational issues should be addressed at Unit level wherever possible. As regards the annual cycle of the Plan this issue has been addressed in 1995 where projects are now of a longer term - up to 3 years in length and this approach will be continued. The proposals of the Consultants relating to the priorities for 1996 (paragraph 59) have already been applied. The Guidelines for a future tourism programme (Paragraph 60) have been considered in the drafting of the proposal for the new Programme. In summary the Commission • wishes to strengthen its partnership with the tourism industry; • will examine with the relevant bodies how information and projects can be better publicised and be of greater practical use to the tourism industry; • will ensure that needs for studies and other work are fully assessed prior to • publication of tender specifications; is currently examining how the Unit can function with greater effectiveness within the framework of the new organisational structure. vm CONCLUSIONS 36 In general the Commission agrees with and, despite the criticisms, welcomes the overall assessment of the Consultants' Report. The complexity of tourism at EU level, together with the widely diverging views of Member States and European Parliamentarians on how best to develop policy in this area, made it difficult to maintain coherence within the Plan. This was particularly reflected in its Annex which was diverse and ambitious, especially when viewed in relation to the objectives and the available budget. 37. The goals of the Plan were first, to achieve better coordination and knowledge of the industry, and secondly, to stimulate innovation, create additional activity and encourage exchanges of information and transfer of experience. The encouragement of innovation and the transfer of experience was to a certain extent achieved by the joint-funding of pilot 12 projects (which always involved at least 2 transnational partners and usually more), and the creation of networks. Work on the dissemination of the results of pilot projects will continue, and the maintenance of networks will encourage the replication of best practice, transfers of experience and coordinated efforts throughout the Community. 38. The technical quality of the actions progressed was not always good. The evaluator recommends that a smaller number of larger projects fulfilling an identified need is required. This could increase the technical and political impact of expenditure and would reduce the administration of this expenditure at the level of the Unit. Without doubt, the extraordinary circumstances that prevailed during the first 18 months of the Plan's operation, and which ended in changes to Tourism Unit staff, affected its overall performance. This is evident in the poor results of certain projects. Improvements made during 1994 and 1995 have had a positive impact. 39. Other initiatives launched during the lifetime of the Plan, such as the Green Paper on the role of the Union in tourism and various intergovernmental tourism meetings, stimulated wide consultation specific to tourism. Indeed the conclusion of the majority as a result of these initiatives was that a consensus does exist as to the appropriateness of ensuring continuing Community action in the field of tourism, rationalising this activity and possibly deepening it. 40. The Commission believes that the Plan, while being of an experimental nature, nevertheless established a framework which provided a useful context for practical action. The Plan has been successful in establishing links between Commission services, Member States and other regional and sectoral interests. It has also provided the basis on which partnerships with the industry can be increased and further developed. The development of robust and effective policies to assist tourism is a challenging task. The diversity, nature and enormous potential of tourism requires appropriate action at many different levels: local, regional, national as well as European. Only concerted and complementary efforts will help to ensure a competitive, high quality and sustainable tourism industry. 41. This report has shown that the Community Action Plan to Assist Tourism has had both shortcomings and positive effects. It has served as a useful basis that has indicated the type of future actions which may assist this vital industry. The Commission therefore is submitting with this Report, a proposal for a first Multiannual Programme in favour of tourism. ANNEX 1 Background to the Community Action Plan to Assist Tourism Al. Prior to the Council Decision of 13. 7. 92 administrative arrangements relating to tourism were based on a Council Decision of 22. 12. 86 which established a consultation and cooperation procedure in the field of tourism, specifically an Advisory Committee made up of representatives of the Member States5. This Committee continues to exist and ran concurrently with the Management Committee set up under the Action Plan. A2. The Plan came into operation on 1. 1. 93 with a duration of 3 years and a proposed budget of 18 MECU. The measures undertaken were to be consistent with the principle of subsidiarity and to be progressed by reference to criteria set out in Article 4. Given the diverse nature of tourism the Plan was concerned with strengthening the horizontal aspects of the sector generally and improving its coordination within the Commission. It made therefore specific reference to the requirement to produce an annual report which evaluated the Community's activities which affected tourism and stressed the requirement for other related actions. The Plan also included as an Annex a list of 11 measures which established a range of priorities which were themselves divided into sections specifying particular actions. This Annex also set out which of these actions were to be taken forward in 1993. A3. In summary the principal objective of the Plan was to assist the sector's development by encouraging and strengthening the coordination of Community actions affecting tourism and by implementing a range of specific measures. The latter were pursued by encouraging integrated actions which were to be transnational, additional, innovative and respectful of the principle of subsidiarity. These actions were to be progressed with reference to criteria by way of pilot projects, research studies, publications and other appropriate means. Administrative Arrangements A4. To assist the Commission with the execution of the Plan the Council Decision specified that a Management Committee be established composed of representatives of Member States. This Committee agreed priorities, delivered its opinion on annual work programmes and guided specific actions. The Management Committee first met in December 1992 and subsequently on 10 occasions over the 3 years of the Plan, (twice in 1993 and 4 times during the other 2 years). A5. In January 1994 membership of the Committee was widened to include the Member States of the European Economic Area, which includes Austria, Finland, Iceland, Norway and Sweden. Three of these States namely Austria, Finland and Sweden became members of the Union in January 1995 and at that time Liechtenstein also became a Member of the Committee. Council Decision 86/664/EEC, OJ L 384 of 31. 12. 86 14 A6. Prior to the Plan the Council Decision of 1986 had established an Advisory Committee made up of representatives of Member States. The Management and Advisory Committee meetings often met consecutively and were usually followed by meetings of representatives industry. The Commission also organised separate meetings with of the tourism representatives from the tourism private sector. Member States' representatives were generally invited to attend these meetings as observers. A7. The Plan was progressed via a series of annual work programmes based on the actions listed in the Annex to the Plan and agreed by the Management Committee. Various mechanisms were used to disburse the funds. These normally included published calls for tender (principally for 100% financed studies and other actions), and published calls for proposals (for co-financed pilot projects with a Community contribution). Further projects were supported via the responsible Directorate's Subvention Committee and in particular cases funds were disbursed via direct contract. Project selection procedures were mainly undertaken within the Tourism Unit with other services invited to participate where appropriate. In almost all cases Commission services alone were solely responsible for the selections made. Mechanisms designed to facilitate coordination included the establishment of an Inter-Service Group which met on 9 occasions (twice in '93, 3 times in '94 and 4 times in '95) and the development of specific technical meetings involving both the Advisory Committee and the tourism sector. Staff and Financial Resources A8. The average number of staff employed in the Tourism Unit was 16, of which 11 were 'A' Grade level. The ratio of permanent staff relative to total staff was 5:17 at the start of the Plan rising to 11:17 by the end of 1995. Staff turnover was 86%. The Unit was occasionally supported by other staff of the Directorate, and where appropriate and possible drew on available administrative and technical services from both the Directorate and elsewhere in the Commission. A9. Financial resources available were increased from the original estimated budget of 18 MECU to a total of 21. 71 MECU. This was due to an increase of 3 MECU in budgetary provision for 1995 and the inclusion of the EEA Member States which contributed 566. 400 ECU in 1994 and 141. 300 ECU in 1995. Financial commitments amounted to a total of 19. 75 MECU. There was a 1. 9 MECU shortfall in commitments (originally assigned for actions on promotion in Japan). The Work Undertaken A10. The Plan can be divided into actions which were concerned with improving the coordination of Community actions affecting tourism and those actions which were related to the 11 annexed measures. In many cases these measures were also subject to initiatives to improve coordination and likewise the objectives and/or effects of many projects were to strengthen such coordination both at the transnational level and within specific technical areas. All. Activities relating to coordination were pursued in a number of different ways. These included mechanisms to improve liaison and coordination within the Commission, and those targeted at the tourism industry or interests associated with its development. Mechanisms used were varied and included those initiated as a result of legal or institutional requirements, and those developed for pragmatic consultative or technical reasons. They included the following: 15 The Management Committee The Advisory Committee An internal Inter-Service Group Established internal procedures Consultation with representatives from the tourism industry Meetings with National Tourist Organisations Ad-hoc groups of experts on specific subjects The 1994 Report on Tourism to the Council and European Parliament The 1995 Report on Tourism to the Council and European Parliament Conferences and exhibitions at tourism trade fairs Other conferences, presentations and publications Technical networks as the rationale or result of projects The Green Paper consultation process A12. The first year's work programme was specifically set out in the Annex to the Plan and the priorities progressed were as stated in that Annex. In Year's 2 and 3 the Commission proposed a programme based on the Annex, the budget available and on the requirements expressed by the Management Committee, the industry and other relevant interests. Over the period work on all the specified measures was undertaken. The degree to which each measure was progressed was dependent upon a range of factors including the priorities agreed by the Management Committee, the extent and similarity of work being undertaken by other Commission Services and the budgetary notes of the Parliament. A13. A number of changes were made to the annual programmes because of changing priorities. The Parliament identified a need to develop technical solutions relating to transport management in tourism, while an Environment Council held in Greece in 1994 expressed the importance of better information on tourism/ environment issues. Both these concerns were addressed in the 1995 call for proposals. The Parliament also invited the Commission to examine the issue of so called 'sex tourism' involving the abuse of minors. The Commission therefore submitted working papers to the Management Committee and consulted with the sector on this topic. 16 ANNEX 2 Expenditure Commitments made under the Plan A14. This annex gives a breakdown of expenditure commitments according to three categories: by each of the 11 measures; by the type of mechanism used to select projects; and by the type of organisation which received the funds. It should be stressed that these commitments do not always reflect the number of actions carried out. For example the Tourism and Environment prize included the disbursement of funds via direct contract to relevant organisations in 17 participating states even though this was essentially one action, albeit a major one. A15. Annex 2a shows a breakdown of the number of commitments made and the level of expenditure for each year by each of the 11 measures specified in the Annex of the Plan. Six of the measures absorbed 95% of the budget. The majority of expenditure was made under the measures 'Improving Knowledge' (37% of the total budget), especially concerning statistics where expenditure until 1997 has been provided and which accounted for 27% of total expenditure. 'Transnational Measures' accounted for a further 16% of expenditure; 'Tourism and the Environment' (14%); and 'Cultural Tourism' (13%). Two other measures that recorded significant expenditure were 'Training' and 'Promotion in Third Countries' (7% and 8% of total commitments respectively). The remaining 5% was spent on the other 5 measures. A16. Annex 2b gives a breakdown of expenditure commitments by selection mechanism. Calls for proposals seeking co-financed projects were published in all three years relating to Transnational Actions (annually); Cultural Tourism (twice); Training (once); and Tourism and the Environment (once). A total of 43% of the Plan's funds was disbursed using this procedure. Seven per cent of the funds were disbursed via the Directorate's Subvention Committee. Open calls for tender were published in all three years and related to various studies, publications and major initiatives funded entirely by the Commission. A total of 22% of the funds was spent via this mechanism. Approximately 26% of the funds were awarded via direct contract, most notably to the Member States. A17. Annex 2c gives a breakdown by type of organisation which benefited from the funds. The main categories are: Public Sector; NGOs/Associations; Universities/Research Institutes and the Private Sector. Because of the widely differing nature and type of tourism bodies and the varied options for their definition this Annex should be treated with caution. However on this basis it is evident that around 33% of the funds were disbursed to the public sector; 14% to NGOs/Associations; 6% to universities; and almost 48% to the private sector. ANNEX 2A 17 No. of Commitments by 1993 1994 1995 Total %of Av. Cost Measure Number ECU (000) Number ECU (000) Number ECU (000) Number ECU (000) Budget Of Action 1. Improving knowledge a) Statistics b) Studies c) Consultation Sub-total 2. Staggering holidays 3. Transnational a) Transnational b) Technology c) PECO Sub-total 4. Tourists as Consumers 2 4 5 11 0 6 9 3 18 0 399 302 188 889 0 458 1448 302 2208 0 1 2 3 6 2 0 1 2 3 1 249 726 211 1186 80 0 114 61 175 161 5. Cultural Tourism 18 1262 12 1245 6. Tourism/Environment a) Prize b) Projects Sub-total 7. Rural Tourism 8. Social Tourism 9. Youth Tourism 10. Training 11. Promotion* 12. Unclassified/ Mise 1 0 1 1 3 1 2 1 3 382 0 382 120 175 10 17 1 18 1 0 2 236 50 286 80 0 218 197 10 1259 1000 25 1 2 30 30 40 4 5 49 0 0 0 6 6 0 1 4656 266 312 5234 0 0 0 822 822 0 66 5 9 14 203 1829 2032 0 2 0 0 1 2 0 70 0 0 500 12 43 10 13 66 5304 1294 711 7309 26. 9% 6. 6% 3. 6% 37. 0% 123 129 55 111 2 80 0. 4% 40 6 10 11 27 1 458 1562 1185 2. 3% 7. 9% 6. 0% 3205 16. 2% 76 156 108 119 161 0. 8% 161 31 2573 13. 0% 83 23 10 33 2 5 3 821 1879 2700 4. 2% 9. 5% 13. 7% 36 188 82 200 1. 0% 100 245 1. 2% 228 1. 2% 49 76 12 1456 7. 4% 121 3 7 1530 7. 7% 510 67 0. 3% 10 Total All Measures 59 6268 58 4750 75 8736 192 19754 100. 0% 103 * NB: The Promotion project started in 1993 was undertaken at a cost of IMECU. The figure of 500. 000 ECU specified in 1995 relates to this project and was an administrative provision. ANNEX 2B 18 Number of Commitments by Type of Selection Mechanism 1993 Number ECU (000) 1994 Number ECU (000) 1995 Number ECU (000) Total Number ECU (000) % 100% Funded Actions Open Calls for Tender Major Initiatives Research Studies Publications Statistics Sub-total Restricted Calls for Tender Conferences Direct Contract (Member States) Statistics Environment Prize Conferences Sub-total Direct Contract (Other) Expert Groups: Travel etc Publications Other Sub-total 2 1 4 2 9 1382 200 201 399 2182 1 110 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 2 2 1 5 2 0 17 1 18 1 0 0 1 0 726 329 249 1304 80 0 236 43 279 15 0 0 15 2 2 0 0 4 1 40 0 1 41 4 2 3 9 646 196 0 0 842 4 5 6 3 18 2028 1122 530 648 4328 10. 3% 5. 7% 2. 7% 3. 3% 21. 9% 84 4 274 1. 4% 4656 0 60 4716 57 70 75 202 40 17 2 59 5 2 3 10 4656 236 103 4995 23. 6% 1. 2% 0. 5% 25. 3% 72 70 75 217 0. 4% 0. 4% 0. 4% 1. 1% Sub-total: 100% Funded Actions 10 2292 26 7678 55 5844 91 9814 49. 7% Jointly-Funded Actions Calls for Proposals Open: Pilot Projects Restricted: Pilot Projects Sub-total Subvention Committee Pilot Projects: Publications Conferences Research Study Other Sub-total 35 0 35 7 1 5 1 0 14 3173 0 3173 684 23 85 11 0 803 23 2 25 2 0 3 2 0 7 2618 61 2679 65 0 248 80 0 393 16 0 16 0 0 1 0 3 4 2658 0 2658 0 0 28 0 206 234 74 2 76 9 1 9 3 3 25 8449 61 8510 42. 8% 0. 3% 43. 1% 749 23 361 91 206 1430 3. 8% 0. 1% 1. 8% 0. 5% 1. 0% 7. 2% Sub-total: Joint financed actions 49 3976 32 3072 20 2892 101 9940 50. 3% Total AH Mechanisms 59 6268 58 4750 75 8736 192 19754 100. 0% ANNEX 2C No. of Commitments by 1993 1994 1995 Total Recipient Organisation Number ECU (000) Number ECU (000) Number ECU (000) Number ECU (000) Public Sector Management Committee National Statistical Organisation National Tourist Organisation Regional / Local Authority Sub total NGOs & Associations NGOs European Associations National Associations Sub total 2 0 0 12 14 2 9 5 16 198 0 0 893 1091 66 445 345 856 11 0 8 2 21 2 6 3 11 195 0 208 281 684 145 470 246 861 University / Research Institutes 4 320 5 648 19 % 5. 3% 16. 0% 2. 1% 9. 4% 32. 7% 1. 1% 8. 5% 3. 9% 13. 5% 648 3160 209 675 4692 0 769 189 958 18 31 12 19 80 4 17 9 30 1041 3160 417 1849 6467 211 1684 780 2675 153 10 1121 5. 7% 5 31 4 5 45 0 2 1 3 1 8 2 1 3 3 2 2 Private Sector Tourism Consultancy Major Management Consultancy Tour operator/Travel agent Non-tourism Company: Conference Organiser TV/Publications/Other media Technology/Consumer General consultancy Sub total Other Experts:Travel & meetings Mise Sub total 8 0 1 0 6 2 6 23 0 2 2 1197 0 10 0 1675 110 987 3979 0 22 22 12 2 2 0 1 1 2 1274 726 61 0 51 161 269 1647 164 91 250 545 70 102 20 2542 21 2869 1 0 1 15 0 15 4 1 5 57 7 64 28 4 4 3 10 5 10 64 5 3 8 4118 20. 8% 890 162 4. 5% 0. 8% 250 2271 341 1358 9390 1. 3% 11. 5% 1. 7% 6. 9% 47. 5% 72 29 101 0. 4% 0. 1% 0. 5% Total all Recipients 59 6268 58 4750 75 8736 192 19754 100. 0% EUROPEAN COMMISSION DIRECTORATE GENERAL XXIII Enterprise Policy, Distributive Trades, Tourism and Cooperatives EVALUATION OF THE COMMUNITY ACTION PLAN TO ASSIST TOURISM FINAL REPORT Prepared by Price Waterhouse February 1996 Price Waterhouse le> DIRECTORATE GENERAL XXIII EVALUATION OF THE COMIVIUNITY ACTION PLAN TO ASSIST TOURISM CONTENTS Page SUMMARY OF FINDINGS AND KEY RECOMMENDATIONS SECTION 1-INTRODUCTION Circumstances of the Evaluation 1. 1- Background and Context 1. 2 - Aims and Objectives of the Evaluation 1. 3 - Scope of the Evaluation 1. 4 - Methodology 1. 5 - Definitions 1. 6 - Structure of the Report SECTION 2 - PROGRAMME MODEL OF THE PLAN 2. 1 - Introduction 2. 2 - Objectives 2. 3 - Policy Areas 2. 4 - Measures 2. 5 - Strategies and Instruments SECTION 3 - TECHNICAL EVALUATION OF ACTIONS 3. 1 - Methodology 3. 2 - Overall Scores 3. 3 - Case Studies 3. 4 - General Assessment SECTION 4 • ADMINISTRATIVE AND OPERATIONAL ASPECTS 4. 1 - Description of the Administrative and Management Arrangements 4. 2 - Assessment of Arrangements in Place 4. 3 - Recommendations SECTION 5 - ASSESSMENT BY CONTRACTORS 5. 1 - Methodology 5. 2 - Analysis 5. 3 - Lessons, Recommendations, Suggestions and Best Practice i 1 7 12 25 33 Final Report • V2. 0 -27 Appendices I Descriptive statistics of actions II Checklist for interviews of key informants III Evaluation Fiche for project assessment by PW experts IV Evaluation Fiche for project assessment by contractors V Checklist for case studies VI Council Decision of 13 July 1992 VII List of contacts VIII Assessment by key informants Final Report -V2. 0 < 2^ EVALUATION OF THE ACTION P UN TO ASSIST TOURISM FINAL REPORT SUMMARY OF FINDINGS AND KEY RECOMMENDATIONS # SUMMARY OF FINDINGS AND KEY RECOMMENDATIONS This section presents a summary of the evaluation of the Community Action Plan to Assist Tourism (hereafter "the Plan") carried out by Price Waterhouse, and gives the key recommendations. This summary makes little reference to individual actions. Summary of Findings The Contents of the Programme In sum, the Tourism Action Plan looks like a patchwork of piecemeal initiatives, a fragmented, and sometimes incoherent and incomplete list of fragmented policies and activities. Most policies and measures contained in the Plan are probably defensible if considered individually, but it lacks an overall consistency. In Section 2 we present our analysis of the Plan, based on the links between actions and their objectives. We give below the main insights from this analysis. They are: 1 2 3 (1 ) The 11 areas (we will call them "policy areas") identified in the Annex to the Council Decision of 1992 do not constitute a coherent set of initiatives (2) There is confusion between ends (objectives) and means (measures such as studies and projects funded) (3) It is unclear how some actions would contribute to the stated policy objectives (4). The policy " transnational measures", since it is not an end in itself, is very difficult to fit into a logical framework. This policy seems to duplicate other policies, and its aims are unclear (5) Some of the 11 policies combine areas which are quite distinct and would be better separated (6) Some of the policies seem to be phrased in a rather oblique and partial way, which leads to apparent incompleteness in the supporting measures (this partly reflects Community areas of competence and subsidiarity) (7) Some of the policies seem undersupported in terms of specific measures. 4 5 6 The Plan also appears to be too ambitious given the human and financial resources available for the preparation and the implementation of the projects. The general lack of continuity between years is another drawback. The general findings of this analysis are corroborated by the main conclusions drawn from our interviews with key informants. Final Report -V2. 0 n -» EVALUATION OF THE ACTION PUN TO ASSIST TOURISM FINAL REPORT SUMMARY OF FINDINGS AND KEY RECOMMENDATIONS 41 The Role and "Market Positioning" of the Tourism Unit The Unit does not seem to have a clear and operational vision on how to position itself vis-à-vis competing and complementary service providers such as National Tourism Offices (NTOs), regional and international organisations, and other Commission Directorates General. Linked to this "market positioning" issue is the issue of the definition of the role and mandate of the Unit. Options include: a b c d e f g a department co-ordinating EU initiatives in the tourism sector a forum through which interested parties can debate and seek consensus a regulatory department involved in managing aspects of the tourism sector, for example in respect of accommodation rating, fire safety, and package holidays a monitoring department (watchdog) tracking and influencing events, for example in respect of statistics, identification of pressure points affecting the tourism sector (such as transportation, air traffic, staffing, seasonality) acting in a "commercial" capacity for example in running external promotional campaigns, in cooperation with the private sector and NTOs a body identifying, funding and promoting pilot schemes of potential European significance or value a body providing technical assistance to other EU institutions (eg to the Directorates General dealing with the structural funds) and the sector generally. Compliance with Article 4 of the Action Plan and Achievement of Objectives As our analysis of the programme model of the Plan indicates (see Section 2), the objectives of the Plan are often defined at a very high level and are sometimes vague. In addition, these are medium and long term objectives. It would be rather unfair to assess the effectiveness of the Plan against these objectives. If such an assessment is to be made, the answer would certainly be: "no, the Plan has not yet achieved these objectives, if ever. " However, Community Institutions should specify the timescales envisaged for an evaluation of such longer term objectives. The question must be rephrased: did the actions implemented under the Plan make a contribution towards the attainment of these strategic, long term objectives? In other words: did these actions (measures) achieve the associated intermediate objectives? Again, as demonstrated by the analysis of the programme model, such intermediate objectives are often missing or inadequately formulated to carry out such an assessment of effectiveness. A number of initiatives have made a positive contribution towards the attainment of certain intermediate objectives "set up" in the Plan (such as the provision of tourism statistics, and the co-ordination with the other EU policies and departments). It must be underlined, however, that the Tourism Unit has faced very difficult circumstances. In particular, most serious administrative disruptions were generated by charges of mismanagement 7 8 9 10 11 12 Final Report - V2. 0 7 it EVALUATION OF THE ACTION PUN TO ASSIST TOURISM FINAL REPORT SUMMARY OF FINDINGS AND KEY RECOMMENDATIONS • made against two senior members of the Tourism Unit staff. It must also be recognized that the nature of the Plan made the task of implementing and delivering it very difficult. Technical Evaluation Under these circumstances, it is easier (and more meaningful) to assess effectiveness at action (eg project and study) level than at a more general level. We carried out two main activities in this area: a technical assessment of all 1993-94 funded actions, and ten case studies of significant actions. Section 3 presents our findings of the technical evaluation of actions undertaken under the Plan. For 1993 and 1994 actions evaluated, individual scores from 1 to 5 were given, where a score of 5 means very good. In 1993, the average scores were 2. 9 for overall assessment and 2. 7 for cost effectiveness. In 1994, both scores increased to 3. 4 and 3. 0 respectively. In other words, the overall quality of the projects improved considerably in 1994 (reflecting improved selection procedures) and the cost effectiveness also improved. The overall number judged good or very good increased from 34% to 44%. The overall quality of the projects and studies carried out is poor to average (24% of the 1993 and 1994 actions reviewed are assessed as very poor or poor) while only 14% of completed projects were judged to be very good. There has been, however, a significant improvement in 1994. In most instances, the impact of the action is limited in space, scope, and time. There were numerous financial commitments (a total of 59 in 1993, 58 in 1994 and 74 in 1995), in our view too many in relation to the Tourism Unifs resources to adequately define and monitor them. Most actions had a size too small to make a significant impact, although a number of successful small actions were very cost effective. The average size of the actions we evaluated was ECU 121144 in 1993, ECU 117030 in 1994 and ECU 127015 in 1995. DG XXIII's achievements in the development of common tourism statistics has been very significant. Data collection is a necessary but expensive procedure in the tourism sector, and the approach adopted by DG XXIII was appropriate in the circumstances. The approach essentially focused on agreeing a common core minimum data set to be collected. Around this common core, related but optional data sets have been defined, again with commonality of definition, so that overall comparability is ensured. A Directive on collection of Statistics has recently been issued (23/11/1995 and will come into force by the end 1996), and Governments are now receiving financial support under the Action Plan for bringing their respective systems into line with this. A total of 4. 5 MECU has been committed to this task over a three year period (1995-1997). Most actions (projects and studies) funded can be classified into two groups. On the one hand, there are actions fulfilling political objectives; these actions promote "European" concepts and networks, and are of a fairly high profile and newsworthy; on the other hand, there are commercially interesting actions, with sustainability built in. These two groups can occasionally coincide. The promotion of "transnational networks" is, in our view, not an objective, but a means to achieve an objective. There is an obligation, under Article 4 of the Council Decision, for cost effective measures. Political objectives eg of promoting transnational networks, may be achieved better and more cost effectively through other means and do not need the cover or justification of a pilot project which is only occasionally really innovative. 13 14 15 16 17 18 19 Final Report • V2. 0 'IS' EVALUATION OF THE ACTION P UN TO ASSIST TOURISM FINAL REPORT SUMMARY OF FINDINGS AND KEY RECOMMENDATIONS <H> 20 21 22 23 24 25 26 27 28 29 A number of studies were funded to look at major issues of importance to the tourism sector. These were generally well executed, even if the choice of topics (youth tourism, business and conference, yield management, for example) did not appear to be based on a careful analysis of needs taking into account existing available data. We are aware that the topics relating to studies were proposed and/or agreed by the Management Committee. We take the view that more time must be used to define precisely the studies to be funded, taking into account existing studies, ongoing research programmes, and the travel industry needs. A number of projects and studies had as their objective the preparation of reference lists, handbooks and guides of potential interest to the sector. All of these suffered from a lack of in-built post project sustainability, which are essential when data need frequent (and often expensive) updating. The Plan made a number of strong contributions for tourists with disabilities. Guides to make Europe accessible to the disabled tourists are examples of excellent and cost effective studies emerging from the Plan. Most of the cultural components of the Plan focused on building-up transnational routes and networks based on a range of themes. Most of these were relatively low cost exercises, which produced interesting results. At the same time, the sustainability and impact on tourism demand are likely to be limited, and the main longer term benefits may arise from the strengthening of linkages between regions. If there is no intention to secure a sustained economic benefit, then this must be made clear and the actions must be judged on that basis. But the need to be cost-effective remains. It is also noted that the concept of routes and trails of linked destinations was already well established before the Plan, and the marginal value of creating further routes/trails may be limited, and should thus be assessed. The innovative aspect of the DG XXIIPs funded trails is their transnational design. The longer impact is still unknown and needs research. The scope of the education and training projects has been quite wide, including development networks, curriculum definition, and identification of new professional needs. Those networking projects which seem the most successful have been the ones reinforcing existing networks, while the creation of new networks is judged in general less successful on sustainability and other grounds. Where projects have focused on curriculum needs with European commonality they have served very useful roles. Less successful in contrast have been a number of projects which have focused on the identification of new professional categories. They have also been a number of one-off training exercises (eg in Eastern and Central Europe) which although in themselves useful, do not fit within an institutional or other framework providing a basis for follow up and sustainability. Though rather in isolation they may have been effective, although they did not fit into a coherent plan and are best avoided in the future unless mechanisms to ensure sustainability are put into place. The technology pilot projects supported under the Plan are broadly either generic, addressing key issues such as data definition procedures for information exchange; or specific in seeking to apply new technology to specific situations. This distinction generic/specific broadly coincides with project responsibilities with DG XXIII, where the essentially generic projects (such as CIRT, EDITIN, ET1, TIRS, all good projects) are considered as "technology" projects and classified under the policy area Transnational measures". A number of Final Report • V2. 0 lé EVALUATION OF THE ACTION P UN TO ASSIST TOURISM FINAL REPORT SUMMARY OF FINDINGS AND KEY RECOMMENDATIONS 4» 30 31 32 33 34 35 36 37 38 projects in the cultural area are dominated by technology solutions and from these specific important conclusions relating to their technology can be drawn. In contrast to the cultural activities funded, there are indications that one way or another, many of the "generic" technology projects funded would have in due course proceeded without EC funding, since they addressed issues of industry significance, such as globalisation of data or access advances. The EC funding, while contributing very usefully to addressing these issues, could be described as supportive. It has however had a positive impact in accelerating the application of new technology to tourism-specific tasks (for example, the 1993 project EDITIN). The specific projects have not been particularly successful because there is a strong likelihood that most will wither following completion of project funding because they lack any in-built sustainability or commercialisation component. These projects appear to have been largely designed to "access" EC funds, rather than to address specific needs. Running through many of the new technology oriented database projects is apparently a failure to address post project sustainabilty. Tourism databases need constantly updating and administrative and financial mechanisms for achieving this are as important a component of project design as the technical components. These follow up and commercialisation issues were in the majority of cases not satisfactorily addressed, with as a result a number of interesting projects completed, but having most probably no lasting contribution to make to the tourism sector. One issue that was not specifically addressed in the design of the technology and other projects was the question of relevance to SMEs. Technology offers both a barrier and an opportunity to SMEs, and this issue has not been satisfactorily addressed. Some projects omit any link to suppliers at all, concentrating on tour operators and travel agents, and omitting hotels. The Internet, with its low cost of entry, represents, it can be argued, a major area of potential for SMEs, yet there has not been any project, so far, addressing this area. Administrative and Operational Aspects In Section 4, we give an overview of administrative and management arrangements and procedures, followed by our assessment. Our main findings are summarised below. Administrative arrangements and procedures were weak at the beginning of the implementation period, but have been significantly improved in 1994 and 1995. Members of the staff we have seen are clearly committed and hard-working. Staff members are overloaded with administrative tasks, the rationale and effectiveness of which are not always evident. The balance between direct applied (project preparation, site visits, contacts with contractors etc) and administrative tasks is unfavourable. Communication lines at DG XXIII are lengthy, procedural and slow, and so is the decision-making process. Procedures have however progressively improved on matters related to the Plan. We also identified the following shortcomings in the area of staffing: high turnover; low percentage of "tourism experts" and generally inadequate presence of tourism skills; understaffing of the financial administration; inadequate training (eg in area of financial and project management). 39 Given the limited staff resources available at the Unit in 1993-95, it was ill-advised to implement the v Final Report - V2. 0 EVALUATION OF THE ACTION PUN TO ASSIST TOURISM FINAL REPORT SUMMARY OF FINDINGS AND KEY RECOMMENDATIONS # 40 41 Plan through the funding of such a large number of projects and studies (mainly through open calls for proposals) because of the heavy administrative burden implied by such an approach. Hence, the quality of a number of selected proposals was poor, without clear objectives, without a strong project team, or with a very limited innovative aspect. The projects selected in 1994 showed an improvement and the result of the 1995 calls for tenders and for proposals display a significant improvement in quality, and expectations of good results are running high. Financial management was poor during the first half of the implementation but has since been improved. It still needs to be strengthened. Poor performance in the area of financial management was due to the conduct by senior members of the staff (who were removed from the Unit in the first half of 1994), and also due to inadequate systems (such as poor use of information technology), inadequate staff training, and insufficient staffing at administrative level (ie grade B). Key Recommendations 42 We now give our main recommendations to the Tourism Unit and DG XXIII. These recommendations are amplified in the relevant sections of the document. Strategic Aspects J 43 We think that the Unit should limit the scope of its activities by concentrating its resources in a few strategic areas in order to become more effective. The Unit should seek to achieve a clearly spelled out vision on its "market positioning", mission and objectives, and should select its means of action accordingly. 44 The Unit must take action in order to regain credibility and increase the "tourism" profile of the Unit (current staff are often perceived by our interviewees as not having sufficient expertise in the area). Operational Aspects 45 46 47 48 Partnership and co-operation with the various stakeholders (private and public, including Member States, National Tourism Offices and other industry interests) should be improved. The three (Management, Consultative and Advisory) Committees could become the main vehicle for dissemination of the results of the activities funded or undertaken by the Unit. These committees must accept that they have a large part to play in dissemination and undertake this responsibility. Committees in place could also be used to help the Unit in the identification of needs (eg possible pilot projects) and to ascertain that EU funding will not duplicate existing initiatives. Independent experts could fulfill a similar role. In the area of staffing, corrective actions should be considered: to reduce staff turnover; to improve the in-house technical tourism expertise, eg through secondments from NTOs; to provide more training to the staff (at all grade levels), perhaps through secondment to the sector, eg in management, financial and administrative aspects of Community procedures. 49 The information systems in place should be reviewed and significantly improved, eg to generate more management information to support staff members. Final Report - V2. 0 1* EVALUATION OF THE ACTION P UN TO ASSIST TOURISM FINAL REPORT SUMMARY OF FINDINGS AND KEY RECOMMENDATIONS «» 50 51 52 53 54 55 56 57 The Unit and the hierarchy should consider a review of its administrative arrangements and select performance indicators as far as possible (with associated targets) in a number of areas, allowing for limitations imposed by the Commission administrative rules. The Unit should review internal procedures to free up time to prepare, monitor, and follow up the actions funded. The staff is devoting a disproportionate amount of time to essentially non productive paperwork. There is a need to strengthen the monitoring and evaluation of projects, and, more generally, to design and implement simple but efficient procedures, designed at the outset and not retrofitted. Also, more site visits should take place on a regular basis. This is an aspect which already improved in 1994-95. Actions should also be considered to ensure that an adequate dissemination/distribution of project results takes place, for example, by earmarking resources in the project budget at the preparation/selection stage. These aspects were better considered in the 1995 call for proposals. The number of projects and studies funded should be significantly reduced in order to increase their impact and to limit administration. The use of the "call for proposals" procedure should be reviewed to reduce the administrative load and to maximise the effectiveness of the procedure. The possibility of contracting out some of the work must be considered. The proportion of meetings organised around a theme or a technical issue should be increased. The Unit should be given more autonomy in a number of strictly operational areas such as the management of the budget for missions. The Unit should be able to respond quickly and flexibly to the needs of project implementation (site visits, participation to workshops etc), within the strict limits of an agreed budget. This should of course not make the Unit less accountable or transparent. 58 The annual cycle of the Plan had a negative impact on the quality of the actions selected and on the their implementation. A longer project timespan is required (eg three years). 59 In the short term (ie 1996), we feel that initiatives should focus on: a b c d dissemination of the best outputs generated by the actions funded (eg publication and distribution of completed reports) follow-up to successful projects and studies to draw lessons, and transfer best practice development of relationships with the tourism industry and other stakeholders continuation of activities in the area of co-ordination with other Directorates General and to develop Community tourism statistics. Guidelines for a Future Tourism Programme 60 A future tourism programme should: a be more focused (fewer priorities, fewer funded actions, larger average project value) vii Final Report - V2. 0 o c EVALUATION OF THE ACTION PUN TO ASSIST TOURISM FINAL REPORT SUMMARY OF FINDINGS AND KEY RECOMMENDATIONS «» b c d e f g increase internal consistency of the programme include actions of a multiannual character have well prepared and specified terms of reference based on a logical framework taking into account measurable indicators of success, post project ownership, responsibility and sustainability issues improve the procedures of preparation and planning, monitoring and evaluation of projects (both internal and external, ie independent) give a high priority to the dissemination of the results and the follow-up of the actions ensure that the most successful sustainable actions carried out in 1993-95 are continued, and reinforce the areas of success. viii Final Report-V2. 0 Jo EVALUATION OF THE COMMUNITY ACTION PUN TO ASSIST TOURISM FINAL REPORT SECTION 1 - INTRODUCTION 4Jà W SECTION 1 - INTRODUCTION Circumstances of the Evaluation Price Waterhouse is aware of certain allegations regarding the conduct of affairs in the Tourism Unit in the period prior to March 1994. Directorate-General XXIII gave Price Waterhouse the following briefing on the matter: "The Director-General of DG XXIII became aware during 1993 of these allegations. He immediately launched in-depth internal inquiries within the Directorate-General into the allegations. As soon as he became convinced that there was at least some truth in these allegations other services were involved. As a result of these various inquiries two members of the Tourism Unit were suspended and later dismissed from the services of the Commission. The Commission informed the civil authorities in a number of Member States of the information which had come to hand and invited them to carry out their own inquiries. " In view of all these inquiries, some of which are still underway, Price Waterhouse were not called upon to investigate the allegations. They were, however, asked to bear in mind the effect which the allegations and inquiries they entailed had on the overall effectiveness of the plan. In the light of this, Price Waterhouse have only commented in their/evaluation on circumstances affected by these events. In particular it should be noted that in addition to the inquiry conducted by DG XXIII itself the other inquiries involved the staff of the Unit in tracing and supplying information. This deprived them of considerable amounts of time which would normally have been devoted to other duties. A further consequence of the allegations and inquiries is that the budgetary and financial authorities of the Community Institutions became, quite naturally, very strict and precise in relation to the application of rules to activities of the Tourism Unit. This led to considerable delay in processing matters some of which would normally have been regarded as routine. This was most frustrating for the Tourism Unit, again costing time normally devoted to direct implementation of the Plan and delaying that implementation. 1. 1 - Background and Context The Community Action Plan to Assist Tourism (the Plan) began in January 1993 further to Council Decision 92/421/EEC of 13 July 1992. The Plan lasted three years finishing at the end of December 1995. Article 6 of the Plan stipulated that an evaluation would be undertaken. The Annex to the Action Plan gives 11 specific groups of measures relating to tourism. Article 4 of the Plan states that actions taken must: a b c be consistent with the principle of subsidiarity be cost-effective and make a significant impact on the Community tourism industry facilitate the development of the industry with particular reference to small and medium sized businesses Final Report-V2. 0 V EVALUATION OF THE COMMUNITY ACTION PUN TO ASSIST TOURISM FINAL REPORT SECTION 1 - INTRODUCTION # d e f g h encourage competition within the Community and increase the competitiveness of Community tourist services on the world market be conducive to preserving and protecting the quality of the natural environment, the cultural heritage and the integrity of local populations be conducive to improving the provision of information and services to the protection of tourists help improve the quality of Community tourist services be implemented through coordination with the national authorities and, if necessary, with the regional or local authorities as well, so as to take account of the importance of tourism for regional development. 1. 2 - Aims and Objectives of the Evaluation 7 8 The principal aim of this study is to undertake an evaluation of the Plan. The evaluation includes a technical evaluation of pilot projects, studies and other initiatives of work undertaken in 1993 and 1994, and an assessment of the (as yet) uncompleted work selected under the 1995 programme. The basis for the evaluation is the general criteria stated above and the specific criteria stated in the Annex to the Action Plan and in subsequent calls for tenders and proposals. In the case of ad-hoc projects the evaluation considered the application as submitted. The evaluation focuses on all the measures within the Action Plan and considers quantitative and qualitative aspects. 1. 3 - Scope of the Evaluation 9 Actions taken under the Plan consist of a range of horizontal and specific initiatives. These include: a b c d e a co-ordination task within the Commission concerning other policies with an impact on tourism transnational pilot projects jointly funded by the Commission and other partners research studies on specific aspects of the tourism industry conferences and networks related to specific aspects of the tourism industry reports and consultative documents on tourism policy and practice for Community institutions and the industry. 10 11 Appendix I contains the list of all the projects and studies supported. In addition to the evaluation of these specific actions we were required to evaluate the priorities given to the various different measures under the Plan, and the administrative aspects relating to the Plan and the extent and allocation of resources. 1. 4-Methodology 12 In this section we describe the programme of work we carried out and the main activities completed. We also present the ten "case studies" selected. 2 Final Report-V2. 0 3" r> EVALUATION OF THE COMMUNITY ACTION PUN TO ASSIST TOURISM FINAL REPORT SECTION 1 - INTRODUCTION «» Programme of Work 13 The programme of work was as follows: Phase 1 - Overview of the Plan. Programme of (face-to-face and telephone) interviews with officials of DG XXIII and of other DGs, members of the Management Committee, members of the industry, independent experts. In total we carried out over 60 interviews (of which 53 were outside DG XXIII). • Phases 2 and 3 - Analysis of projects. We carried out a technical evaluation of all projects completed in 1993 and 1994 (Phase 2), and of the proposals of the projects selected in 1995 (Phase 3). Most 1993 and 1994 contractors received a questionnaire covering the following aspects: project description, achievement of objectives, project implementation. • Phase 4 - Analysis, conclusions and recommendations. Activities Completed 14 15 16 17 18 19 The assignment started on 10 November 1995 (launch meeting at DG XXIII). Our final report was accepted by DG XXIII on 26 February 1996. Price Waterhouse was assisted by Mr Esmond Devas, tourism expert. j In November and December 1995 we carried out a first round of interviews and meetings with DG XXIII officials. About ten interviews were completed by the end of December. The main objectives of this first round of interviews were to establish contact with the Tourism Unit staff, and to gain an overview of the activities and to collect views on the (past, current, present) role of the Tourism Unit. We then contacted 69 key independent informants and experts (external to DG XXIII), and carried out 53 face-to-face or telephone interviews. The key issues discussed were (see Appendix II for checklist): a b c d e f g Exposure to the Tourism Action Plan Interface with Tourism Action Plan (co-ordination, complementarity, overlap etc) Assessment of the rationale of Tourism Action Plan: does the programme makes sense? Assessment of impact and effects: what has happened as a result of the programme? Achievement of programme objectives: has the programme achieved what was expected? Alternatives: are there better ways of achieving the results/objectives? Lessons, recommendations, suggestions, best practice. All the members of the Management Committee and all the members of the Consultative Committee received a letter from PW explaining the objectives of the assignment and encouraging them to contact us if they wanted to contribute. We received written responses from Germany, Greece and Portugal. We interviewed the representatives of Italy, Ireland, Spain, France, Finland, Belgium and Luxembourg. We set up a project database containing descriptive and financial information for all the 1993-95 actions (see Appendix I). 77 contractors for the 1993 and 1994 projects received a short questionnaire covering project description, achievement of objectives, implementation, and assessment of the implementation. The Final Report -V2. 0 V EVALUATION OF THE COMMUNITY ACTION P UN TO ASSIST TOURISM FINAL REPORT SECTION 1 - INTRODUCTION «» proforma used is given in Appendix IV. We received 47 responses which represents a response rate of 61%. We treated the findings from this survey with caution because of the possible bias expressed in the responses. 20 We carried out a technical evaluation of all completed actions (1993 and 1994 contracts, 90 in total). To carry out this assessment, we prepared an Evaluation Fiche for project assessment (see Appendix III). Our approach to project evaluation was, as far as feasible, to proceed in five steps, as follows: a b c d e group the projects for review according to the official(s) in charge set up a first discussion with the official(s) who made a brief introduction to the projects complete the Evaluation Fiches discuss the fiches with the EC official (s) finalise the fiche and enter the score in the database. 21 We carried out ten case studies (see Table 1. 1). We used the following criteria to make our selection, which was agreed with DG XXIII: a b c d need to cover all three years in so far as possible need to select, for each year, the projects with most significant budgets need to be representative of the annual priorities need for balance between successes and failures. Final Report-V2. 0 EVALUATION OF THE COMMUNITY ACTION PLAN TO ASSIST TOURISM FINAL REPORT SECTION 1 - INTRODUCTION •• Table 1. 1 -Case Studies Contractor Title 1. L&R Leisure European Project for the Training of Operators in Tourism in Bulgaria and Slovakia 2. HTL Tourism & Environment Prize 3. Delvico/Bates Tourism Promotion of Europe as a Destination for Tourists in Long Distance Source Markets Year Funding ('000 Ecus) 1993 220 Domain Training Central & Eastern Europe 1993 1993 381 1000 Environment Promotion 4. Regione Toscana ETI - European Tourism Information & Booking 1993 200 New Technologies 5. Hjoerring Business College Development of a Training Strategy for Employees and Trainers within the Tourism Sector in the Cold Water Regions around the North Sea 1994 195 Training 6. FECTO/FOTVE The "Art Cities in Europe" Programme 1994 7. Arthur Andersen Yield Management in Small and Medium Sized Enterprises in the Tourism Industry 1994 195 392 Culture Study 8. National Institutes of Stat. & Others Theme: Development of Community Tourism Statistics On going 4500 (1995-97) Statistics 9. WTTC Theme: analysis of procedures through analysis of 1995 Environment projects, in particular the project European Community Network for Environmental Travel and Tourism 1995 631 Analysis of tendering, assessment and implementation procedures 10. - Theme: Coordination by Tourism Unit On going - Horizontal activity We also reviewed the administrative aspects of the Plan, in particular selection and monitoring procedures and management arrangements. 1. 5-Definitions The Plan consists of 11 groups of Community measures to assist tourism (see Appendix VI). We call these "policy areas', and reserve the term "measure" for the actions taken to implement the Plan. 22 23 24 We distinguish between three types of actions: a b c studies (100% funded actions) projects (jointly-funded normally up to a maximum of 60%) other activities such as the co-ordination within the Commission. Final Report-V2. 0 3T EVALUATION OF THE COMMUNITY ACTION PLAN TO ASSIST TOURISM FINAL REPORT SECTION 1 - INTRODUCTION 4 1k \\\W 1. 6 - Structure of the Report 25 26 The rest of the report is structured into five main sections. Section 2 presents our analysis of the Plan, based on the links between actions and their objectives. Section 3 summarises the findings of the technical evaluation of actions (studies, projects and other activities) undertaken under the Plan. In Section 4 we give an overview of administrative and management arrangements and procedures, followed by our assessment. Section 5 gives the findings of the postal survey we carried out for the 1993 and 1994 projects and studies. Appendix VIII contains a summary of the assessment by the key informants we talked to (we provide a list of contacts in Appendix VII). Waterhouse. It should be noted that these are not necessarily the views of Price Final Report • V2. 0 3/ EVALUATION OF THE COMMUNITY ACTION PLAN TO ASSIST TOURISM FINAL REPORT SECTION 2 - PROGRAMME MODEL OF THE PLAN • SECTION 2 - PROGRAMME MODEL OF THE PLAN 2. 1 - Introduction 27 A structured approach is required to analyse policy objectives and their outcomes in a systematic way. We began our evaluation process with the preparation of a "profile" of the Plan's policy and its component parts. This section gives our analysis of the Plan, based on the notion of a programme model - a schematic representation of the links between measures (or actions) and their objectives (ie a "hierarchy of objectives"). A programme model is an important tool not only for monitoring and evaluation purposes but also for strategy planning and policy-making. 28 We used a simple four-level structure of objectives, which could be refined and further sub-divided. These levels are: a b c the immediate objectives which relate to particular actions (measures) and which must be expressed in quantitative terms as far as possible. the intermediate objectives are more detailed and specific. They generally represent a "cause- and-effect" assumption. the ultimate objectives (also called final or strategic objectives) of a policy are broad, long term objectives, eg "to promote economic growth and employment". 29 Table 2. 1 gives an example of a four-level hierarchy of objectives. Table 2. 1 - Example of Hierarchy of Objectives Measures Immediate Objectives Intermediate Objectives Ultimate Objectives to increase the competitiveness of the European tourism industry to generate employment and growth in the European tourism industry - campaign sustained over three years - to promote Europe as a destination - improve use of existing infrastructures - support cultural itineraries - training (eg languages) - to improve communication links (eg links with Japan) - to launch new products targeting US and Japanese markets -etc. - to improve level of services -etc. 30 31 The Plan appears to have been built from the bottom up ie starting with individual measures trying to aggregate these into broad policy areas and only implicitly linking these to the ultimate objectives. The approach taken here, as a test of the coherence and completeness of the programme, was to analyse the programme top down, and to ask at each level in the hierarchy of objectives: Final Report-V2. 0 1> EVALUATION OF THE COMMUNITY ACTION PLAN TO ASSIST TOURISM FINAL REPORT SECTION 2 - PROGRAMME MODEL OF THE PUN • (1) are the items at each level consistent with each other? (horizontal consistency) (2) (3) do the items at the next level down in the hierarchy contribute to the items at this level? (vertical consistency) are the items at the next level down the most effective ones to enable us to achieve our objectives, or are there other possibly more effective options? (completeness) 32 This section does not explicitly consider consistency with other Community objectives. DG XXIII needs to spell out these linkages because: a b c other policies contribute to the objective of fostering tourism tourism policy can contribute to the objectives of policy in other domains and policy in other domains can act as a constraint on tourism policy. 2. 2 - Objectives 33 Figure 2. 1 shows a logical model setting out the links between the measures, intermediate objectives and ultimate objectives (immediate objectives have been left out for the sake of clarity) implicit in the Plan and the 11 policy areas (groups of measures). I 34 The ultimate objectives of Community tourism policy seem to be: a b c to promote economic growth and employment to promote economic and social cohesion to promote "European citizenship". 35 36 37 38 39 40 But the programme can only be understood in terms of other objectives, explicit in the documentation but relating to other policy areas, notably to protect and promote the natural and cultural heritage and to protect the consumers' interests. There may be tensions between these two sets of high level objectives (eg between promoting growth and protecting the environment). It is not clear how some of the 11 "policies areas" identified in the Annex to the Council Decision contribute to these high level objectives, implying that there are other implicit but unstated aims: eg social tourism (8); viewed solely as an instrument for achieving the higher level objectives this must rank as a very low priority, although it might be important in support of some other (unstated) social aims. The objectives of promoting competition and of helping SMEs, stated in the Council decision, do not seem to be clearly supported by specific measures or policies. Other policies only make sense if intermediate objectives are added eg "to improve knowledge of the tourist industry" requires an intermediate objective such as "To improve the quality of decision making at Community, Member State, local and private level" (see Figure 2. 1). The policy area " Transnational Measures" designed to promote transnational tourist development initiatives, since it is not an objective but a mechanism, is very difficult to fit into such a logical framework. Final Report-V2. 0 It Measures To ensure that the environment is more fully taken into account in the development of tourism To highlight the importance of the cultural heritage for tourism and to promote a greater knowledge of the culture, traditions and ways of life of Europeans To make Europe a more attractive destination for tourists from distant countries To promote a better seasonal distribution of tourism To provide support for increased professionalism To develop tourist activities in a rural environment Figure 2. 1 - Programme Model of the Plan Intermediate objectives Ultimate objectives To protect and promote the natural and cultural heritage ? Tension To promote the diversification of tourist activity To generate income and employment To promote the development of the tourist industry To improve the quality and competitiveness of the Community's tourism services on offer To contribute to economic and soda! cohesion To facilitate access to tourism by groups who have difficulty in taking holidays To expand the demand for the Community's tourism services To promote youth tourism To help tourists as consumers To improve knowledge of the tourist industry To promote transnational initiatives To allow people of all kinds to gain a better knowledge of Europe's cultural roots and of the cultures and ways of life in the Member States To improve the quality of decision making at Community, Member State, local and private level To ensure greater consistency of Community measures To preserve free competition \ \ -© ©- To foster "European Citizenship" To protect consumers Indirect impact - feedback via other measures To promote small and medium sized enterprises Tf EVALUATION OF THE COMMUNITY ACTION PLAN TO ASSIST TOURISM FINAL REPORT SECTION 2 - PROGRAMME MODEL OF THE PLAN • 2. 3 - Policy Areas 41 42 43 The word " policy area" is used here to refer to the 11 groups of measures identified in the Annex to the Council decision of 1992. There is a confusion between ends and means; the 11 policy areas are a mixture of instrumental objectives or ends eg " [to promote] staggering of holidays" (2), and types of measure or means eg "transnational measures", " promotion in third countries". Some of the 11 policy areas combine two areas which are quite distinct and would be better separated, for example: (1 ) "Improving knowledge of the tourism industry" and "ensuring greater consistency of Community measures " (1); whilst both of these are important in their own right, neither will necessarily lead to the other, nor do the measures contributing to the attainment of one necessarily contribute to the attainment of the others (2) The policy "transnational measures" seems to duplicate other policies, and its aims are unclear. It is a tactic, not a policy objective. 44 45 There is a degree of confusion between certain policy areas. For example the "support for the exchange of experience in the field of visitor management techniques", included in the policy "cultural tourism" (5. b) should come under the policy "tourism and the environment" (6). Some of the policies seem to be phrased in a rather oblique and partial way, which leads to apparent incompleteness in the supporting measures (this may reflect Community areas of competence and subsidiarity) eg (1 ) "to highlight the importance of the cultural heritage for tourism and to promote a greater knowledge of the cultures, traditions and ways of life in Europeans" (5); this is expressed in terms of awareness rather than substance (a substantive policy would be eg to safeguard the cultural heritage in the interests of tourism1), and accordingly substantive measures such as actions to protect ancient monuments are omitted. (2) This contrasts with policy 6 (Tourism and the environment) where a much more direct approach is taken to environmental protection, leading to a more coherent set of measures. 2. 4 - Measures 46 47 The measures (studies, projects and other actions) identified in the Annex to the Council Decision of 1992 in the main bear a logical relationship to the 11 policy areas. However, there are exceptions as detailed below. It is unclear how some measures would contribute to the stated policy objectives eg "development of tourist cooperation with central and Eastern Europe and the Maghreb" (3 c): how does this promote tourism in the Community? 48 Some of the measures are confused, eg (1 ) "[to promote the interests of] tourists as consumers" (4): is a policy, to be implemented by measures to Final Report-V2. 0 U^ EVALUATION OF THE COMMUNITY ACTION PLAN TO ASSIST TOURISM FINAL REPORT SECTION 2 - PROGRAMME MODEL OF THE PUN • (2) (3) improve information and protection: " in areas such as existing classification systems, signposting symbols, time share arrangements, over-booking and procedures for redress". It is unclear whether it is intended merely to inform tourists of their rights, or to strengthen those rights. These are quite distinct and should be distinguished. " consultation of (sic) tourism professionals within the Community" (1. a); consultation about what and with what purpose? Why should it be done independently of either the development of statistics (1. a) or studies and strategies (1. b)? the policy objective "to make Europe a more attractive destination for tourists from distant countries" (policy area 11) implies action to change the tourism product itself. Yet the proposed measures appear to be confined to promotion ie increasing awareness of the product in third countries. If the aim is in fact "to increase the number of visitors from third countries" then there would be a much wider range of possible measures as shown in Figure 2. 2 which is an example of how one area of policy might be analysed; the measures are purely illustrative. ' Figure 2. 2: Example of Low Level Logical Framework (Hierarchy of Objectives) for Policy Area 11 Measure Immediate objectives Intermediate objective 49 Some of the policies seem undersupported in terms of specific measures - ie the measures do not form a sufficiently complete set to be likely to contribute significantly to the achievement of the policy eg (1 ) "ensuring greater consistency of Community measures" (part of policy 1 ) appears to be supported only by the measure "preparation of strategies adapted to keep pace with demand" (other measures under this heading being in support of the other aspect of policy 1); this seems very weak for what surely should be a central aim (2) "youth tourism"; the two measures indicated are far more specific than those in other policy areas; they may be very effective but are there not other possible measures? 10 Final Report-V2. 0 Inf EVALUATION OF THE COMMUNITY ACTION PLAN TO ASSIST TOURISM FINAL REPORT SECTION 2 - PROGRAMME MODEL OF THE PLAN 4» 2. 5 - Strategies and Instruments 50 It would be possible to group the actions in terms of the underlying strategies: a b c d implementation of specific measures, complementary to those taken at national level coordination of measures undertaken by Commission departments which affect tourism, and ensuring better integration of tourism with the various Community policies promoting the development of transnational measures ensuring close cooperation between all public and private bodies in the sector. 51 52 53 Similarly, two main types of instruments - a policy instrument is a mechanism chosen to achieve an objective - are used: legislation and financial support. In further plans, the Commission should undertake an analysis of the completeness of the programme for each of the policy areas, and an assessment of the appropriate strategies. More generally, the Commission should prepare a programme model of the actions to assist tourism, focusing on a reduced number of policy areas and linking in a systematic way the elements of the model, ie instruments, measures, immediate objectives, intermediate objectives and ultimate objectives. Output and performance measures or indicators, and targets (ie quantified objective to be attained at a specified future date) should then be set. These should be intrinsic to the whole process of management, including planning, monitoring, evaluation, and public accountability. Examples of indicators are: percentage increase in the number of visitors from third countries; number of pilot projects leading to commercially viable transnational products; evolution of the EU market share in world tourism. 11 Final Report - V2. 0 li EVALUATION OF THE COMMUNITY ACTION PLAN TO ASSIST TOURISM FINAL REPORT SECTION 3 - TECHNICAL EVALUATION OF ACTIONS • SECTION 3 - TECHNICAL EVALUATION OF ACTIONS 3. 1 - Methodology 54 Whenever possible, our approach was in five steps, as follows: a b c d e we grouped the files for review according to the EC official(s) in charge, the officials gave a brief introduction to the actions, the consultants assessed the actions against agreed criteria, discussed his/her evaluation with the relevant EC official(s) and then finalised the fiches. 55 56 57 58 59 60 In total, we reviewed 90 files, 53 1993 actions and 37 1994 actions. We also reviewed the studies and projects selected in 1995, but since these actions are just starting, our review was limited to the terms of reference and the contractors' proposals. We also carried out ten case studies, of which three were "thematic" case studies. All actions have been reviewed. However, an assessment for each policy area specifed in the Plan was not practical because a number of groups of measures (Staggering holidays, Tourists as consumers, Rural tourism and Youth tourism) include only a very small number of actions. These four groups represented only 3. 2% of the total spend under the Plan. In this report, we focus on the other policy areas but make reference to specific actions included in these groups of measures where appropriate. 3. 2 - Overall Scores Tables 3. 1 to 3. 3 give a summary of the main results for the projects and studies funded in 1993 and 1994. In order to avoid any misinterpretation of these findings, it is important to emphasize that the overall score attributed for each action is based on a large number of criteria (see Appendix III). In particular, both the rationale of the project (ie does the project make sense?) and its effectiveness (achievement of planned objectives) are taken into account. Therefore, this score is OQÎ solely a judgment on the contractor. A score of 5 means "very good". A score of 3 means "average", and less "poor" or "very poor". We classified a satisfactory action - which, using an examination metaphor, would be a pass but without honours - as "average" (score of 3). "Average" is therefore not to be confused with an arithmetical mean. In that sense, one should ideally expect most actions to achieve a score above 3. This technical assessment, although expressed for convenience in quantitative terms, is a subjective one. Criteria against which actions are judged are mixed and the actions themselves have been concluded recently, so that their longer term impact is uncertain. It must be noted that we consider that the criteria used are not additional to the ones specified in Article 4 of the Council Decision, but that they are integral part of the Plan as a whole and implied (on any rational assessment) in most of the projects themselves. For example, how can any project involving database creation be cost effective (Article 4) if there is no built in mechanism to update information which by definition will be quickly out of date? 61 In 1993, the average scores were 2. 9 for overall assessment and 2. 7 for cost effectiveness. In 1994, both scores increased to 3. 4 and 3. 0 respectively. In other words, the overall quality of the projects 12 Final Report - V2. 0 EVALUATION OF THE COMMUNITY ACTION PLAN TO ASSIST TOURISM FINAL REPORT SECTION 3 - TECHNICAL EVALUATION OF ACTIONS • improved considerably in 1994 (reflecting improved selection procedures) and the cost effectiveness also improved. The overall number judged good or very good increased from 34% to 44%. Table 3. 1 - Overall Technical Assessment of 1 £ 93 and 1994 Ac tions. Breakdown by Year (% of total) Poor or very poor Average Good Very good Not relevant or not possible* 1993 36% 25% 19% 15% 5% (53 actions reviewed) 1994 8% 46% 30% 14% 2% (37 actions reviewed) * In a small number of cases, we consider that it was not possible to give a credible/robust assessment given the available information. 62 In Section 3. 4 we discuss the main factors explaining the variations displayed in Table 3. 1. Table 3. 2 gives a breakdown by policy area. It shows that the group measures "Improving knowledge of the tourist industry" has performed, on average, better than the other groups of measures. In 1994, the actions undertaken in the field of cultural tourism have, on average, been of good quality. * Table 3. 2 - Overall Technical Assessment of 1993 and 1994 Actions. Breakdown by Type of Action 1993 Group 1 ** Overall Assessment Cost Effectiveness Number of actions 3. 3 3. 3 9 Groups 2 and 4 No action funded Group 3 Group 5 2. 9 2. 6 17 2. 8 2. 6 18 1994 Group 1 Groups 2 and 4 Group 3 Group 5 Overall Assessment Cost Effectiveness Number of actions 3. 8 3. 0 6 3. 0 2. 0 3 Only two actions 3. 5 3. 3 12 Groups 6 to 11 2. 7 2. 6 9 Groups 6,9 and 11 4. 3 4. 5 4 Group 10 All Actions Only two actions, included in groups 6 to 11 2. 9 2. 7 53 Group 10 All Actions 2. 7 2. 2 10 3. 4 3. 0 37 * We do not report scores on groups of measures with less than three actions. ** i=lmproving knowledge; 2=Staggering holidays; 3=Transnational; 4=Tourists as consumers; 5*Cultural tourism; 6=Tourism and environment; 7= Rural tourism; 8=Social tourism; 9=Youth tourism; 10=Training; 11 «Promotion. Group 3 is a composite group of measures; "transnational" actions scored on average higher than the two other types of actions, "technology" and "cooperation with Eastern and Central European countries". 63 Table 3. 3 gives a breakdown by size, expressed in Ecus. It indicates that there was in 1993 a much larger proportion of small actions (38% against 19% in 1994 and 23% in 1995); that, in 1994 and 1995, large actions (above ECU 100000) represented about one half of the total; and that, in 1993, smaller projects achieved better results and were more cost effective. However the negative relationship between size and (cost) effectiveness is less marked in 1994. 13 Final Report-V2. 0 44 EVALUATION OF THE COMMUNITY ACTION PLAN TO ASSIST 1 OURISM FINAL REPORT SECTION 3 - TECHNICAL EVALUATION OF ACTIONS <§ Table 3. 3 - Overall Technical Assessment of 1993 and 1994 Actions. Breakdown by Size (ECU) Up to 50000 50001 to 100000 100001 to 400000 Above 400000 21 38% 3. 5 3. 9 7 19% 3. 9 4. 1 16 23% 11 20% 2. 9 1. 9 12 32% 3. 3 2. 8 17 25% 21 38% 2. 2 1. 7 18 49% 3. 3 2. 6 33 48% 2 4% 0 3 (4%) 1993 Number of actions As % of total reviewed Overall Assessment Cost Effectiveness Average size: 121144 1994 Number of actions As % of total reviewed Overall Assessment Cost Effectiveness Average size: 117030 1995 Number of actions As % of total Average size: 127015 3. 3 - Case Studies 64 Tables 3. 4 (parts A and B) give a summary of the main findings of the case studies carried out. 14 Final Report - V2. 0 EVALUATION OF THE COMMUNITY ACTION PUN TO ASSIST TOURISM FINAL REPORT SECTION 3 - TECHNICAL EVALUATION OF ACTIONS • Table 3. 4 - Summary of Findings from the Case Studies (Part A) Project Description Assessment & Key Learning Points European Project for the Training of Operators In Tourism in Bulgaria and Slovakia Strengths - Significant involvement of local communities; successful pilot, new course material (fully customized) prepared with possible dissemination; achievement indicators indicate targets were met (No of attendees, assessment of university partners, response of participants); site visited by DG XXIII; good press coverage in Bulgaria Weaknesses - Innovative character limited; project delayed due to slow response by DG XXIII; project takes place in isolation; no follow up, no strategic framework; no local presence of EU delegation; expensive project Learning Points - Need to take trainees' needs into account; use of "consecutive translation" (instead of simultaneous translation) was a success; ex post evaluation of impact needed; any future action must be co-ordinated with PHARE (DG I); use local PR consultant for visibility. Tourism & Environment Prize Strengths - Important project with high visibility and potentially large spin off effects; overall good organisation; good co-operation with experts; if repeated and improved, could raise profile of DG XXIII; use of Euro Info Centres was a success; very good logo; good contribution of the PR firm Weaknesses - Difficulties in co-operation between main contractor and DG XXIII; lack of experience of main contractor; concepts and strategy slow to develop; mobilisation of Member States ("Steering Groups") poor in certain cases; weaknesses in selection of projects (mainly because of a language problem); political and geographical mix considerations; proposal/contract did not specify procedures/arrangements for follow up and supervision j Learning Points - For maximum visibility, need dedicated staff in every participating country; suggestion to modify the format to adopt a "Festival de Cannes' format with several prizes; publication of Newsletter good idea; consider translation all short listed applications in a common language; need to improve the checklist used for screening and evaluation; better synergy with other projects in field; more time needed for applications to be prepared; more involvement of the experts needed ; should become a focal point of DG XXIII's activities. Tourism Promotion of Europe as a Destination for Tourists in Long Distance Source Markets Strengths - Good logo and tagline; highly rated consultants used; good idea to use themes to sell Europe as a destination to avoid national sensitivities Weaknesses - Widely perceived by the travel industry as a waste of time and money; no sponsorship obtained; are logo and tagline suitable for all markets? Conflict with ETC Learning Points - "Pilot" wasted because not followed up; long term view and adequate resources needed for a promotion to yield results; it is a challenge to sell Europe as a destination; need to use existing networks and to involve NTOs more closely; a campaign without sponsors does not make sense. 15 Final Report-V2. 0 UL EVALUATION OF THE COMMUNITY ACTION PLAN TO ASSIST TOURISM FINAL REPORT SECTION 3 - TECHNICAL EVALUATION OF ACTIONS • ETI - European Tourism Information & Booking Strengths - Good partnership structure and international network; standardisation of existing local tourism information services can be a key factor in encouraging use of technology by tourists; vision of a European-wide network providing standardised information for tourists; link with other initiatives such as EDITIN Weaknesses - Innovative character of the technical solution now largely obsolete because of the development of the Internet (we recognise that Internet has become only recently a viable alternative); sustainability and follow up not ensured but meeting between partners planned for March 1996; not all outputs promised in the proposal delivered (eg 60 outlets); involvement of two of the partners very limited Learning Points - Technical pilot studies in new technologies quickly obsolete; this problem was compounded by delays in implementation, and thus need for DG XXIII to fund is unclear unless objectives are to establish network and raise awareness; need close links with other programmes in the field (eg TIM); project underlined the role of DG XXIII as overall co-ordinator, facilitator of trans-regional project. Strengths - Strong rationale because addresses a key problem ie youth unemployment; several interesting ideas/concepts and innovative solutions developed; long term impact might be high; good follow up and strong local support in Denmark; Weaknesses - Proposal of very general nature ("aim is to develop a concept"), no clear description of outputs; not a pilot project, more like fundamental research and network building; follow up in UK, NL and D uncertain because weak local support as illustrated by poor attendance at conference in Denmark; partnership more at the stage of research and intellectual debate than operational partnership to design and pilot schemes; expensive project given outputs are mainly local and uncertain in other countries Learning Points - DG XXIII in fact funded the development of a network of partners linked more by the similarity of problem they address than by a common operational project: this may be useful but was it the role of the Plan and of DG XXIII? Development of a Training Strategy fa Employees and Trainers within the Tourism Sector in the Cold Water Regions around the North Sea The "Art Cities in Europe" Programme Strengths - Innovative basic idea developed in a few years into a good product supported by a large network of cities; members of the network comprise both famous and less known cities which may benefit from being linked with the established cultural cities; good distribution and marketing; links with key operators (travel agencies, tour operators); impressive project management and efficient implementation Weaknesses - Proposal focusing more on means than on objectives; a number of key cities and important cultural institutions not yet (or no longer) in the network; benefits accruing to the small cities still uncertain; success partly depending on ability of cities to generate bookings Learning Points -1996 will be an important year for the future of the project which must now demonstrate its commercial viability, ie generate bookings; level of development of network of tourism offices varies widely from one city/country to the other one. thus need to strengthen structures in ceratin cities/countries; US market must be covered to create additional effect not diversion; travel agencies and tour operators must be convinced of the commercial viability of the project; need to have a key attraction. Yield Management in Small and Medium Sized Enterprises in the Tourism Industry Strengths - High quality final report with good executive summary for dissemination of key findings; concept of yield management powerful Weaknesses - High cost, particulalry because unclear if the concept is relevant for SMEs Learning Points - Poor relevance for SMEs should have been anticipated (eg through discussions with experts); the study should have been redefined to meet the needs of the smafl hotels in particular, many of which have no computerised reservation systems, or on-line reservations capability. 16 Final Report -V2. 0 o EVALUATION OF THE COMMUNITY ACTION PUN TO ASSIST TOURISM FINAL REPORT SECTION 3 - TECHNICAL EVALUATION OF ACTIONS • 3. 4 - General Assessment Projects and Studies Undertaken in 1993 and 1994 65 66 67 68 69 70 The overall quality of the projects and studies carried out is poor to average (24% of the 1993 and 1994 actions reviewed are assessed as very poor or poor, 22 in total) while 14% (13 in total)of completed projects were judged to be very good. There has been, however, a significant improvement in 1994. In most instances, the impact of the action is limited in space, scope, and time. There were numerous financial commitments (a total of 59 in 1993, 58 in 1994 and 74 in 1995), in our view too many in relation to the Tourism Unit's resources to adequately define and monitor them. Most actions had a size too small to make a significant impact, although a number of successful small actions were very cost effective. The average size of the actions we evaluated was ECU 121144 in 1993, ECU 117030 in 1994 and ECU 127015 in 1995 (see Table 3. 3 above). Most actions can be classified into two groups. On the one hand, there are actions fulfilling political objectives; these actions promote "European" concepts and networks, and are of a fairly high profile and newsworthy; on the other hand, there are commercially interesting actions, with sustainability built in. These two groups can occasionally coincide in instances such as the project "Routes to the Roots", a 1993 project conducted by the University of Oldenburg. I The promotion of "transnational networks" is, in our view, not an objective, but a means to achieve an objective. There is an obligation, under Article 4 of the Council Decision, for cost effective measures. The problem with this is that the political objectives eg of promoting transnational networks, may be achieved better and more cost effectively through other means and do not need the cover or justification of a pilot project which is only occasionally really innovative. Alternative actions that could have been considered to promote European concepts and networks include: a "roadshow" going round each European country with a high class presentation of European issues and challenges for the sector, and including a European agenda; hiring a public relations company with appropriate terms of reference and budget; working with existing database providers across Europe to put together a consolidated database of European research; launching a Newsletter etc. Studies A number of studies were funded to look at major issues of importance to the tourism sector. These were generally well executed, even if the choice of topics (youth tourism, business and conference, yield management, for example) did not appear to be based on a careful analysis of needs taking into account existing available data. We are aware that the topics relating to studies were proposed and/or agreed by the Management Committee. We take the view that more time must be used to define precisely the studies to be funded, taking into account existing studies, ongoing research programmes, and the travel industry needs. For example, the value of the study on conference and business tourism is limited by the fact that this subject area is already quite well researched. Examples of recent studies are: the 1990 survey of European Incentive Travel by Touche Ross; the 1994 study by the Economist Intelligence Unit - EIU on the European Conference and Meetings Market; a 1993 publication by the Maison de France on the "Tourisme de Congrès". 71 Another example is the study on yield management (one of our case studies), which is an example of a well executed study. However, the relevance of the concept for SMEs is limited. The study focused 17 Final Report-V2. 0 13 EVALUATION OF THE COMMUNITY ACTION PUN TO ASSIST TOURISM FINAL REPORT SECTION 3 - TECHNICAL EVALUATION OF ACTIONS • on the use of computer systems, but most establishments are small and they have no such systems applied to reservations. At the same time, they already apply (to a greater or lesser extent) yield management for instance when a client can "negotiate" a favourable room rate at an off peak period. Most hotels and other accommodation establishments are independent and have rather few rooms. They lack marketing strength, including access to computer reservation systems. They are ways to help them, for instance in promoting group marketing exercises such as the Gîtes de France scheme. A number of projects and studies had as their objective the preparation of reference lists of potential interest to the sector, for instance a Who's Who of personalities and organisations, a multilingual Thesaurus of tourism terms, and a database of student employment opportunities. All of these suffered from a lack of in-built post project sustainability, which are essential when data need frequent (and often expensive) updating. Some of these projects and studies competed with existing commercially available products. These studies had rather low viability and served little useful purpose. Guides to make Europe accessible to disabled tourists (one for the disabled, one for the tourism industry) and "the study "Older persons and mobility" are examples of excellent and cost effective studies emerging from the Plan. The guide books for the blind are further examples of the strong contribution made through the Plan for tourists with disabilities- Cultural Tourism I Most of the cultural components of the Plan focused on building-up transnational routes and networks based on a range of themes. Most of these were relatively low cost exercises, which produced interesting results. At the same time, the sustainability and impact on tourism demand are likely to be limited, and the main longer term benefits may arise from the strengthening of linkages between regions. If there is no intention to secure a sustained economic benefit, then this must be made clear and the actions must be judged on that basis. But the need to be cost-effective remains. If the strengthening of linkages is the main output, then it is one that may be achieved in other more cost effective ways, for instance by organising conferences. Designing routes and trails is in itself not- sufficient, interesting as these may be, to justify project funding. For example, the route linking Wexford in Ireland to Dover in England, France, Portugal and Spain is likely to have very limited value except in bringing people together. But it was an expensive way to achieve this. If we take post project sustainability as a measure of success, then the "Routes to the Roots" project (first funded in 1993 with an extension in 1994) is noteworthy. It has developed a commercial momentum indicative of longer term sustainability. The project "Art Cities in Europe" (one of our case studies) may also develop such a commercial momentum although it too early to define exactly what it may be. Another example of a successful cultural route project was that undertaken by Naturfreunde Landesberband Saarland (1993) which linked a sensible set of parties, transnational but in relative close proximity, and created a range of interesting routes with excellent prospects for sustainability. The Province of North Holland project (1993) is one example of a feasibility study, in this case for cultural trails relating to military heritage in three regions. It had the advantage that the nature of participants (local authorities involved in tourism) does mean that there is a reasonable prospect of it being sustained. 72 73 74 75 76 77 78 Much of the work on routes can be classified, and is perceived as essentially promotional within Europe, and its impact may largely be to divert existing tourists along the promoted routes. The 18 Final Report -V2. 0 hf EVALUATION OF THE COMMUNITY ACTION PUN TO ASSIST TOURISM FINAL REPORT SECTION 3 - TECHNICAL EVALUATION OF ACTIONS • impact of these routes may not be additional. This does not apply to the routes targeted at non- European markets (such as "Routes to the Roots" targeted at the North American market). We do recognise, however, that the diversification of tourist destinations within Europe is also a legitimate objective. It is also noted that the concept of routes and trails of linked destinations was already well established before the Plan, and the marginal value of creating further routes/trails may be limited, and should thus be assessed. The innovative aspect of the DG XXIIPs funded trails is their transnational design. The longer impact is still unknown and needs research. Pilot projects are, by definition, test cases. There is no case for funding innovative and speculative projects without proper post project follow up, in detail, by independent evaluators, followed by publication and dissemination of findings. Apparently successful projects can then be replicated elsewhere with a further evaluation process, leading to a definite conclusion. Training The scope of the education and training projects has been quite wide, including development networks, curriculum definition, and identification of new professional needs. Those networking projects which seem the most successful have been the ones reinforcing existing networks, while the creation of new networks is judged in general less successful on sustainability and other grounds. Where projects have focused on curriculum needs with European commonality - for instance curriculum development for sustainable tourism by the University of Deusto-Bilbao (1994 project)- they have served very useful roles. Less successful in contrast have been a number of projects which have focused on the identification of new professional categories, for instance Club des Métropoles project (1993) and the Greta du Golfe project (1994), and the work by Conzorcio Civita (1994) on new professional categories. They have also been a number of one-off training exercises (eg in Eastern and Central Europe) which although in themselves useful, do not fit within an institutional or other framework providing a basis for follow up and sustainability. Though rather in isolation they may have been effective, although they did not fit into a coherent plan and are best avoided in the future unless mechanisms to ensure sustainability are put into place. An example is the 1993 project undertaken by L&R Leisure in Bulgaria and Slovakia. It was well executed, but had little innovation and no institutional funding availability for follow-up on site post project. The Tourism Belarus project implemented by Frotsi du Limousin (1993) was rather similar, but its budget more modest. Technology The technology pilot projects supported under the Plan are broadly either generic, addressing key issues such as data definition procedures for information exchange; or specific in seeking to apply new technology to specific situations. This distinction generic/specific broadly coincides with project responsibilities with DG XXIII, where the essentially generic projects (such as CIRT, EDITIN, ET1, TIRS, all good projects) are considered as "technology" projects and classified under the policy area "Transnational measures". A number of projects in the cultural area are dominated by technology solutions and from these specific projects 79 80 81 82 83 84 85 86 19 Final Report -V2. 0 EVALUATION OF THE COMMUNITY ACTION PLAN TO ASSIST VOURISM FINAL REPORT SECTION 3 - TECHNICAL EVALUATION OF ACTIONS • 87 88 89 90 91 92 93 94 important conclusions relating to their technology can be drawn. Specific examples often include the design of multimedia databases of product information, such as the project Odyssey by International Interactive Multimedia (1993), and the feasibility study on music festivals by Sygna (1994). In contrast to the cultural activities funded, there are indications that one way or another, many of the "generic" technology projects funded would have in due course proceeded without EC funding, since they addressed issues of industry significance, such as globalisation of data or access advances. The EC funding, while contributing very usefully to addressing these issues, could be described as supportive rather than essential. It has however had a positive impact in accelerating the application of new technology to tourism-specific tasks (for example, the 1993 project EDITIN). The more specifically targeted technology-rich projects have mostly focused on transnational databases, often multimedia. These projects, while addressing different specific tasks, have to some extent duplicated each other in terms of the technical approaches and software used to implement them. In this sense, the most innovative aspects have been their transnational character, and the attempt to apply new technology to new areas, such as information handling, previously addressed through guide books and maps. These specific projects have not been particularly successful because there is a strong likelihood that most will wither following completion of project funding because they lack any in-built sustainability or commercialisation component. These projects appear to have been largely designed to "access" EC funds, rather than to address specific needs. The 1993 project for a multimedia CD Rom presentation of cities linked to the Odyssey (conducted by International Interactive Multimedia) is technically similar to the project conducted by Turinform (EPTIS, 1993) and the Fenici Route (Consorzio Civita, 1993). All three cover widely dispersed tourist centres, and none have strong commercialisation or sustainability prospects. All three give the appearance of projects designed specifically to respond to the availability of EC funding. These projects could usefully be followed up to identify lessons that can be learnt. The above three projects compare unfavourably with the Infonet project (TIRS, 1993) which is designed around a more logical geographical grouping of parties (Saarland in Germany and Lorraine in France) and has a clear link with existing database systems, ensuring long term sustainability. Running through many of the new technology oriented database projects is apparently a failure to address post project sustainabilty. Tourism databases need constantly updating and administrative and financial mechanisms for achieving this are as important a component of project design as the technical components. These follow up and commercialisation issues were in the majority of cases not satisfactorily addressed, with as a result a number of interesting projects completed, but having most probably no lasting contribution to make to the tourism sector. Some of the technology projects were essentially feasibility studies, sometimes for projects which have a long term objective behind them (eg Sygna on new technology in cultural tourism). The results may be interesting but question marks are attached in respect of ownership of results (particularly when co- financed) and the associated confidentiality issues. A further example of such a feasibility study is the Eurolodging project conducted by AGTE (1993). 95 One issue that was not specifically addressed in the design of the technology and other projects was 20 Final Report-V2. 0 9/ EVALUATION OF THE COMMUNITY ACTION PUN TO ASSIST TOURISM FINAL REPORT SECTION 3 - TECHNICAL EVALUATION OF ACTIONS • 96 97 98 99 100 101 102 the question of relevance to SMEs. Technology offers both a barrier and an opportunity to SMEs, and this issue has not been satisfactorily addressed. Some projects omit any link to suppliers at all, concentrating on tour operators and travel agents, and omitting hotels. The Internet, with its low cost of entry, represents, it can be argued, a major area of potential for SMEs, yet there has not been any project, so far, addressing this area. The Eurogites project (1993) is an example of one project successfully addressing the specific needs of an important SME sector (Bed & Breakfeast and rural accommodation. ) It is also noted that other directorates within the Commission are financing database projects targeted at the needs of the tourism sector (eg SAM), yet the compatibility, complementarity and competitive aspects of the schemes funded by DG XXIII have not, to the best of our knowledge, been explicitly examined in relation to these. Actions Initiated in 1995 We give below comments on a selection of the actions initiated in 1995. The programme for 1995 is dominated by the statistical programme, in particular support to the individual countries and major contracts with consultant groups (eg Gruppo Clas) as these countries get their statistics into line with the agreed format set out in the Directive. There is a demonstration "Heritage trails" project (conducted by CTA) which is interesting. As a pilot project it is to be hoped that sustainability issues will be fully addressed, and that independent post project review - covering also similar 1993 and 1994 projects - will be in due course undertaken to draw conclusions and learn lessons. Another significant project relates to Cyprus, where the objective is the development of cultural and special interest tourism, with a focus on SMEs. Another project extending beyond the EU proper covers thermalism in the Eastern and Central Europe and in the Mediterranean area. An "exchange of views" is a major output identified. The project should also include a formally published report documenting overall market characteristics (demand and supply on an European basis) including a description of the pilot thermal plants to be studied, together with an assessment of overall future prospects for this market segment. The Network of Tourism and the Environment (EcoNett) being developed by WTTC represents a major commitment (ECU 631000) whose potential for success is strengthened by the strong industry backing of WTTC. This project has high visibility and good sustainability prospects. To what extent WTTC's EcoNett will serve as a practical tool, accessible and useful to not only large multinational organisations but also to modest tourism related businesses (small hotels, restaurants, attractions) is difficult to judge at this stage. The 1995 programme covers other important topics such as transport management (eg Airport of Palma de Mallorca), and visitor management. In the latter category, the research and piloting to be undertaken by the Tourism Company ("Demonstration of the potential for voluntary financial schemes within Europe") addresses a very important issue, ie funding mechanisms for environmental conservation. Although projects are still at an early stage, the outcome looks less promising for other projects dealing with visitor management, such as the "Mantova Ticket Tour" and " HELLO tourism". We would have preferred to have seen earlier projects of this kind (eg in Germany, Northern Italy and Austria) subjected to a detailed independent technical and financial evaluation before proceeding further down this route. 21 Final Report-V2. 0 C 7 EVALUATION OF THE COMMUNITY ACTION PUN TO ASSIST TOURISM FINAL REPORT SECTION 3 - TECHNICAL EVALUATION OF ACTIONS • Other Actions Carried Out Under the Plan 103 104 Activities undertaken under the Plan are described in detail in the various reports to the Council and the European Parliament prepared by DG XXIII. The Tourism Unit has prepared two "horizontal reports" on Community measures affecting tourism, in 1994 and 1995. They are comprehensive and very informative. The 1994 report is almost exclusively descriptive and does not include an assessment of the Community measures; the 1995 report (still a draft at the time of our evaluation of the Plan) tries to address this weakness. 105 In Table 3. 4 we summarise the findings (assessment and key learning points) from the three "thematic" case studies carried out, ie: a b c the development of Community tourism statistics the selection procedures (based on the 1995 environmental projects generally and on the WTTC's "European Community Network for Environment Travel & Tourism" project more specifically) the internal co-ordination by the Tourism Unit with other Directorates General of the Commission (activity connected with the preparation of the "horizontal report"). 106 We describe below the achievements in the area of Community tourism statistics, where achievements were significant and co-ordination by DG XXIH effective. 107 108 109 The approach adopted by DG XXIII for the introduction of a common statistical set was to work closely from the beginning with the key existing stakeholders, Eurostat at the Commission level and the National Statistical Authorities at the National level. At the national level, there are practical commitments to longstanding data series reflecting their specific needs, and representing substantial investments, and there are also different emphases concerning the data collected, reflecting in particular geographic realities. For instance the Irish and UK data sets reflect the relative ease of collection of demand data in Island situations, while continental Europe has tended to concentrate more on supply side measurement. Data collection is a necessary but expensive procedure in the tourism sector, and it is considered that the approach adopted by DG XXIII, following a long period of consultation with National Governments, and with the Commission's own Statistical service (Eurostat), was appropriate in the circumstances. The approach essentially focused on agreeing a common core minimum data set to be collected. Around this common core, related but optional data sets have been defined, again with commonality of definition, so that overall comparability is ensured. 110 While the overall co-ordination has been by DG XXIII, the approach has been essentially tripartite (DG XXIII, Eurostat, and national governments), and given the challenge represented in reaching agreement for a common approach covering such a wide range of countries, the achievement has been very significant. A Directive on collection of Statistics has recently been issued (23/11/1995 and will come into force by the end 1996), and Governments are now receiving financial support under the Action Plan for bringing their respective systems into line with this. A total of 4. 5 MECU has been committed to this task over a three year period (1995-1997). 22 Final Report - V2. 0 c EVALUATION OF THE COMMUNITY ACTION PUN TO ASSIST TOURISM FINAL REPORT SECTION 3 - TECHNICAL EVALUATION OF ACTIONS 41k ^ilF 111 The process of identifying and defining data sets, and responding to the methodological problems and hurdles presented by varying conditions in member countries, has been assisted by external consultants, selected by Eurostat and contracted on a trilateral basis. Consultants have also assisted in the definition of appropriate dissemination procedures, in particular for the Statistical Yearbook for tourism, and for the publication Tourism in Europe". The most recent edition of this excellent publication appeared in 1995, and it is scheduled for annual publication in future. 23 Final Report-V2. 0 St EVALUATION OF THE COMMUNITY ACTION P UN TO ASSIST TOURISM FINAL REPORT SECTION 3 -TECHNICAL EVALUATION OF ACTIONS 0 Table 3. 4 - Summary of Findings from the Case Studies (Part B) Description of Action Assessment & Key Learning Points Theme 1 : Development Ot C o m m u n i ty tourism Statistics Strengths/Achievements - Overall, achievements are very significant and the approach appropriate; Directive "welcome as a first step"; co-operation with other organisations (OECD, WTO, and of course Eurostat and national institute of statistics) takes p|a ce eg to avoid duplication of efforts and develop core common set of data and methodologies; closely with the key existing stakeholders, Eurostat at the Commission level and the National Statistical Authorities at the National level. Weaknesses - Methods for implementation (Eurostat, National Institutes of Statistics) slow and "bureaucratic"; certain interviewees feel that the Directive has taken the "lowest common denominator"; a number of areas perceived as not properly covered, such as inbound tourism for long haul tourism and more generally on the demand side; lack of ambition, too conservative an approach? But it is also recognized that going too fast/far may put the whole endeavour at risk; political factors also slowing the process Learning points - Must be a focal point for DG XXIII because of clear need, demand expressed by industry, unanimity among member states and industry participants; use of private statistical organisations raises controversy, being supported by a number of organisations representing the private sector while being questioned by interviewees from international public organisations; however, there is a consensus for more co-operation private/public in particular for demand statistics. T h e me 2: Selection p r o c e d u r es (based on 1995 environment projects). Strengths/Achievements - Establishment of network will receive EU support for three years; contractor is a major international professional body ie good trade links and political strength; contract specifies project outputs and the content of progress and annual reports; annua) audited financial statement to be produced annually ; involvement of DG XI, DG XII and tourism industry Weaknesses - How will the data base be updated regularly? What is the incentive for suppliers to provide data on a regular basis to maintain the database? The exact coverage of the concept of "environmental travel" needs to be specified Learning points - Need to set up mechanisms to ensure that the database is maintained and regularly updated. T h e me 3: Internal CO-ordinatlon by Tourism Unit Strengths/Achievements - Increasing awareness of the importance of co-ordination with other EU departments; Inter Service Group (ISG) meet regularly; DG XXIII "good at organising"; good bi-lateral contacts; there is a "permanent flow of information" between departments; consultation for the Green Paper was a success; good co- operation with DG V in the context of the "HELIOS" group; General Directors of DG V and DGXXIII to co-preface publications Weaknesses - Participants to ISG not well prepared, but situation improving; turnover of representatives for some DGs and thus lack of continuity; DG XXIII staff perceived as over-worked, "not always able to cope" Learning points - ISG should be better prepared by participants (DG XXIII normally sends the agenda and relevant documents about ten working days in advance); interest in ISG meetings increasing as participants more familiar with issues; thematic discussion well received; Suggestion of format made by one interviewee: 1) information given by DG XXIII, 2) round table on work in progress in other departments, 3) take up precise points and questions; Unit needs more tourism specialists to be able to claim a co-ordinating role within EC; need to designate a unique, permanent correspondent/representative to ensure good follow up. 24 Final Report - V2. 0 EVALUATION OF THE COMIVIUNITY ACTION PLAN TO ASSIST TOURISM FINAL REPORT SECTION 4 - ADMINISTRATIVE AND OPERATIONAL ASPECTS • SECTION 4 - ADMINISTRATIVE AND OPERATIONAL ASPECTS 4. 1 - Description of the Administrative and Management Arrangements 112 113 114 115 116 Administrative arrangements, financial resources and activities undertaken under the Plan are described in detail in the 1994 and 1995 "horizontal" reports to the Council and to the European Parliament prepared by DG XXIII. In this section we give an overview of the main arrangements in place; we then give our assessment. Management, Advisory and Consultative Committees The Commission is assisted in implementing the Plan by the Management Committee composed of representatives of the Member States of the European Union (Council Decision of 13 July 1992). In January 1994, membership of the Management Committee was broadened to include the member states of the Economic European. Area, and, in January 1995, to include Liechtenstein. The Management Committee met on nine occasions over the three years, twice in 1993, four times in 1994 and three times in 1995. An earlier Council Decision of 22 December 1986 established an Advisory Committee under the auspices of the Commission, made up of members designated by each Member State. The task of this Committee was to facilitate exchange of information, consultation and cooperation on tourism, and, in particular, on the provision of services for tourists. This Committee met four times in 1994 and three times in 1995. It did not meet in 1993. Also prior to the launch of the Plan, the Commission established a Consultative Committee made up of the representatives of various industry interests. The Consultative Committee has no legal basis and was established by the Commission to allow dialogue and consultation with the industry. During the implementation of the Plan, the Consultative Committee met on five occasions, once in 1993, twice in 1994 and twice in 1995. Staffing The number of staff employed in the Tourism Unit (permanent and temporary staff, all grades and statuses, except stagiaires) was 17 in 1993 (11 grade A, one grade B and five grade C), 14 in 1994 (11 grade A, one grade B and two grade C), 16 in 1995 (11 grade A, one grade B, and four grade C) (situation on 1 January). At the end of 1995, the number of staff employed was 17 (13 grade A, one grade B, and three grade C). (Grade A staff are the administrators and above, grade B the assistants and grade C the support staff. ) 117 A total of 29 stagiaires were employed for periods of three, or, more usually, five months. 118 119 120 In 1994, the head of the Unit was transferred, and later dismissed from the Commission following disciplinary hearings. His acting deputy was suspended in 1994 and later dismissed. A total of 36 different staff were employed in 1993-95. Only five staff members were employed by the Unit through the whole 1993-95 period. Staff turnover was thus high at 86%. The ratio of permanent staff to total staff (excluding stagiaires) was 5:17 at the start of the Plan and rising significantly to 11:17 by the end of 1995. 25 Final Report -V2. 0 Ç6 EVALUATION OF THE COMMUNITY ACTION PLAN TO ASSIST TOURISM FINAL REPORT SECTION 4 - ADMINISTRATIVE AND OPERATIONAL ASPECTS • 121 122 123 Out of the 36 staff members, we estimated that ten (28%) can qualify as "tourism experts" (we define "tourism expert" as someone who is seconded from a Ministry of Tourism or equivalent, who has worked in the industry, or has a degree in the field). Certain other staff members had gained tourism experience "on the job" before joining the Unit. Implementation Mechanisms and Administrative Procedures The Plan was implemented via three annual work programmes. A "timetable of priorities" for 1993 was specified in the Plan. The priorities and work programmes for 1994 and 1995 were prepared by DG XXIII and agreed with the Management Committee. 124 The funds were disbursed through five different mechanisms: a b c d e open calls for tenders (published in the Official Journal) for 100% funded actions. A total of 15 projects were financed through this mechanism (7 in 1993, 4 in 1994, 4 in 1995), representing a total spend of ECU 3. 68 million open calls for proposals (published in the Official Journal) for co-financed actions (jointly- funded up to a maximum of 60%). In total 74 were co-financed (35 in 1993, 23 in 1994,16 in 1995), representing a total spend of ECU 8. 45 million 25 co-financed projects were supported via DG XXIIPs Subvention Committee, ie without a competitive process (14 in 1993, 7 in 1994 and 4 in 1995) representing a total spend of ECU 1. 43 million 73 direct contracts of which 43 were for the development of Community statistics (2 in 1993, 20 in 1994 and 51 in 1995) representing a total of ECU 5. 95 million in total, of which 84% went to the Member States and restricted selection procedures were used in five cases (two pilot projects and three conferences), amounting to ECU 251000. 4. 2 - Assessment of Arrangements in Place Our assessment is based on a desk analysis of procedures in place and of staff statistics; on the evidence collected through the case studies and project file reviews; and on the results of our interviews with the Tourism Unit staff, members of the Management Committee, and members of the Consultative Committee. We also reviewed most minutes of the meetings of these Committees. Relations with External Bodies The contacts with the travel trade have not been strong enough. The main non -governmental contacts are with the associations and pressure groups of various kinds and with consultants. The Unit does not seem to have a clear and operational vision of current "market positioning" vis-à-vis competing and complementary service providers such as: National Tourism Offices (NTOs), international associations of importance (such as ETC, IATA, WTTC, WTO), internal competing and complementary service providers such as DG I, DG VIII and other EU directorates. 125 126 127 26 Final Report -V2. 0 EVALUATION OF THE COMMUNITY ACTION PLAN TO ASSIST TOURISM FINAL REPORT SECTION 4 - ADMINISTRATIVE AND OPERATIONAL ASPECTS 41k fllF Administrative Arrangements and Procedures 128 129 130 131 Administrative arrangements and procedures were weak at the beginning of the implementation period, but have been significantly improved in 1994 and 1995. Members of the staff are clearly very committed and hard-working. Staff members are overloaded with administrative tasks the rationale and effectiveness of which are not always evident. The balance between quality work (project preparation, site visits, contacts with contractors etc) and administrative tasks is unfavourable. Communication lines at DG XXIII are lengthy, procedural and slow, and so is the decision-making process. The morale of the staff is rather low and staff members are displaying an exemplary resilience. Low staff morale is partly explained by the factors described above (administrative workload, lengthy communication lines etc), but also by the negative impact on the Unit of the allegations relating to senior staff members which have tarnished its reputation and image. 132 We observed that the amount of administrative work performed by the Unit is considerable. The Unit estimates that they receive around 100 external letters weekly. Telephone calls come in "non stop". Parliamentary questions come in regularly, usually with tight deadlines. Many documents must be translated and often re-formatted because of a software conflict between the Translation Office in Luxembourg and DG XXIII. 133 We also identified the following shortcomings in the area of staffing: a b c d e high turnover low percentage of "tourism experts" and generally inadequate presence of tourism skills inadequate training (eg in area of financial and project management) understaffing for the financial administration high number of stagiaires: it takes time to supervise them if they are to be used effectively. 134 Given the limited staff resources available at the Unit, it was ill-advised to implement the Plan through the funding of such a large number of projects and studies (mainly through open calls for proposals) because of the heavy administrative burden implied by such an approach as illustrated by Table 4. 1 which gives an analysis of the results of the various calls for proposals published in the Official Journal. 27 Final Report - V2. 0 a EVALUATION OF THE COMMUNITY ACTION PLAN TO ASSIST TOURISM FINAL REPORT SECTION 4 - ADMINISTRATIVE AND OPERATIONAL ASPECTS • Table 4. 1 - Project Se ection. Calls for Proposals (Part-Funded Projects) 1993 1994 1995 Stage 1 Formal Conditions 368 (86 failed) 281 ( 28 failed) Stage 2 Screening 282 (158 failed) 253 (139 failed) Stage 3 In-Depth Evaluation Final Selection 124 (63 failed) 114 (93 failed) 61 shortlisted 33 selected 21 selected 288 (39 failed) 249 (130 failed) 119 (72 failed) 47 shortlisted 15 selected 135 136 One negative effect of inadequate staffing and existing procedures was that the quality of a number of selected proposals was poor, for example without clear objectives, without a strong project team, or with a very limited innovative aspect. The projects selected as a result of the 1995 calls for tenders and for proposals display a significant improvement in quality, and expectations of good results are running high. The terms of reference of the 1993 and 1994 calls for proposals and invitations to tender were very, weak and lacked clarity. In particular, outputs (deliverables) and sustainability requirements were riot adequately covered. Current contracts have much improved in terms of specifying outputs, but these could still be more precise in terms of measurable indicators, eg by giving target number of attendees at seminars and conferences. Contractors should however be required, as much as possible, to respond specifically to each output and target identified in their contract in their final reports. 137 We understand that in certain instances proposals were selected for "political" reasons, ie to achieve some "fair" distribution of the funds available among Member States. More efficient mechanisms to achieve this objective should be designed. 138 As illustrated by the case study looking at the selection in 1995 of the contractor for the tourism and environment projects, the procedures now in place have improved significantly. Much improved procedures and forms have been introduced, including progress reporting and an interim evaluation by the Unit at the interim report phase. The evaluation procedure (as described in an internal Explanatory Note on Evaluation Procedure) is now well structured. 139 However, there is still a lack of clarity in a number of areas: a b whether and how the political and geographical mix is taken into account at an early stage the consultation of other Directorates General on the proposed projects prior to the final selection. 140 We were indeed concerned by the fact that the evaluation and screening of projects take place without any input from officials or experts external to DG XXIII prior to the final selection. Although we understand that this is common practice at the Commission, we take the view that this is inadequate. 141 Until 1995, measurable indicators of performance were not systematically defined at the outset (for example as part of the contractors' contract). Without objective measures, it is difficult both for the contractors and evaluators to measure progress against expectations. 28 Final Report • V2. 0 It EVALUATION OF THE COMMUNITY ACTION PLAN TO ASSIST TOURISM FINAL REPORT SECTION 4 - ADMINISTRATIVE AND OPERATIONAL ASPECTS 0 142 Financial management was poor during the first years of the implementation but has then improved. It still needs to be strengthened. Poor performance in the area of financial management was not only due to the mismanagement by senior members of the staff (who were dismissed in 1995), but also to inadequate systems (such as poor use of information technology), inadequate staff training, and insufficient staffing (only one grade B through most of the Plan implementation). 4. 3 - Recommendations Relations with External Bodies 143 The effectiveness of the Plan could be enhanced by: a b c d e f improving the partnership with the Member States. The Unit could make a number of propositions to the members of the Management Committee and put this topic on the agenda of a forthcoming meeting encouraging a more proactive role of the various Committees, with the support of experts, in identifying projects (innovative, trans-European, well funded, etc. ) improving consultation mechanisms with the industry, for example by setting up ad-hoc working groups j increasing the involvement of NTOs involving other Directorates General in the selection process before the final selection is made by DG XXIII involving independent and external experts in the selection process (one possibility is to have every Member State to nominate one or two experts who would be called in to review proposals submitted by potential contractors from other Member States). 144 145 146 147 More generally, there is a need to extend the level of participation of relevant stakeholders/partners (both private and public) to broaden ownership and commitment. The (Management, Advisory and Consultative) Committees could become the main vehicles for dissemination of the results of the activities funded or undertaken by the Unit. These Committees could also be used to help the Tourism Unit identify needs (eg possible pilot projects) and to ascertain that EU funding will not duplicate existing initiatives. Independent experts could fulfill a similar role. DG XXIII has been very active in the area of co-ordination within the Commission. The co-ordination with the other DGs is a very important role for the Unit that must be given even more emphasis. The Unit should seek to set the standard in this area. This role has several facets, such as bringing tourism within the orbit of other EU programmes, and informing the tourism (both private and public) stakeholders of the developments taking place at the Commission. 29 Final Report-V2. 0 &t> EVALUATION OF THE COMMUNITY ACTION PLAN TO ASSIST TOURISM FINAL REPORT SECTION 4 - ADMINISTRATIVE AND OPERATIONAL ASPECTS 0 Staffing 148 In the area of staffing, corrective actions should be considered: 149 150 151 a b c d to reduce staff turnover to improve the in-house technical tourism expertise, eg through secondment from NTOs, and improvements in training provision to provide much more training to the staff (at all grade levels) to review the information technology in place. Administrative Arrangements and Procedures In order to establish administrative standards, the Unit should review its administrative arrangements to specify a number of performance indicators and targets. For example: every external letter to be acknowledged (by fax, telephone, or letter) within five days and to receive a reply within eg 15 working days (if realistic). There is a need to review internal procedures to free up time to prepare, monitor, and follow up the projects. The staff are apparently using a disproportionate amount of time to essentially non productive paperwork. There is a need to strengthen the monitoring and evaluation of projects, and, more generally, to design and implement simple but efficient procedures, designed at the outset and not retrofitted. Also, more site visits should take place. Monitoring and evaluation procedures should be reviewed along the following lines, although we recognise that some of the elements are already being developed: a b c d e f to put in place simple procedures to monitor and evaluate the actions as they are implemented (on-going monitoring and evaluation) to set up ad-hoc working groups to prepare, steer and evaluate actions. These groups should include a small number of recognised experts in the field and representatives of the private and public partners directly concerned to commission an interim evaluation at the mid-point of the Plan to draw first lessons, take corrective actions and inform the preparation of future actions to achieve a much closer involvement of DG XXIII in monitoring (in a spirit of partnership) the contractors. This would require more visits to the contractors, attendance at key events (eg conferences, workshops) and systematic follow up longer term follow up of completed projects should take place, with on site visits or follow up by phone. Key aspects to cover include: sustainability, outcomes, commercialisation and dissemination of results the Unit should consider drawing from experience of other DGs and programmes, in particular in the area of monitoring and evaluation, consider adopting best practice from other Directorates General of the Commission. For example, DG I and DG VIII have standardised 30 Final Report - V2. 0 L' EVALUATION OF THE COMMUNITY ACTION PLAN TO ASSIST TOURISM FINAL REPORT SECTION 4 - ADMINISTRATIVE AND OPERATIONAL ASPECTS 4Là HIF their approach to project design using the so-called "Logical Framework" approach, fully documented by the Commission, and used for training not only EC staff but also their partners such as consultants. Use of this framework provides a standard format and checklist through which the project design, implementation, and evaluation cycle naturally falls g a copy of all monitoring and evaluation forms prepared during the course of an action should be filed centrally and analysed on a regular basis (say every six months). A short summary report drawing together the key lessons should be prepared and circulated within the DG and the other Directorates. 152 Actions should also be considered to ensure that an adequate dissemination/distribution of project results takes place. Options include: a b to earmark resources in the project budget at the preparation/selection stage (we understand that this has been done in 1995) in relevant cases, to make the last payment conditional on the preparation of a publishable output, in which case such a publication must be explicitly itemised as an output in the contract. 153 A rigorous and continuous pre-registration process for all consultants covering some basic information could have prevented some of the problems that have arisen from "phantom" companies, and have provided back up in ensuing litigation and other problems. One possibility is to operate a system inviting consultants to confirm regularly (eg annually) their interest to be on the list, and if they don't their name would be automatically deleted. 154 All projects with media outputs (disc, CD, video, others) should have those separately evaluated by an appropriate mix of technical and professional staff. Some of the existing outputs, integral to projects, were not (to the best of our knowledge) examined. 155 Some outputs are in non-standard formats. DG XXIII should specify output formats eg one copy of all videos in VHS format, discs in PC format unless otherwise stated. 156 Other corrective actions to be considered by the Unit are: a b c d to reduce the number of projects and studies funded in order to increase their impact and to limit administration to review the "call for proposal" procedure to reduce the administrative load and to maximise the effectiveness of the procedure to increase the proportion of meetings organised around a theme or a technical issue, and drastically reduce the number of formal meetings too often characterised by the passivity of the audience. to launch a Newsletter or similar (possibly including a copy on the Internet) to present the activities of the Tourism Unit. 157 Action should be taken to improve the internal information systems (such as project databases) so that more management information is generated to support staff members and to consider the systematic and generalised use of e-mail for example to improve communications within the DG and with the 31 Final Report -V2. 0 61 EVALUATION OF THE COMMUNITY ACTION PLAN TO ASSIST TOURISM FINAL REPORT SECTION 4 - ADMINISTRATIVE AND OPERATIONAL ASPECTS 4Lfc ^IIF 158 159 160 other DGs. The Unit should be given more autonomy in a number of strictly operational areas such as the management of the budget for missions. The Unit should be able to respond quickly and flexibly to the needs of project implementation (site visits, participation in workshops etc), within the strict limits of an agreed budget. This should of course not make the Unit less accountable or transparent, but simply more efficient. Miscellaneous Timing. The annual cycle of the Plan had a negative impact on the quality of the projects selected and on the their implementation. A longer timespan is certainly required (eg three years) and projects should take account of the special seasonal considerations of the industry. Co-financing issues. There are a number of issues related to co-financing that must be addressed. The maximum level of EU contribution of 60% should be reviewed and possibly made more flexible. Other issues are how to check that the contributions of other sources of financing are real and how to avoid funding projects that have already started or are even completed. 161 EC logo. The continuing use of the EC logo needs to be controlled: only good projects should be entitled to use the EC logo. Registration issues should be clarified and resolved. 32 Final Report -V2. 0 &3 EVALUATION OF THE ACTION PLAN TO ASSIST TOURISM FINAL REPORT SECTION 5 - ASSESSMENT BY CONTRACTORS g\k fllF SECTION 5 - ASSESSMENT BY CONTRACTORS 5. 1 - Methodology 162 In order to obtain an assessment by contractors of the main aspects of project (and study) implementation, we sent a short questionnaire to 77 contractors for the 1993 and 1994 projects. This questionnaire covers the following aspects: a b c d e project description achievement of objectives project implementation assessment of various aspects of project implementation lessons, recommendations, suggestions and best practice. 163 164 The proforma used is given in Appendix IV. Note that in this section, the term "project" covers both projects and studies funded under the Plan. Although we told the contractors that their individual responses will remain private and confidential to Price Waterhouse, and will only be aggregated prior to discussion with DG XXIII, we feel that there is a bias in the overall assessment of the project (Tables 5. 1 and 5. 2 below) because of the self- assessment nature of the questions. To address this issue, we treated certain responses with caution. However, we donl think that this bias has a significant influence on the other aspects of the questionnaire such as the assessment of the various aspects of project implementation. 5. 2 - Analysis 165 We sent 77 questionnaires, and received 47 responses, thus achieving a response rate of 61 %. The number of valid responses varied between questions, but there were typically between 42 and 36 valid responses. Degree of Achievement of Objectives 166 We asked the contractors to rate the degree of achievement of the project objectives. More than half the contractors considered that they have been very successful in achieving the project objectives, while one third of them replied that they have been fairly successful. In total 86% of the respondents felt that they have been successful in achieving the main objectives of their action. There is no marked difference between 1993 and 1994 projects. Table 5. 1 below summarises the results. 33 Final Report -V2. 0 •it EVALUATION OF THE ACTION PLAN TO ASSIST TOURISM FINAL REPORT SECTION 5 - ASSESSMENT BY CONTRACTORS 0 Table 5. 1 - Degree of Achievement of Objectives Not at all successful Not very successful Fairly successful Very successful Mixed Not relevant or no reply 1322 Number of valid responses: 19 1324 Number of valid responses: 23 Total Number of valid responses: 42 0 0 0 0 2 7 8 9 12 2 1 1 0 2 (5%) 15 (36%) 21 (50%) 3 (7%) 1 (2%) Additionality: What Would Have Happened in the Absence of the Plan's Funding 167 To obtain an assessment of the Plan's additionality, we asked the contractors what would have happened in the absence of EU funding. The majority of respondents (58%) consider that EC funding has been additional ie the project would not have taken place in the absence of the Plan, while another 36% thought that the project would have taken place later or on a smaller scale. Project Implementation 168 We also asked the contractors to assess various aspects of the project implementation, and to give an overall assessment of the project's implementation. Tables 5. 2 and 5. 3 below give the scores obtained. Table 5. 2 - Overall Assessment of Pro ect Implement ation Poor or very poor Average Good Very good Do not know Not relevant or no reply 1322 Number of valid responses: 19 1994 Number of valid responses: 23 2 (10%) 4 (21%) 7 (37%) 6 (32%) 0 0 15 (65%) 7 (31%) Total number of valid responses: 42 2 (5%) 4 (10%) 22 (52%) 13 (31%) 0 1 (4%) 1 (2%) 0 0 0 169 Table 5. 2 indicates that 83% of the respondents think that the project implementation has been good or very good. This positive assessment is not surprising as the contractors are in fact to a large extent 34 Final Report-V2. 0 EVALUATION OF THE ACTION PLAN TO ASSIST TOURISM FINAL REPORT SECTION 5 - ASSESSMENT BY CONTRACTORS 0 assessing the quality of their own work. When asked to assess the different components of project implementation, the contractors' answers depict a much less rosy picture of the situation, as shown by Table 5. 3. Table 5. 3 - Assessment of Project Implementation* Indicators No of contractors "dissatisfied" ** (No of responses) As a % of total No of valid responses A - Planning & preparation A1 - Clarity of objectives A2 - Quality of project design A3 - Effectiveness of partner(s)' participation in preparation and planning B - Dissemination B1 - Follow up of project after completion B2 - Dissemination of main results C - Management by Tourism Unit staff C1 - Financial & administrative monitoring by Tourism Unit staff C2 - Overall efficiency of Tourism Unit staff C3 - Responsiveness of Tourism Unit staff D - Co-ordination D1 - Quality of dialogue between Tourism Unit staff and other partners D2 - Coordination with other EU departments and non EU agencies 6 5 3 16 10 15 13 9 12 9 11 7 9 8 14% 12% 7% 38% 24% 36% 31 % 21 % 29% 21 % 26% 17% 21% 19% * Multiple response is possible ** Measured by total of responses in three categories: average, poor, very poor. 170 Table 5. 3 above indicates the a number of shortcomings, weaknesses, and areas for improvement. The main shortcomings/weaknesses of project implementation are perceived to be in the following areas: 35 Final Report-V2. 0 C^ EVALUATION OF THE ACTION PUN TO ASSIST TOURISM FINAL REPORT SECTION 5 - ASSESSMENT BY CONTRACTORS 0 (1) Effectiveness of partner(s)' participation in the preparation and planning of the project: 38% of contractors had difficulties in this area (2) Dissemination and follow up: these areas are clearly perceived as the weakest aspect of project implementation: about one third of contractors take the view that the follow up after completion and the dissemination of the results have not been satisfactory. (3) Management by the Tourism Unit staff: more than one quarter of respondents were not satisfied by the quality of financial and administrative monitoring and by the responsiveness of the Unit's staff. 5. 3 - Lessons. Recommendations. Suggestions and Best Practice 171 A number of lessons/recommendations/suggestions were also made by the contractors who responded to our questionnaire. Monitoring and Evaluation 172 173 174 175 176 It is essential to have more communication between the Unit, the project coordinator and the other project stakeholders. In particular, during the preparation/planning phase of the project, more co- ooperation between the Unit and the contractors is needed. There is s need for more exchange of information on (past, current) projects in similar fields of intervention. DG XXIII should supervise much more closely the projects funded. There is also a demand for more contacts (such as site visits) by Unit officials. Indicators of achievement and targets should be specified (and quantified as much as possible) at the beginning of the project. Procedures followed by DG XXIII should be streamlined, simplified. In particular, the decision-making process must be made much more efficient. 177 Steps must be taken to improve the responsiveness of DG XXIII staff. Follow Up and Dissemination of Results 178 Suggestions were made to improve the dissemination of results: a b c to circulate copies of the main deliverables to all the contractors to earmark in the project budget an amount for dissemination and distribution of key outputs to include in the budget, where relevant, an element for marketing the outcome of the project. 179 More consideration should be given to the sustainability of actions. 36 Final Report-V2. 0 EVALUATION OF THE ACTION PLAN TO ASSIST TOURISM APPENDIX I - DESCRIPTIVE STATISTICS OF ACTIONS gjjk f HF This appendix contains the database of the actions implemented under the Plan. Key for Classification of Actions bv Type 1 = Improving knowledge (includes statistics, studies and consultation) 2 = Staggering holidays 31 = Transnational 32 = Technology 33 = PECO (Eastern and Central European Countries) 4 = Tourists as consumers 5 = Cultural tourism 6 = Tourism and environment 7 = Rural tourism 8 = Social tourism 9 = Youth tourism 10 = Training 11 = Promotion Final Report -V2. 0 £>% APPENDIX I EVALUATION OF THE COMMUNITY ACTION PLAN TO ASSIST TOURISM TOURISM AC" •ION PLAN - PROJECTS & STUDIES 1993-1995 Contract Number Main C o n t r a c t or Initial Budget (ECU} Type ùf Action Delvico Bates GIAT Forum Dello Svilluppo FIPE AA-Automobile Association International Interactive Multimedia GSI Transport Tourism Regione Toscana GEIST Club des Eurométropoles TIRS Chambre Régionale de Commerce Rhône-Alpes AGTE CISET Province of North-Holland Instituto Geogrâfico Nacional (IGN/CNIG) Wexford County Council Regione Emilia-Romagna Conselleria de Cultura, Educacio, i esports EETAA - Hellenic Agency Preseli Pembrokeshire Universitât Oldenburg Interface Naturfreunde Landesberband Saarland R E D- Ruralité Environnement Dévelopement Royal National Institute for the Blind Landkreis Eisleben Ministero del Toujismo e dello Spettacolo GruppoClas System TV CEPFAR F y n T o ur Eurogites ETAG L & R Leisure PLC AIT/FIA Contour Mobility International Bates & Wacker Directors General de Turismo Mobility International Associano Nacional de Jovens E T E N1 TURINFORM Frosti du Limousin DGF - Deutsche Gesellschaft fur Freizeit EFCO TV5 ITE ETEN 2 FEG CETOS Agrupacion de Asesores Mouvement HTL Conseil 1993 93 180001 93 530001 93 540004 93 540007 93 540009 93 540010 93 540012 93 540013 93 540014 93 540015 93 540017 93 540018 93 540019 93 540020 93 540021 93 540022 93 540023 93 540024 93 540027 93 540028 93 540029 93 540030 93 540031 93 540032 93 540035 93 540036 93 540037 93 540039 93 540040 93 540041 93 540043 93 540044 93 540045 93 540046 93 540047 93 540048 93 540054 93 540055 93 540056 93 540058 93 540059 93 540060 93 540062 93 540063 93 540064 93 540066 93 540068 93 540069 93 540071 93 540072 93 540076 93 540077 93 540079 93 540080 93 540081 No projects 93: Total 1993: Mean value 93: Maximum 93: Minimum 93: 11 1 1 33 32 32 32 32 32 31 32 31 32 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 3 31 5 31 1 33 1 1 8 1 1 8 9 10 32 33 1 7 5 3 10 12 8 1 1 6 55 999628 697800 23250 50000i 99000j 1200001 2132001 200000! 244635 j 130000; 102300 ; 96000 i 114000! 61980: 70000j 606061 63600; 460751 900001 136445i 29275 ! 61000: 36000: 46000' 49659, 47586 j 102000; 48000 105000. 160000 15000; 9840 ; 34278 10200 220000 11400 49000 60000. 42100 150000 50000 10189 100800 154890 32000 20000 120000 198000 200000; 95750 20000 65000 199620 110258 381550, I j i I ; i ! i ! ! i i i ! j i 55 i 6663 < "000 ECU 121144 999628 ECU ECU 9840 i ECU 61 APPENDIX EVALUATION OF THE COMMUNITY ACTION PLAN TO ASSIST TOURISM TOURISM AC ION PLAN - PROJECTS & STUDIES 1993-1995 Contract Number Main Contractor Initial Budget LECU) Typ» Of Action 1994 94 540011 94 540018 94 540019 94 540023 94 540024 94 540025 94 540026 94 540027 94 540028 94 540029 94 540031 94 540032 94 540033 94 540034 94 540035 94 540036 94 540037 94 540039 94 540043 94 540044 94 540045 94 540046 94 540047 94 540048 94 540049 94 540050 94 540051 94 540053 94 540054 94 540064 94 540065 94 540067 94 540075 94 540077 94 900107 94 900118 94 900120 Greek National Tourist Organisation (conference) Earthwatch Europe World Systems Europe Gemeinnutziger Verein zQr Forderung der Region Sandl unteres Muhlviertl Naturfreunde Saarland Projekt Sanfter Tourismus SYGNA A/S FOTVE/FECTO - Fédération des Offices du Tourisme des Villes Européennes Royal National Institute for the Blind (RNIB) BC Brandenburg-Consult GmbH Universitët Oldenburg/FB3 Forschungsstelle Niedersachsische - Nausa Baltic Tourism Cooperation Il Quartettone Foden Fyn Tour Arthur Andersen SA Horwath Axe Consultants-Tourisme Hôtellerie Loisirs Movimento Consumatori CTS Centra Turistico Studentesco e Giovanile Conference Etalement des Vacances Gruppo Clas Greta du Golfe Lancashire Enterprises Universidad de Deusto-Bilbao - Inst. de Estudios de Ocio Fondo Iberico para la Conservaciôn de la Naturaleza ITD-lnstitute of Tourism and Services economics Hjoerring Business College EMI-Effective Management International Consorzio Civita (ARIFOC) Association Réunionaise Interprofessionelle pour l'Insertion et la Formation Continue Centre Méditérannéen d'Environnement d'Athènes ETOA - European Tour Operators Association AFI Alpenforschunginstitut Ministère de rEquipement, Transports et Tourisme Atlas (Tilburg University) Concorde Travel Rita de Landtsheer European Travel Commission (ETC-CET) Agence Européenne pour la Culture - AEC (UNESCO) 43200 50000 249233 59694 94995 79175 195000 71587 99610 183300 129000 64031 49800 392600 333000 160610 167615 50963 167000 146830 62500 120000 127000 115000 195200 78995 176872 134416 102000 30187 104040 63553 50000 11100 71000 50000 51000 j 1 6 1 5 5 5 5 5 5 5 5 5 5 1 1 4 9 2 5 10 10 10 10 10 10 10 10 10 10 11 1 1 33 33 2 9 5 No projects 94: Total 1994: Mean value 94: Maximum 94: Minimum 94: 37 4330 117030 392600 11100 •000 ECU j ECU ECU ECU ! 37 | i 1 i ! ^ O APPENDIX I EVALUATION OF THE COMMUNITY ACTION PLAN TO ASSIST TOURISM TOURISM AC "ION PLAN - PROJECTS & STUDIES 1993-1995 Contract Number Matrt Contractor 1995 y J Initial Budget (EÔUI Type of Action 9S530001 95530002 95530003 95530004 95530005 95530006 95530008 95530009 95530010 95530011 95530012 95530013 95530014 95530015 95530016 95530017 95530018 95530019 95540013 95540014 95540015 95540016 95540017 95540018 95540019 95540027 95540028 95540030 95540031 95540032 95540033 95540034 95540035 95540037 95540038 95540039 95540040 95540041 95540042 95540043 95540044 95540045 95540046 95540047 95540048 95540049 95540050 95540054 95540055 95540056 95540057 95540058 95540059 9554006 95540060 95540061 95540062 95540063 95540065 95540066 95540067 95540068 95540069 95540070 95540072 95540075 95540076 95540079 95540080 No projects 95: Total 1995: Mean value 95: Maximum 95: Minimum 95. Osterreichisch Werbung (A) Statistics Finland (FIN) Ministère Equipement, Transports et Tourisme (F) The Stat. Bureau of Iceland (IS) Inst. Nacional de Estadistica (E) Inst. National de Statistiques (B) Stat. Norway (N) Danmarks Statistik (DK) Stat. Sweden (S) Stat. Portugal (P) Central Statistical Office - CSO (UK) Istituto Naz. di Stat. (I) Centraal Bureau Voor de Stat. (NL) ETI (L) Central Statistical Office - CSO (IRL) Stat. Bundesamt {D)^ Gruppo Clas (I) Nat. Stat. Service of Greece (GR) AFI Alpenforschunginstitut Gugg & Partner Delvico Bates Ministry of Communications Nortra Arthur Andersen Office Natinal du Tourisme Grand Prix - 2e meeting of jury Grand Prix - 2e meeting of jury HTL Conseil Osterr. Stat. Zent. (A) Inst. Nat. de Stat. (B) Inst. Nat. de Stat. (D) InsLNat. de Stat. (DK) Min. Eq. Transp. & Tourism Inst. Nat. de Stat. (GR) Inst. Nat. de Stat. (IRL) InsLNat. de Stat. (IS) Inst. Nat, de Stat. (I) InsLNat. de Stat. (L) - ETI InsLNat. de Stat. (NL) InsLNat. de Stat. (UK) Swedish Tourist Authority InsLNat. de Stat. (P) InsLNat. de Stat. (N) Min. Comercio y Turismo - Secretaria General de Turismo (E) InsLNat. de Stat. (IRL) Gruppo Clas Conférence "Livre Vert" du 12/12 - SCIC CTA Economie & Export Analysis Gemeente Budel OING Lumières de la Méditerranée Kienbaum Dev. Services ASTER WTTC InsLNat. de Stat. (FIN) AENA - Aeropuerto de Palma de Mallorca Nord Info GmbH MEWCAT - Milos European Women's Cooperation for the Advancement in Tourism St Andrews Tourism Management Group The Tourism Company Mobility International University of Thessaly Azienda di Promozione Turistica del Mantovano ARGE Sanfte Mobilitat NIT Messe Berlin (ITB) BIT, Milano Watch TV Près. Consiqlio Minis. Turismo nstituto Municipal de Formacion y Empleo (Ayuntamiento de Granada) j 40000 40000 120000 40000 120000 70000 40000 70000 40000 70000 120000 120000 70000 20000 70000 120000 259700 70000 27900 205000 499628 7500 9500 66742 7500 22100 12000 146450 130000 130000 230000 130000 230000 130000 130000 ! 70000 230000 40000| 1300001 230000 130000 130000 70000 230000 130000 500240 84478 238792 38630 ! 137670 ! 156486| 1893901 6310001 1300001 91380 60700 178300 195757 127800 30000 152971 98552 190000 98749 99840 66121 40000 60000 163159 69 8764 127015 631000 7500 •000 ECU ECU ECU ECU 1 i ; 1; ii ii 1 • 1, 1 ; 1 ; 1 , i; ij ij 1 1 i i! ij 1 i| 11i ii ij ii 6i '«I 6 6| 1i 1 1 1 : 11 i! n ii 1 , 1 ! 1 ; 1 ; 1! 1! ij il 11 1 i 1 ; 3 2; 32] 32; 3 2| 32; 6 1 6| 32; 6i 61 6i 8i 6! el •1 1 1. 5i i| il 6i , 1 ! ! i EVALUATION OF THE ACTION PLAN TO ASSIST TOURISM APPENDIX II - CHECKLIST FOR INTERVIEWS OF KEY INFORMANTS 41a *MW Final Report • V2. 0 it EVALUATION OF THE TOURISM ACTION PLAN PROGRAMME - CHECKLIST 1 - Exposure to Tourism Action Plan PROMPTS: 1. 1- How familiar are you with the Tourism Action Plan programme? Please describe your exposure to Tourism Action Plan. 1. 2 - Please describe the objectives, the role of Tourism Action Plan as you perceive them? Did you perceive any significant change, evolution, trend? 1. 3 - If relevant, please describe your current involvement in area related to tourism policy. 2 - Interface with Tourism Action Plan (Co-Ordination. Complementarity. Overlap. Etc) PROMPTS: 2. 1 - Please describe the objectives of your programme and compare them with Tourism Action Plan in terms of: - objectives, priorities - instruments (means, techniques used) - processes. 2. 2 - In what manner and to what extent does Tourism Action Plan complement, duplicate, overlap or work at cross-purposes with other programmes? 2. 3 - Please assess the quality of the co-ordination between Tourism Action Plan activities and the activities undertaken by your programme. 3 - Assessment of the Rationale of Tourism Action Plan: Does the Programme Make Sense? PROMPTS: 3. 1 - Are the activities and outputs of Tourism Action Plan consistent with its mandate and realistically linked to the attainment of objectives? 3. 2- To what extent are the objectives and mandate of Tourism Action Plan relevant (in the past, present, future)? 3. 3 - What should Tourism Action Plan's role and activities look like in the future (exercise of wishful thinking)? 4 - Assessment of Impacts and Effects: What Has Happened As A Result of the Programme? PROMPTS : 4. 1 - What impacts and effects, both intended and unintended, resulted from carrying out Tourism Action Plan? 4. 2 - If relevant: do the (unintended ) negative effects outweigh the benefits of Tourism Action Plan? ?3 4. 3 - Give examples, illustrations of success stories, also failures. 5 - Achievement of Programme Objectives: Has the Programme Achieved What Was Expected? PROMPTS : 5. 1 - In what manner and to what extent were the appropriate objectives achieved as a result of Tourism Action Plan? 5. 2 - What are the reasons for failures to meet objectives and can these be remedied? See above 4. 3 - Give examples, illustrations of success stories, also failures. 6 - Alternatives: Are There Better Ways of Achieving the Results/Obiectives? PROMPTS : 6. 1 - Are there most cost-effective alternative programmes which might equally achieve the objectives? 6. 2 - Are there more cost-effective ways of delivering the existing programme? 7 - Lessons. Recommendations. Suggestions. Best Practices 7. 1 - Any key lessons/recommendations/suggestions to convey to Tourism Action Plan staff? 7. 2 - Any best practices from other programmes to convey to Tourism Action Plan staff? 8 - Overall Assessment 8. 1 - Main strengths of the Tourism Action Plan? 8. 2 - Main weaknesses/shortcomings of the TAP? 8. 3 - What should be done differently? ^ EVALUATION OF THE ACTION PLAN TO ASSIST TOURISM APPENDIX III - EVALUATION FICHE FOR ASSESSMENT BY PW EXPERTS 4» Final Report • V2. 0 15 Price Waterhouse - Evaluation of the TOURISM ACTION PLAN Proforma for Assessment by Expert Consultant Status of Assessment U Discussed with Project Official BEFORE completion LI Discussed with Project Offical AFTER completion U To be completed U Pending issue(s) Project Title Contractantfs) Contract Number Project Official Documentation Used for Evaluation U Call for Tender LI Proposal by Consultant LI Contract Li I Proposal Assessments by DGXXIII LI In-House Interim Evaluation LI In-House Final Evaluation LI Final Report LI Interim Report (only if final report not submitted) L) Other Evidence If other, please specify: 7é Score (1-5) Comments 1 - Quality of Goal-Setting 1. 1- Clarity of objectives in call for tender 1. 2 - Quality of goal-setting in proposal 1. 3 - Degree of innovation in the European context 1. 4 - Is the project transferable? 1. 5- Overall quality of project design Subtotal I I 2 - Outputs of the Proiects - Incidence of Effects | | Score (1-5) Comments , " " ' " 2. 1 - Degree of achievement of objectives in terms of quantity 2. 2 - Degree of achievement of objectives in terms of quality Subtotal 2. 3 - What if any are the weaknesses of the project? 2. 4 - What if any are the positive outcomes of the project? X X i | _i i j j _ _ _ _ _ _ _ —i I J ^ ^ I i i j j ! 1 j ! | 77 3 - Rnal Report/Deliverable & Follow Up Score (1-5) Comments 3. 1 - Overall quality of final report (presentation, format) 3. 2 - Overall quality of final report/deliverable (contents) 3. 3 - Assessment of final deliverable in terms of distribution/dissemination( ie is the final deliverable of immediate use?) Subtotal n 4 - Compliance with Article 4 of the Council Decision (13 July 1fl921 Score (1-5) Comments 4. 1 - Cost-effectiveness 4. 2 - Impact on the Community tourist industry 4. 3 - Facilitate development of SMEs 4. 4 - Improve quality of tourist services 4. 5 - Encourage competition within EU 4. 6 - Increase competitiveness of EC tourist services j 4. 7 - Conducive to preserving and protecting the environment 4. 8 - Conducive to preserving and protecting the cultural heritage i 4. 9 - Conducive to preserving and protecting integrity of local | populations i | 4. 10 - Conducive to improving the provision of information and to the protection of tourists Subtotal. *T 5 - Overall Assessment of the Project Score (1-5) Comments Overall Assessment of the Project 6 - Other Comments 7 - Pending Issues. Questions. Queries fo EVALUATION OF THE ACTION PLAN TO ASSIST TOURISM APPENDIX IV - EVALUATION FICHE FOR ASSESSMENT BY CONTRACTORS 0 Final Report - V2. 0 if Price Waterhouse - Evaluation of the TOURISM ACTION PLAN Proforma for Assessment by Project Contractors Thank you very much for agreeing to help us in this important evaluation commissioned by the European Commission (DGXXIII). Please note that individual responses will remain strictly confidential to Price Waterhouse and no information about your assessment of individual projects will be provided to the European Commission. We will aggregate and analyse the results prior to reporting to the European Commission. If you consider that a question does not apply to your project, please say so. This document contains two sections: Section A - Project Description and Section B - Assessment by Contractor. Section B is made of three subsections: Achievement of Objectives; Project Implementation; Lessons, Recommendations, Suggestions, Best Practice. A - PROJECT DESCRIPTION A1 - Project Title A2 - Year started: A3 - Project Description (full description, maximum 10 lines) A4 - Principle contractor names, not acronyms): If consortium, please name ajj participating organisations (give full A5 - If project co-financed, please name other bodies involved in co-financing (give full names, not acronyms): A6 • Please name other stakeholders and project partners (ie excluding TOURISM ACTION PLAN and Other CO-financing bodies) of t he project - Stakeholders are groups and individuals that have an interest in the outcome of the project, such as: international agencies, national administrations, research institutes, consulting firms, project implementing agencies, andintended beneficiaries. u A7 - State of advancement: I—I Not started I I Abandoned I I I I I Still ongoing but NOT delayed I Still ongoing but delayed I Completed and final report submitted I Completed but no final report submitted 1_J Other A8 - If project not started, abandoned or delayed, give reasons: A9 - If final report submitted, please give date of submission of the final report: MONTH: YEAR: A10 - If project completed but final report not submitted, give reason and expected date of submission: A11 - Type of Project: LJ Pilot Project I I Research Study I I Conference I I Publication U Other A12 - If other, please specify: A13 - Countries - Please give specific countries where the project is taking place : A14 - Please give other sources of funds (ie other than DGXXIII-TOURISM ACTION PLAN): A15 - COMMENTS - Please give any further comments on project description: 1* B - ASSESSMENT BY CONTRACTOR B1 - Achievement of Objectives (including impact) B1. 1 - If any, please specify target group(s)- Target Groups are the intended beneficiaries of the project, the groups that should benefit from it. If no target group(s), please explain why. B1. 2 - Please specify the main initial objectives of the project, starting with the most important one: Objective 1 (most important): Objective 2: Objective 3: Other objectives, if any: B1. 3 - Please rate the degree of attainment of these objectives - Tick only one box Attainment of Project Objectives Not at all successful Not very successful Fairiy successful Very successful Do not know Not relevant Objective 1 (as defined above) Objective 2 (as defined above) Objective 3 (as defined above) Objective 4 (as defined above) Objective 5 (as defined above) Overall Assessment B1. 4 - Main Indicators of Achievement Please give the main indicators of achievement specified for the project, which may be either quantitative (ie expressed in numbers) or qualitative (ie expressed in words): Indicator 1: Indicator 2: Indicator 3: Other indicators, if any: I l I j i i i Vr B 1. 5 - D e g r ee of A c h i e v e m e nt of P e r f o r m a n ce Indicators Please indicate t he e x t e nt to w h i ch t he p e r f o r m a n ce indicators have been a t t a i n e d. 1 Indicator of Achievement Target(s) initially planned Actual outcome(s) at completion 1 1 ! | Reasons for differences from j target — i | j i I | j l ! i j i Indicator 1 (as defined above) Indicator 2 (as defined above) Indicator 3 (as defined above) Other: I I Benefits a nd N e g a t i ve Effects Arising f r om t he Project B 1. 6 - Please give t he m a in benefits arising f r om t he project: Benefit 1 : Benefit 2: Benefit 3: / Other benefits. If any: B 1. 7 - If a n y, please give t he m a in n e g a t i ve e f f e c ts arising f r om t he project: Negative effect 1 : Negative effect 2: Negative effect 3: Other negative effects, if any: B 1. 8 - W h at w o u ld have h a p p e n ed in t he a b s e n ce of T O U R I SM A C T I ON PLAN f u n d i n g? I I The project would NOT have taken place I I I The project would have taken place but LATER I The project would have taken place but AT A SMALLER SCALE I I Don't know/no opinion LI Other If other, please specify: tf B1. 9 - COMMENTS - Please give any further comments on achievement of objectives: B2 - Project Implementation Changes in Project Design B2. 1 - Were there significant changes in project design such as changes in scope, objectives, target groups, approach? YES NO B2. 2 - If yes, please specify: B2. 3 - Please give the reasons for these changes in project design? Delays B2. 4 - Was the start of the project delayed? YES NO B2. 5 - If yes: By how many months? MONTHS B2. 6 - If yes: Reasons for these delays? B2. 7 - Was the project completed on time? YES NO B2. 8 - If no: By how many months was the project delayed? MONTHS B2. 9 - If no: Reasons for these delays? Cost Overruns B2. 10 - Was the project completed within the initial budget? YES NO B2. 11 - If no: Give size of cost overrun (final budget divided by initial budget): ECU B2. 12 - If no: Reasons for cost overruns?. 4 B2. 13 - An Index for Assessing Project Implementation Please help us to assess the implementation of the project. Please tick one box only per r o w. Indicators Poor Very poor Average Good Very good Don't know Not relevant A - Planning & Preparation of Project i A1 - Clarity of objectives A2 - Quality of project design A3 - Effectiveness of the partner(s)' participation in preparation & planning B • Dissemination B1 - Follow up of project after completion B2 - Dissemination of main results C - Management by TOURISM ACTION PLAN staff C1 - Financial & administrative monitoring by TOURISM ACTION PLAN staff C2 - Overall efficiency of TOURISM ACTION PLAN staff C3 - Responsiveness of TOURISM ACTION PLAN staff D • Co-ordination D1 - Quality of dialogue between TOURISM ACTION PLAN staff and other partners D2 - Co-ordination with other EU departments and non EU agencies Your overall assessment of the project implementation 1 ! i i j 1 1 j J j 1 1 i 1 I i i i j i i I Sv B2. 14 - COMMENTS - Please give any additional comments on project implementation: B3 - LESSONS. RECOMMENDATIONS. SUGGESTIONS. BEST PRACTICE B3. 1 - Any key lessons/recommendations/suggestions to convey to the TOURISM ACTION PLAN staff? B3. 2 - Any best practice to convey to the TOURISM ACTION PLAN staff? Have you any comments to make about this proforma? If yes, Price Waterhouse would be very grateful if you could indicate how to improve it: J Completed by: Name: Position: Tel number: Fax Number: Date: Thank you very much for your co-operation Please fax the completed document to: Price Waterhouse Project TOURISM ACTION PLAN - EC SERVICES c/o Luc MOENS Fax: + 44 171 9 39 3 4 74 Tel: + 44 171 9 39 3 3 31 EVALUATION OF THE ACTION PLAN TO ASSIST TOURISM APPENDIX V - CHECKLIST FOR CASE STUDIES 0 Final Report - V2. 0 it CASE STUDIES ASSESSMENT BY KEY INFORMANTS NAME OF THE CASE STUDY: • ASSESSMENT OF PROJECT BY KEY INFORMANTS - CHECKLIST Foreword A very happy New Year and thank you for your co-operation. You will find below a list of prompts/questions for each issue we need to address to complete this case study. The objective of the interview we wish to set up with you is M to discuss in detail every item in this checklist. We wish to focus on the areas where your contribution is most valuable and relevant. If your organisation was the main contractor for this project, you will also receive from Price Waterhouse a short questionnaire designed to record descriptive information and to collect the contractor's views on the project implementation. Please return it completed as soon as possible as we have to report to the Commission by the end of January 1996. Introduction/Interface with the Case Study A. B. Involvement in the project: since/until when? Role? Exposure to the Tourism Action Plan: How familiar are you with the Tourism Action Plan programme? Preparation & Rationale 1. 2. 3. 4. 5. Origin of the project: who initiated the project? Brief history/background. Rationale: principles underlying the project? Justification? Which problems is it addressing? Main relevant characteristics of the environment in which project is taking place? Describe preparation and planning of the project. Briefly describe how the consultant contracts have been awarded and the reasons for the selection. Summarise key dates and milestones (including future ones if project not completed), and main (realised or expected) outputs/events (reports, key meetings, seminars/workshops, other key events). Stakeholders & Key Informants 6. Please name all important stakeholders and describe their role/contribution. Stakeholders are groups and individuals that have an interest in the outcome of the project such as: other EU and non-EU international agencies, national administrations, research institutes, consulting firms, project implementing agencies, and intended beneficiaries. Objectives 7. If any, please specify target group(s)- Target Groups are the intended beneficiaries of the project, the groups that should benefit from it. If no target group(s), please explain why. 8. Main initial objectives of the project, starting with the most important one: t\ù CASE STUDIES ASSESSMENT BY KEY INFORMANTS NAME OF THE CASE STUDY: • Objective 1 (most important): Objective 2: Objective 3: Other objectives, if any: 9. Please rate the degree of attainment of these objectives - Tick only one box per row in the table below. Not at all successful Not very successful Fairly successful Very successful Do not Know Not Relevant • Attainment of Project Objectives Objective 1 (as defined above) Objective 2 (as defined above) Objective 3 (as defined above) Objective 4 (as defined above) Objective 5 (as defined above) Overall Assessment Main Indicators of Achievement 10. Please give the main indicators of achievement specified for the project, which may be either quantitative (ie expressed in numbers) or qualitative (ie expressed in words): Indicator 1 : Indicator 2: Indicator 3: Other indicators, if any: CASE STUDIES ASSESSMENT BY KEY INFORMANTS NAME OF THE CASE STUDY: O 11. Please indicate in the table below the extent to which the performance indicators defined above have been attained. Indicator of Achievement Target (initially planned) Actual (at completion) Reasons for differences from target Indicator 1 (as defined above) Indicator 2 (as defined above) Indicator 3 (as defined above) Other: Interface of Tourism Action Plan with other programmes/policies (Co-Ordination. Complementarity. Overlap. et£l 12. 13. In what manner and to what extent does Tourism Action Plan complement, duplicate, overlap or work at cross-purposes with other EU and non-EU programmes? Please assess the quality of the co-ordination between Tourism Action Plan activities and the activities undertaken by other EU and non-EU programmes. Impacts and Effects: What Has Happened As A Result ot the Programme? 14. What positive impacts and effects, both intended and unintended, resulted from carrying out the project? 15. What negative impacts and effects resulted from carrying out the project? Project Implementation Monitoring & Evaluation 16. Describe procedures/arrangements to monitor the project implementation 17. Describe procedures/arrangements to evaluate the project during and after implementation n CASE STUDIES ASSESSMENT BY KEY INFORMANTS NAME OF THE CASE STUDY: Changes in Project Design • 18. Were there significant changes in project design? If yes, please specify and give the reasons for these changes in project design? Delays 19. Was the start of the project delayed? If yes: By how many months? If yes: Reasons for these delays? 20. Was the project completed on time? If no: By how many months was the project delayed? If no: Reasons for these delays? Cost Overruns 21. Was the project completed within the initial budget? If no: Give size of cost overrun (final budget divided by initial budge) If no: Reasons for cost overruns? Dissemination & Follow Up 22. 23. Has the project results been disseminated? What was the follow up, and who did it? Has the project been replicated? (If relevant) For how long are the operational cost (partly) covered by Tourism Action Plan? What will happen next? Lessons. Recommendations. Suggestions. Best Practices 24. What are the reasons for failures to meet objectives and canthese be remedied? 25. Any key lessons/recommendations/suggestions for future projects? 26. Which elements can be replicated, under which conditions? OVERALL ASSESSMENT 27. Main strengths of the project? 28. Main weaknesses/shortcomings of the project? 29. Contribution/value added by Tourism Action Plan? 30. What would have happened in the absence of the Tourism Action Plan funding? Would the project exist? If yes, would it be different? 31. What would you do differently? 31 EVALUATION OF THE ACTION PLAN TO ASSIST TOURISM APPENDIX VI - COUNCIL DECISION OF 13 JULY 1992 0 Final Report - V2. 0 • ft No L 231726 Official Journal of the European Communities 13. 8. 92 COUNCIL DECISION. of 13 July 1992 on a Community action plan to assist tourism (92/421/EEC) THE COUNCIL OF THE EUROPEAN COMMUNITIES, Having regard to the Treaty establishing the European in particular Article 235 Economic Community, and thereof Having regard to the proposal from the Commission ('), Whereas, tourism can make an effective contribution to achieving economic and social cohesion in the Community and whereas the Community a harmonious development of economic activity, continuous and balanced expansion, a higher standard of living and closer relations between the States which it links; it can promote in Having regard Parliament (2),. to the opinion of the European Whereas the measures tô be implemented under the action plan must comply with certain criteria in particular the need to comply with the subsidiarity principle; Having regard to the opinion of the Economic and Social Committee (3), Whereas tourism occupies- an important place in the economy of the Member States, with tourist activities. representing a large potential source of employment; Whereas tourism in the Community will have to show consideration for local populations and for the natural and cultural environment, in order to improve the quality of services offered; Whereas tourism allows people of all kinds to gain a better knowledge of Europe's cultural roots and of the cultures and ways of life in the Member States, thus making a' contribution to the progress of the idea of 'European citizenship'; j Whereas the results of the European. Year of Tourism should be taken into account; in view of Whereas, the above, Community action regarding tourism should take the form of a strengthening of the' horizontal approach to tourism in Community and national policies, and of the implementation of specific measures, and whereas that apporach should also include coordination of the measures, undertaken by Commission departments which affect tourism; whereas certain Communie/ policies, in particular transport policy, have a major impact on tourism in the various regions of the Community; Whereas the Community can contribute to improving the quality and competitiveness of the Community's tourism services on offer, by encouraging a joint apporach to the medium-term problems tourism; by promoting the development of the tourist industry and thç diversification of tourist activity and the development of transnational measures, and by developing the promotion of European tourism on- the main ' markets of third countries; facing European Whereas free competition should be preserved in the sector, both for the benefit of consumers, and in order to promote small and medium-sized enterprises (SMEs); Whereas it is necessary to encourage not only better integration of tourism into the various Community policies but also close cooperation between all public and private bodies in the sector, including- représentâmes of tourist regions, and whereas the implementation at Community level of à number of specific measures, complementary to those taken at national level, is the best way of achieving such cooperation, while avoiding any distortion of competition which may be caused; Whereas statistics on tourism, should be developed and forward analysis of new types of tourism carried out; Whereas a, plan of three years' duration is called for; («) OJ No C 120, 12. 5. 1992. p. I J. (*} OJ NoC/57. 16. J. 1992, p, 2J5. (J) Oj. Nu C 49. 24. 2, 1992. (». 4J_ Whereas an amount of ECU 18 million is deemed necessary to implement (his-plan;. fr 13. 8. 92 Official Journal of the. Europeah Conimunitics No L 231/27 Whereas the amounts to be committed for thé financing of the plan will have to come within the Community financial framework in force; The Commission shall refer to the committee referred to in paragraph 2 and the Council those initiatives adopted in the framework of Community policies which h aw a major effect on tourism. Whereas procedures should. be laid down for the exercise of the powers for implementing this plan conferred on the Commission pursuant to Decision 87/373/EEC ('); Whereas the Treaty does not provide for any powers for the adoption of this Decision other than those mentioned in Article 235, HAS DECIDED AS FOLLOWS: Article I A Community action plan to assist tourism shall be drawn up. The measures forming the subject of this plan are contained in the Annex. • - i Article 2 1. The duration of the action plan shall be three years from 1 January 1993. 2. The Commission shall be assisted in implementing the action plan by a committee composed of representatives of the Member States and chaired by the reprcsentativeof the Commission. The representative of the Commission shall submit to the to be taken. The committee a draft of the measures committee shall deliver its opinion on the draft within a time limit which the chairman may lay down according to the urgency of the matter. T he opinion shall be delivered by the majority laid down in Article 148 (2) of the Treaty. in the case of decisions which the Council is required to adopt on a proposal from the Commission. The votes of the representatives of the committee shall be_weighted in the manner set out in that Article. The chairman lhall not vote. the Member States within if these measures are not The Commission shall adopt measures which shall apply \ immediately. However, in accordance with the opinion of the committee, they shall be communicated by the Council the Commission to forthwith. the Commission shall defer application of the measures which it has decided for a period of two months from the date of communication. that event In 2. The Community financial resource|_decmed necessary for its implementation amount to ECUM8ynilIion and shall fall within the Community financial framework in force. The Council, acting by a qualified majority, may take a different decision within the time limit referred to in the previous paragraph. 3. The budget authority shall determine the appro priations available for each financial year, taking into account the principles of sound management referred to in Article 2 of the Financial Regulation applicable to the general budget of the European Communities. Article J the action plan I. The Commission shall put into operation. In order to fulfil the objectives of the plan, it may undertake measures other than those set out in the \nncx, where, exceptionally, additional, action is required n order to carry out one of the measures in full. Such tdditional action shall be assessed to existing priorities and to available financial resources. The -ommission shall coordinate the action with the various Community policies, and through the various Dircctoratcs- /cncral -current' roceduces. accordance • with in relation both concerned, in ) OJ No I. 197. 18. 7, 1987, p. JJ. Article 4 1. The measures must be consistent with (he principle of subsidiarity. 2. A selection shall be made, for the different measures proposed, by reference to the following criteria: (a) the measures must be cost-effective and make a significant impact on the Community tourist industry; (b) they niust facilitate the development of the industry with particular medium-sized businesses; reference tourist to small and (c) they must help improve the quality of Community tourist services;. (d) "they must encourage Community and increase ' C o m m u n i ty tourist services on the world market;. competition within the the competitiveness of M> No L 2 3 1 / 28 Official Journal of the European Communities 13. 8. 92 (e) they must be conducive to preserving and protection the quality of the natural environment, the cultural heritage and the integrity of local populations; (f) they must be conducive to improving the provision of information and services and to the protection ot tourists. be through shall 3. The measures coordination with if and, the national necessary, with the regional or local authorities as well, so as to take account of the for regional development. implemented authorities importance of tourism Article 6 The Commission shall regularly evaluate the results of the action plan. This evaluation will include wherever possible measurable outputs of the plan and be in accordance with the criteria in- Article. 4. T he committee will be informed by the Commission of the latter's evaluation of the plan, and of the results thereof. No later than 30 June 1 9 9 5, the Commission will submit a report on this evaluation to the European Parliament and the Council. On the basis of that report, the Council shall decide, in accordance with the provisions of the Treaty, whether or not to extend the plan for a further period. Article S Done at Brussels, 13 July 1 9 9 2. Every year from. the date of adoption of the action plan, the Commission, in a report to the European Parliament and the Council, shall evaluate the Community's activities which affect tourism. For the Council The President J. CUMMER f-r* 13. 8. 92 Official Journal of the European Communities No L 231/29 I. COMMUNITY MEASURES TO ASSIST TOURISM ANNEX V. Improving knowledge of the tourist industry and ensuring greater consistency of Community measures Community action 'is intended to improve the consistency of the measures taken to assist tourism by increasing knowledge of its characteristics, components and development. This will be carried out by means of the following measures: (a) development of Community statistics on tourism; (b) detailed studies aimed at improving knowledge of tourism as an activity, assessment of the impact of current Community. policies to assist tourism, forward analysis of new types of tourism, and the preparation of strategies adapted to keep pace with demand-, (c) consultation of tourism professionals within the Communicy. 2. Staggering of holidays Community action is designed to promote a better seasonal distribution of tourism. This will be carried out by means of the following measures: - (a) support for the setting-up of an international framework Jwhose purpose would be to exchange information and monitor the activities of governments and the tourist industry; (b) support for measures aimed at coordinating actions and strategies to encourage the use of tourism infrastructure and facilities outside the peak season. 3. Transnational measures Community action is designed to promote transnational tourist development initiatives covering many different specialist sectors of the industry. This will be carried out by means of the following measures: (a) support for cooperation between border regions; (b) support for transnational initiatives contributing to the improvement of tourist information, in particular those using new technology; (c) development of tourist cooperation wich Central and Eastern Europe and the Maghreb through the transfer of k n o w - h ow on training and the implementation of strategies for promotion, as well as on marketing and the crearion of small and medium-Sized tourist enterprises; (d) support for tourist and technical cooperation in the context of partnerships between towns; (e) support for. pilot projects aimed ar cooperation between the public and private sectors for the development of traditional tourist regions in decline as well as less developed rural regions. 4. Tourists as consumers Community action aims to support initiatives which improve the information of tourists and their time-share existing classification protection, arrangements, over-booking and procedures for redress. systems, ' signposting in areas such symbols, is 5. Cultural tourism , Community action is designed both to highlight theimportancc of the cultural heritage for tourism and to promote a greater knowledge of the cultures, traditions and ways of life of Europeans, ff No L 231/30 Official Journal of the European Communities 1 3. 8. 92 This will be carried out by means of the following measures: (a) support for initiatives to develop n ew European cultural tourism routes, in cooperation with the - Member States, regions arid local authorities concerned, and to disseminate information on these routes by means of brochures and publications; (b) support for the exchange of experience in the field of visitor management techniques; (c) promotion and assistance in the use of European networks enabling tourist operators and cultural institutions to exchange experience, -especially as regards highlighting the value of cultural heritage. 6. Tourism and the environment T he aim of Community action in the area of the interaction between tourism and the environment is to ensure that the environment is more fully taken into account. This will be carried out by means of the following measures: (a) support for iniriatives aimed at informing arid increasing the awareness of tourists and suppliers of services about the interaction between tourism and the environment and in particular through the creation of a European environmental prize;' (b) support for innovative pilot projects to reconcile tourism and nature protection at local or regional level, in particular coastal and mountain areas, nature parks and reserves, e. g. by measures for the guidance of visitors; (c) support for the development of networks including experience of environmental problems and their possible solution management at sites; involving transnational exchanges of experience, through visitor (d) support for initiatives'encouraging forms of environment-friendly tourism. 7. Rural tourism I Community action in this field is designed to develop tourist activities in a rural environment, notably farm tourism, small family-run hotels or facilities set up by associations or local authorities. This will be carried out by means of the following measures: (a) support for partnership initiatives between operators at local, regional, national or European level, to facilitate exchanges of experience and the transfer of good practice through the organization of visits, seminars, exchanges of experts and the development of transnational pilot schemes, in particular in the field of vocational training; (b) improved information for rural operators and better access for them to the various Community aid schemes available for rural tourism, in-particular through the publication of documents for mass circulation and the publication of an operators' manual-, (c) encouragement for. improving the quality of rural tourism supply and support for measures to facilitate access to tourism in a rural environment. 8. Social tourism Community activity in this field seeks to facilitate access to tourism hy groups of people w h o, for various reasons, but especially for social or health reasons, have difficulty in taking holidays. This will be carried out by means of the following measures: (a) shared information at Community level between public and'private sector partners concerning the various methods used in the Member States to encourage holiday-taking by certain categories of tourists; (b) support for the coordination between Member. States of measures aimed at eliminating barriers to the development of tourism for the disabled, and for the exchange"of information in (his field. 9. Youth tourism Community action in this field is, through support for existing Community policies, aimed both at promoting young people's knowledge of cultures and lifestyles'in the various Member States and at making it easier for young people'to take holidays. " ft 1 3. 8. 9 2,. Official Journal of the European Communities •• No L 2 3 1 / 31 This will be carried out by means of the following measures: (a) a feasibility study into establishing links between ' y ° -tn cards'; (b) support for research into the need to create a network of exchanges with regard to 'European classes* (school travel for pupils from several Member States). 10. Training Community action in this field is aimed, through support (or existing Community policies, at making the tourist industry in the Community more competitive through support for increased professionalism in Community tourism. This will be carried out by means of the following measures: (a) dissemination of information among young people on tourist resources and profession; (b) support for ongoing measures to draw up professional profiles for the industry and improvement of mutual information on the qualifications attained in the various Member States; (c) encouragement of the participation of tourist businesses and their employees in existing Community training programmes and measures; (d) support for transnational cooperation projects, between universities, tourism schools, tourism professionals, or the authorities concerned, especially for training in the fields of rural, cultural and environmental tourism; (e) support for networks aimed at improving the quality of vocational training so as to raise the quality of tourism services. 11. Promotion in third countries Community action in this field is directed at making Europe a more attractive destination for tourists from distant countries. This will be carried out by means of measures confined to pilot projects to promote Europe as a tourist destination on the markets of distant countries, particularly North America and Japan, whose growth is likely to have an impact on tourism within the Community. I II. TIMETABLE OF PRIORITIES When implementing measures in accordance with thé procedure laid down in Article 3, priority will be given for 1993. to the following measures. These priorities may be modified for the financial years 1994/95 according ro the procedure referred ro in Article 3 (2). Priority measures for 1993: I. Improving knowledge of the tourist industry and ensuring greater consistency of Community measures: (a) development of Community statistics on tourism; (b) detailed studies aimed at improving knowledge of tourisme as an activity, assessment of the impact of current Community policies to assist tourism, forward analysis of new' types of tourism and the preparation of strategics adapted to keep pace with demand; (c) consultation of tourism pofcssionals within the Community. 2. Staggering of holidays: • i- (a) support for the setting-up of an. international framework whose prupose would be to exchange information and monitor the activities of governments and the. tourist industry. Ac® No L 2 3 1 / 32 Official j o u r n al of the E u r o p e an C o m m u n i t i es 1 3 : 8. 92 '3'. Transnational measures: (a) support of cooperation between border regions: (b) support for transnational initiatives contributing to the. improvement of tourist information, in particular those using new technology; • ' ,. '- (c) development of tourist cooperation with Central and Eastern Europe and the Maghreb through the transfer of k n o w - h ow on training and the implementation of strategies for promotion, as well as on marketing and the creation of small and medium-sized tourist businesses. 5. Cultural tourism: (a) support for initiatives to develop new European cultural tourism routes, in cooperation with the Member States, regions and local authorities concerned, and to disseminate information on these routes by means of brochures and publications; (b) support for the exchange of experience in the field of visitor management techniques-, 6. Tourism and the environment: (a) support for initiatives aimed at informing and increasing the awareness of tourists and suppliers of services about the interaction between tourism and the environment and in particular through the creation of a European environmental prize; (b) support for innovative pilot projects to reconcile tourism and nature protection at local or regional level, in particular coastal and mountain areas, nature parks and reserves, e. g. by measures for the guidance of visitors-, (c) support for the development of networks transnational exchanges of experience, including experience of environmental problems and their possible solution through visitor management at sites; involving (d) support for initiatives encouraging forms of environment-friendly tourism. 7. Rural tourism: (b) improved information for rural operators and better access for them to the various Community aid schemes available for rural tourism, in particular through the publication of documents for mass circulation and the publication of an operators's manual; / (c) encouragement for improving the quality of rural tourism supply and support for measures to • facilitate access to tourism in a rural environment. 8. Social tourism: (b) support for the coordination between Member States of measures aimed at eliminating barriers to the development of tourism for the disabled, and for the exchange of information in this field. 10. Training: (b) support for ongoing measures to draw up professional profiles for the industry and improvment of mutual information on the qualifications attained in thevarious Member States; (c) encouragement of the panicipation of tourist businesses and their employees in existing Community training programmes and measures; (d) support for transnational cooperation projects between universities, tourism schools, t o u r i s m, professionals, or the authorities concerned, especially for training in the fields of rural, cultural and environmental tourism. • i ' 1 I. Promotion in third countries: Pilot projects to promote Europe as a tourist destination on the markets of distant countries, particularly North America and Japan, whose growth is likely to have an impact on tourisrk within the Community. ^1 EVALUATION OF THE TOURISM ACTION PLAN TO ASSIST TOURISM APPENDIX VII - LIST OF CONTACTS • Final Report - V2. 0 -M APPENDIX VII EVALUATION OF THE COMMUNITY ACTION PLAN TO ASSIST TOURISM ILIST OF CONTACTS (excluding DG XXIII) No Nam» ;; piyantwdon Department of National Heritage (UK) Minlsterio de Comercio y Turismo (E) Department of Trade & Tourism (IRL) Department of Trade & Tourism (IRL) Borde Fallte (IRL) ETOA ETC AIT/FIA ECTAA Deutscher Stadte und Gemeintebund (D) EUTO WTTC HOTREC ECATRA IATA ETAG UFTAA European Travel Monitor (Munich, D) Luxembourg Finland France Greek National Tourism Organisation Belgium 1 Daniela Hesse-Wagner 2 Robert Hollier 3 Lisa Davies 4 De Blust 5 E Giesen 6 W Burnett 7 G Llpman 8 M Sequaris 9 F H Aldous 10 Mr Smithies 11 LLickorish 12 Lepage 13 MrFreitag 14 RZimmer 15 Bent PilstrSm 16 JL Deroide 17 E Hadzinikolaou 18 Claude Pirard 19 Dr. ssa. Cesarina Mislani Dipartimento del Turismo (1) 20 Keith Gibbins 21 Teresa Castillo 22 Shane O'Neill 23 Paul Bates 24 Ciaran Tulte 25 JP Bastos Salazar Lelte Directorate General for Tourism (P) 26 Mr Schneider 27 L Harrison 28 E McMillan-Scott 29 P Comelissen 30 Mr Hansen 31 Giacomo Regaldo 32 Kirchberger 33 Van Hasselt 34 E Power 35 D Givord 36 Comaert 37 M Lepelletier 38 BWeherns 39 Manuel Ruiz Sanz 40 Xavier Coyer 41 Danae Penn 42 Pyrhus Mercouris 43 Mr Enthoven 44 T O'Dwyer 45 G Barrett 46 E Pongas 47 Professor V Middleton 48 E Pad 49 Mr Dupeyras 50 Prof John O'Hagan 51 George Moustakas 52 Palle Damkjaer 53 Rene Ladiges 54 Elsisabeth de Bodinat 55 Christine Bussière 56 Yves Comu 57 Sylvie Blangy 58 Paolo Bonglni 59 Gerry Carver 60 A de Varax 61 Elizabeth Jeffries 62 Hans Domenlcus 63 Kal Gelger 64 Jan Kobemuss 65 Falco de Clerck 66 Andrew Lennox 67 Carlos Freite 68 Ana Alonso 69 lEulogio Bordas Bundesmlnlsterium fur Wirtschaft (D) European Parliament European Parliament European Parliament Committee of the Regions Economic & Social Committee DG XXII Air Transport Policy - DG VII DG XVI « Regional Policies Rural Development - DG VI DGXI DG VII DGV DGV DGV DGVII DG X DGXl DGXX1I DGVIII Eurostat World Trade Organisation (WTO) OECD Trinity College (Dublin. IRL) International Interactive Multimedia Hjoerring Business College (DK) Hjoerring Business College (DK) Agence Vitesse (Paris. F) Agence Vitesse (Paris. F) HTL - Bossard Sociologie (Paris. F) Expert (Montpellier. F) Regione Toscana (Florence, I) L&R Leisure (Haywards Heath, UK) Office du Tourisme (Lyon, F) Greater Manchester (UK) Amsterdam Tourist Office (Amsterdam, NL) IfB (Konstanz. D) Institute fur Freizeit (Cologne. D) Institute for Service Management (Leeuwarden. NL) Central College of Commerce (Glasgow, UK) Arthur Andersen (Lisbon. P) DeMco Bates (Madrid. E) |THR (Barcelona, E) I AD _? EVALUATION OF THE ACTION PLAN TO ASSIST TOURISM APPENDIX VIII - ASSESSMENT BY KEY INFORMANTS 0 In this Appendix, we give a summary of the assessment made by the key informants (professional associations representing industry interests, representatives of the Member States, independent experts, Commission officials from other Directorates General). The views expressed are not necessarily shared by Price Waterhouse. Exposure to the Plan The vast majority of interviewees are familiar with the Plan, although some of them consider their knowledge of the plan stems from extensive contacts and length of time within the industry, rather than from the Plan's contact with the tourism organisation. Whilst many interviewees have only a more general understanding of the Plan, certain interviewees feel they have more of a knowledge of individual projects rather than an understanding of the Plan's strategy itself. Many interviewees gained familiarity with the Plan as a result of attending Management or Consultative Committees. The degree of familiarity with the Plan by officials from other Directorates General appear to be the lowest. An important feature is the background of the interviewees, especially which part of the travel and tourism industry they represent. There is a strong tendency for the "major players" in the industry such as the organisations representing the European travel agents and tour operators to consider that the Plan focuses mainly on peripheral tourism issues (green / social) and not on the key mainstream issues such as transport links and infrastructure. On the other hand, organisations such as those representing tourist offices in local communities consider the Plan only consults major forces in the industry and does not seek or take account of the views of other important but less powerful representatives. Interface with the Plan The interviewees have highlighted a number of organisations, particularly at the "global" level working towards the same broad objectives as the Plan: creating jobs by stimulating tourism and developing tourism with due regard for the environment. Whilst the Plan consults regularly with these organisations through attendance at meetings, advisory committees etc, it is felt that there is a need for the Plan to ensure greater coordination with the strategies of these other organisations. In particular, difficulties were expressed vis-à-vis the ability to co-ordinate what member states are doing with what the EU is doing. One interviewee felt that the Plan was a "small scale" model of what an overall strategic programme of Travel and Tourism in Europe should be aiming to achieve. Another common theme was that the Plan required much greater coordination and integration with the infrastructure projects handled by other DGs in the Commission eg air traffic control systems, transport measures for vehicles etc. Final Report - V2. 0 4 &^i EVALUATION OF THE ACTION PLAN TO ASSIST TOURISM APPENDIX VIII - ASSESSMENT BY KEY INFORMANTS gtfk ^IlF Rationale for the Plan 10 11 12 13 14 15 16 The majority of interviewees expressed a view on whether the Plan made sense. Whilst these interviewees generally felt that the overall framework was commendable and "going in the right direction", they considered the actual outputs and activities were too fragmented and that the individual projects were on too small a scale. There is a general view that undertaking these small projects spread over a number of different tourism aspects has made it difficult to achieve the Plan's objectives. A number of interviewees to date consider the activities have focused on peripheral issues eg cultural and social tourism rather than on the mainstream tourism industry which accounts "for about 80% of all tourism activity". Some also felt that the Plan had concentrated too much on theoretical issues and research studies. Even more importantly, many considered that the results of these studies had not been made available to a wide enough audience and in particular to those bodies and organisations that could make best use of them. One interviewee felt there was too much emphasis on supporting the public sector's role in tourism and that there were insufficient projects related to the private sector's role. / There is a general view that one of Plan's most valuable roles has been to act as a focal point for tourism within the EU and to get the many different stakeholders within the industry together and thinking about the same issues. To some extent, the Plan has acted as a catalyst for future action. Generally, interviewees think that the following factors have contributed to "the small and fragmented project" approach: a b c d e the need to satisfy a very broad framework of objectives ("a patchwork") the tendency to attempt to meet the needs of every lobby with an interest in tourism which has led to funding fragmented, small projects on a number of areas inadequate funding given the ambitious objectives of the Plan inadequate staffing of the Tourism Unit (both in terms of number and expertise) the lack of top level policy commitment and an insufficient authority vested in DG XXIII to enable it to fulfill adequately its coordination role with other EU mainstream policy areas such as Agriculture (DG VI) Regional Policy (DG XVI). 17 These factors have meant that, to date, the Plan has been unable to play a significant role in maximising tourism's potential contribution to economic growth and job creation. Final Report • V2. 0 <i e>f EVALUATION OF THE ACTION PLAN TO ASSIST TOURISM APPENDIX VIII - ASSESSMENT BY KEY INFORMANTS gVk ^S\W Assessment of Impacts and Effects 18 19 Whilst a majority of interviewees consider that the net effects of the Plan are positive, several of them take the view that the negative effects have outweighed the positive effects. It is generally felt that the overall impact of the Plan on the competitiveness of the EU tourism industry has been very limited. 20 The Plan has had the following positive impacts: a b c brought Member States together for useful discussions on Tourism (catalyst role) increased the awareness of particular key issues amongst Member States, for example the environmental aspects enhanced cooperation on Tourism issues amongst Member States (eg through the consultation process of the Green Paper) d activated several sectoral/regional projects in Europe. The general opinion is that DG XXIII has done as well as it could have done, given the very broad framework for Plan and the constraints on funding. There is however a minority view that the implementation of the plan has been a "lost opportunity" and that the Unit has been ineffective. One reason sometimes given is the lack of tourism experience of most staff members. Travel agents and tour operators generally consider that the Plan has provided few benefits and has had little impact on the tourism industry. This sector and the industry generally feel they should have been consulted much more on the strategy at the outset. They generally felt that the Plan should have concentrated on the mainstream areas of tourism; that there should have been more effective coordination internally across the EU directorates responsible for Transport policy etc. as well as those responsible for the Structural Funds. There should also have been more coordination between other external organisations, especially those responsible for environmental issues. More emphasis should be placed on the great potential which the Tourism industry has for promoting sustainable development, and improving and maintaining the environment. Most representatives of the Member States contacted think that they need to be involved much more directly in the formulation of tourism policy and programmes, so that they take "ownership" of the programme and subsequently work actively together to implement it. The Plan has been regarded much more as the Commission's plan rather than that of the Member States'. There is a feeling that the Plan is "caught between two stools", ie in meeting the subsidiarity principle, yet still being expected to deal with core issues: if the Plan adheres to subsidiarity, then it may only add value in areas not seen as core such as social and green tourism. There is a view that many of the Plan's initiatives are related to local authority activities and not so much to the private sector. There is no unanimity on the effectiveness of the implementation of the Plan by the Tourism Unit. Whilst some interviewees feel that DG XXIII have done as well as they could "given the broad 21 22 23 24 25 26 27 28 Final Report - V2. 0 s?£><3 EVALUATION OF THE ACTION PLAN TO ASSIST TOURISM APPENDIX VIII - ASSESSMENT BY KEY INFORMANTS 4» framework and the constraints or funding", others take the view that the Commission has not been effective. The latter group gives evidence such as: cancelation of Conference on Quality of Tourism, lack of expertise in tourism, very long and bureaucratic procedures, delays in payment, lack of responsiveness of DG XXIII staff, lack of transparency in the selection of projects, poor execution of the budget (eg in 1994). 29 It is also clear from our interviews that the misallocations of funds have given the Plan a "bad name": the credibility of the Tourism Unit and of a possible EU tourism policy has been seriously tarnished. It has also to some extent created the some of the administrative difficulties and weaknesses referred to above. Relative Successes and Failures at Project Level 30 Interviewees cited the following as "successes" at project and study level: a b c d e f I Cultural itineraries, such as "Routes to the Roots" and "Art Cities in Europe" Baltic Sea Region Cultural Project A number of projects in the area of Tourism & the Environment (eg Prize) Certain projects in the area of information technology Projects developed in Greece to develop training modules on training, management and technical skills Senior citizens' and youth tourism; both were seen to be examples of projects which were expanding tourism in culturally and environmentally sensitive ways 31 The Directive of Community Statistics and certain conferences organised by the Tourism Unit are other activities which appear to have been appreciated. 32 Interviewees cited the following as relative "failures" at project and study level: a b c The study on "overbooking in hotels etc" was cited as unnecessary since it was looking at an issue already being resolved by the industry itself The promotion in the USA, "a monumental worry", a "complete waste of money" The failure of the promotion in Japan (1995 call for tender). Role and Activities in the Future 33 For the future, a number of common themes emerged, although there is no unanimity. Among the representatives of the Member States at the Management Committee, examples of common themes include: a more and better coordination and dialogue (partnership) with: (a) the EU directorates responsible for the relevant mainstream programmes such as Transport, Regional Policy, issues of Taxation and Single Currency (including a better Final Report • V2. 0 4 ° ? EVALUATION OF THE ACTION PUN TO ASSIST TOURISM APPENDIX VIII - ASSESSMENT BY KEY INFORMANTS use of the existing EU instruments) (b) the national tourist strategies and plans 0 (c) regional and local authorities, Chambers of Commerce and other industry stakeholders b c d e f formally recognise Tourism's potential in the areas of economic development and generating employment opportunities concentrate on promoting "Europe" to third party countries such as Japan and the Far East - with individual EU countries left to promote themselves at international, national, regional and local level need for planning a Tourism strategy which will have the greatest impact on economic growth and job creation within the EU, and for supporting projects which meet these objectives greater involvement of the private sector in both the formation and implementation of policy need to rethink the co-operation with third countries in order to make it consistent with the objectives of tourism policy g need to publicise the results and the outcomes of seminars, projects etc. I Summary of Findings We now summarise the findings from our interviews with key informants. Note that these are not necessarily the views of Price Waterhouse. The Plan has given "Tourism" more recognition within the EU. It has brought interested parties and stakeholders together to discuss common issues. But the Plan's framework covers too many objectives. Some of these objectives are on fringe or peripheral issues. Consequently, the plan did not include all the mainstream issues, especially regarding infrastructure aspects. It is felt by many key informants inten/iewed that funding and staffing of the Tourism Unit have not been adequate and these are seen as a key factors in limiting the programme's impact. There is a need for greater consultation of all key parties at the strategy formulation stage. This needs to include both the public and private sectors. Both Member States and industry representatives consider they were not actively involved in deciding on the Plan's strategy. There needs to be greater coordination on key issues (such as competitiveness) both internally within the Commission and with external bodies to ensure compatibility of policies, especially on transport and environmental issues. Need to be clear what the "high level" key objectives are and ensure that projects are assessed against these "key criteria" eg from which areas of the world is the EU trying to attract tourists to come to Europe? Which are the target markets? 34 35 36 37 38 39 40 41 This leads on to thinking about what needs to be developed eg. high standards and common Final Report - V2. 0 yip EVALUATION OF THE ACTION PLAN TO ASSIST TOURISM APPENDIX VIII - ASSESSMENT BY KEY INFORMANTS 0 classification of these standards for hotels etc across Member States. This is also seen to link with the need to develop educational programmes on tourism in colleges etc in the different Member States as well as to widen the use and capability of information technology for tourism on a pan - European basis. i Final Report • V2. 0 ^1 ISSN 0254-1475 COM(96) 166 final DOCUMENTS EN 16 Catalogue number : CB-CO-96-209-EN-C ISBN 92-78-03582-3 Office for Official Publications of the European Communities L-2985 Luxembouig