The Internet Software Visualization Laboratory1 John Domingue Paul Mulholland Knowledge Media Institute The Open University Walton Hall Milton Keynes , UK MK7 6AA +44 1908 65-5014 -LCB- J.B.Domingue , P.Mulholland -RCB- @open . ac.uk ABSTRACT The Internet Software Visualization Laboratory -LRB- ISVL -RRB- combines our research in Software Visualization and teaching over the internet to tackle the problem of how computer programming can be effectively taught to students working from home . Our approach provides a rich , collaborative environment for exploring and demonstrating programming constructs . ISVL supports both asynchronous and synchronous working , and allows students to move seamlessly from a tutor-led teaching scenario , to personal or peer exploration . ISVL provides a rich source of empirical data which will shed light on how Software Visualization can be most effectively incorporated within a computer programming curriculum . INTRODUCTION Teaching computer computer programming has long been known to be a complex process . This problem is compounded on our own courses where teaching is carried out a distance , raising the following issues : 1 -RRB- The lack of face-to-face support - contact between student and tutor is either by telephone or email , meaning the tutor establishing the nature of the students ? problem can take a great deal of time and be prone to misunderstanding . 2 -RRB- Students can rarely work together - as the students work part time and fit their studies around other committments it is impossible for them to work together synchronously . 3 -RRB- The lack of a computer laboratory - students work at home using a range of computers , many of fairly low specification , meaning a range of platforms have to be supported . The approach taken to alleviate these problems was to develop a software visualization laboratory . Software Visualization -LRB- SV -RRB- is the use of graphical and textual formalisms to describe the execution of computer programs . We view SV as having a valuable role within the teaching of computer programming , particularly for languages with a complex execution model . Our view is influenced by empirical work which found that a crucial objective when teaching programming was to provide a clear execution model -LRB- Eisenstadt et al. 1985 -RRB- . This fits well with the aims behind SV . A key role envisaged for SV is in computer science education . SV permits students to have an overall conceptual story of how the program works , explore and debug programs , and use the visualizations as a form of language in which to convey their understanding . The laboratory has the folowing characteristics : 1 -RRB- an internet client server architecture using platform independent software for the client -LRB- a java compliant browser -RRB- and placing most of the work on the server -LRB- to support low specification machines -RRB- , 2 -RRB- an environment that can provide a context for synchronous communication , so that the tutor can quickly establish what the student wishes to discuss , and 3 -RRB- a facility which enables asynchronous communication by allowing tutor and student explanations to be archived for future reference . The approach of using SV over a network as an educational tool raises futher issues as to how SV can be most appropriately included within a computer programming curriculum . The key theoretical issue we to address concerns the use of single or multiple representations to support learning . One one hand specific representations could be hand-picked to help student at particular stages of the course -LRB- i.e. a 1Submitted to PPIG ? 97 , Psychology of Programming Interest Group 9th Annual Workshop Jan. 3rd - 5th 1997 Sheffield Hallam University , Sheffield , UK The Internet Software Visualization Laboratory Page 2 stage appropriate approach -RRB- . This means each representation will be carefully tailored to the students needs at that time , but will mean the student will have to learn a greater number of representations . On the other hand a single ? best bet ? representation could be used throughout -LRB- i.e. a cradle-to-grave approach -RRB- . Our new laboratory will allow us to investigate this issue . The next section describes the theoretical underpinnings of our approach . This is followed by an outline of the design requirements for the new environment and a tour of the ISVL environment . The final section contains a brief discussion on the benefits of ISVL and some conclusions . THEORETICAL BASIS Research into SV was inspired by the Sorting Out Sorting film -LRB- Baecker and Sherman , 1981 -RRB- which used animations to explain sorting algorithms . SVs can be used to present different perspectives of program execution , and are generally classified into two distinct groups : Algorithm Animation systems and Program Visualization systems . Algorithm Animation systems illustrate program execution in an algorithmic language independent way . Notable examples include BALSA -LRB- Brown , 1987 -RRB- , Zeus -LRB- Brown , 1991 -RRB- and AACE -LRB- Gloor , 1992 -RRB- . Program Visualizations such as the Transparent Prolog Machine -LRB- Eisenstadt and Brayshaw , 1988 ; Brayshaw and Eisenstadt , 1991 -RRB- for Prolog and ZStep for Lisp -LRB- Lieberman and Fry , 1995 -RRB- present an account of execution at the code level . ISVL takes the form of a visualization framework , able to represent execution on both the algorithmic and code level . Such frameworks already exist , including TANGO -LRB- Stasko , 1990 -RRB- and Viz -LRB- Domingue et al , 1992 -RRB- , though ISVL is a platform independent internet-based environment , supporting other aspects of the educational experience including peer collaboration , tutorials and the handling of assignments . ISVL will form a central component of a new web-based programming course . Though many claims have been made by SV designers for the educational suitability of their product , little has been said as to the stage in the learning process when particular visualizations are most appropriate . Conversely , some SV designers have claimed their product applies equally well from novice through to expert . For example , this claim was made by Brayshaw and Eisenstadt -LRB- 1991 -RRB- for the Transparent Prolog Machine -LRB- TPM -RRB- . TPM presents program execution graphically as an AND/OR tree , in order to give the detailed account of execution an ? at a glance ? quality allowing information to accessed quickly -LRB- see figure 1 -RRB- . Figure 1 . The Transparent Prolog Machine -LRB- TPM -RRB- AND/OR tree showing an overall perspective of the execution , and a TPM fine-grained view of the top most node showing execution details . An alternative viewpoint would be that particular types of representation are appropriate for particular stages in the learning process . This debate can be summarised as a cradle to grave versus stage appropriate view of how SV should be integrated into the learning experience . There is a wealth of evidence to suggest that the kind of knowledge structures and strategies used by expert and novice computer programmers differ qualitatively rather than quantitatively , and this affects how they are able to use different kinds of SV . For example , Chi et al. -LRB- 1981 -RRB- found that novices tend to focus on the surface features of the language whereas experts tend to use higher level structural representations . Recent SV evaluation studies suggest important differences in the way novices and experts are able use SVs -LRB- Mulholland , 1995 -RRB- . The empirical work suggests that novices tend to utilise a fairly specific group of strategies which use core programming information . These were termed comprehension strategies . Three types of comprehension strategy were deemed central to the effective novice use of the SV . These were review , mapping and test strategies . The Internet Software Visualization Laboratory Page 3 A review strategy was defined as using the historical perspective present in the SV display to work out how the current position in the execution was reached . A test strategy was defined as predicting how the execution will develop during future steps of the SV and then moving forward to test this prediction . A mapping strategy was the process of mapping between the information contained in the SV and the source code being visualized . Novices were only able to use these strategies if the information on which they relied was explicit . Implicit information , using what Petre and Green -LRB- 1990 -RRB- term the secondary notation , could not be utilised effectively by the novices . These findings motivated the development of the Theseus SV for Prolog , which has been found to be very effective for a novice population -LRB- see figure 2 -RRB- . ? ? fun -LRB- What -RRB- ? 1 fun -LRB- -LCB- What = X -RCB- -RRB- ? bike -LRB- What -RRB- +1 bike -LRB- -LCB- What = honda -RCB- -RRB- ? blue -LRB- honda -RRB- - m blue -LRB- honda -RRB- ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ? > 2 fun -LRB- -LCB- What = X -RCB- -RRB- ? ? car -LRB- What -RRB- ? +1 car -LRB- -LCB- What = mini -RCB- -RRB- ? ? red -LRB- mini -RRB- ? +1 red -LRB- mini -RRB- ? +2 fun -LRB- -LCB- What = mini -RCB- -RRB- Figure 2 . The Thesesus SV which shows execution near to the code level . Experts , on the other hand , will rely on explicit information when initially unfamiliar with the display but unlike novices are later able to use the secondary notation to draw out high level patterns -LRB- Mulholland , 1995 -RRB- . This indicates that experts , when using tools such as SVs are able to draw on a qualitatively distinct set of strategies . This leads to an interesting dilemma when considering how SV should be incorporated into a computer programming curriculum . As the student commences the course as a novice and -LRB- hopefully -RRB- completes the course as an expert ; what kind -LRB- s -RRB- of SV should be used ? On one hand the novice could start by using a ? novice SV ? -LRB- e.g. Theseus -RRB- and be introduced to an ? expert SV ? -LRB- e.g. TPM -RRB- at a later time , i.e. a stage appropriate approach . This has the advantage of using SVs appropriate to each learning stage , though the disadvantage of students being required to shift from one supporting representation to another at some intermediate stage . On the other hand , the novice could be initially introduced to the ? expert SV ? and helped to grow into it , i.e. a cradle to grave approach . This has the advantage of having only one supporting representation to learn during the course . It has the disadvantage of the students requiring extra support from day one in the use of a complex environment . This conundrum lead us to the development of the ISVL environment , whereby a range of SVs can be used under one roof , and different kinds of support can be provided to help the students integrate the SV into the wider learning experience . Support is provided by allowing the SV to be used collaboratively as well as individually within the same environment . This draws on the work of Chi et al. -LRB- 1989 -RRB- who have shown the beneficial effects of self explanation on learning . ISVL therefore redefines the role of SV as being a method by which students and the tutor can communicate their understanding of a program , as well as solitary working . The next section outlines the design requirements of ISVL . DESIGN REQUIREMENTS The design requirements underlying ISVL are taken from three sources : those governed by the pragmatics of the course , motivation from related empirical work , and practical issues relating to the underlying architecture . Requirements From Pragmatics Of Course Our students are studying part time and are required to fit their studies around other commitments , which means they will tend to study at different times . The Open University currently runs courses across wide areas of Europe and is starting to offer courses to non-European countries . Students therefore require systems which are : -LRB- i -RRB- easy to use , -LRB- ii -RRB- low cost , -LRB- iii -RRB- multiplatform , -LRB- iv -RRB- accessible from anywhere in the world , and The Internet Software Visualization Laboratory Page 4 -LRB- v -RRB- enables asynchronous communication with their tutor and peers . Principles From Experiments In the theoretical section we described how experts were able to identify and strategically use implicit information from the SV display . Novices were unable to use such strategies , but did show the seeds of such a strategy when using TPM , in the form of what was termed an overview strategy . When using an overview strategy , the student would try and look for patterns in the display , though with little success . The richer collaborative environment will be used to support the student in developing the simple overview strategy into a form allowing more complex and abstract patterns to be appreciated . Such a strategy will help the student to view the program in more abstract and diverse ways . This will be particularly important when the student moves onto larger programs , with correspondingly larger execution spaces . Eisenstadt -LRB- 1993 -RRB- found that many difficult bugs are due to the distance between the bug cause and bug symptom . Locating such bugs requires the programmer to be able to appreciate larger patterns in the display . Scaffolding the student?s use of TPM with annotated sound which draws the students ? attention to such patterns will encourage them to develop these skills , which will be crucial at higher levels of expertise . ISVL also supports multiple representations of the execution to enable the student to take different perspectives depending on their task . Currently , ISVL features two SVs , TPM and Theseus , specifically chosen because of the different ways in which they present Prolog execution . Theseus shows a detailed textual account of execution near the code level . TPM provides more of a high level graphical account . Future studies aim to show how these two views can be used at various stages along the course , and how lessons learnt in one can be assimilated with the other view . ISVL also has a code highlighting option to emphasise the mapping between the visualization and the code . Design Principles from the Web-server based Architecture The fact the ISVL is composed of a central server and a client requires that we think about where the components should go . In general data which is on the client and is therefore local will be rapidly accessible . But local data increases start up time -LRB- because the Java code has to be downloaded -RRB- . Storing structures on the server reduces the student requirements for processor power and memory and facilitates collaboration . This has to be balanced , however , with the fact that access to server structures are slower and have a ? dial-up ? cost . The split we have used in ISVL is shown in figure 3 . The program execution history and views are stored on the server . The history is made up of events which cause players to change state -LRB- see -LRB- Domingue et al. , 1992 -RRB- for details -RRB- . A view consists of a series of locations -LRB- e.g. we have tree based and table based views -RRB- . A mapping maps a history structure to an icon/image class . The views and maps form the basis of the ISVL HTTP protocol which is used to transmit the SV . The client interprets the view and map data to produce the on screen images which the user manipulates through a navigator . The navigator allows large execution spaces to be viewed by using techniques such as scale and compression . Using a protocol specific to SV systems means that the amount of data transferred is minimal . Reproducing views and maps on the client means that an SV can be examined without the need for repeated accesses to the server . Program execution history -LRB- players , events , states -RRB- Views Mappings Images Navigator Client Server ISVL HTTP protocol Views Maps Figure 3 . The server/client split in ISVL . The Internet Software Visualization Laboratory Page 5 INTERNET SOFTWARE VISUALIZATION LABORATORY -LRB- ISVL -RRB- In this section we shall describe how ISVL embodies the approach outlined in the introduction , using a scenario involving a hypothetical student Bill Hoad and tutor Ingrid Johnson . Within the scenario we shall show how ISVL supports students debugging their programs and synchronous and asynchronous communication between students and their tutors . Bill is working on an exercise from his course workbook . The exercise asks students to write a sorting program , called qsort , which uses the quick sort algorithm . The quick sort algorithm works by splitting a list around an element into a list of lower numbers and a list of higher numbers which are then recursively sorted . The program should take an unsorted list and return a sorted list . For example , the query : qsort -LRB- -LSB- 4 , 3 , 1 -RSB- , _ y -RRB- where _ y represents an unbound variable , should return : YES _ y = -LSB- 1 , 3 , 4 -RSB- signifying that the query has been proved and the value of _ y is -LSB- 1 , 3 , 4 -RSB- . Bill loads his solution by copying it into the text area and clicking on the ? Load ? button in the control panel -LRB- see figure 4 below -RRB- . The following steps occur as numbered in figure 4 : 1 . Bill types his query : qsort -LRB- -LSB- 4 , 3 , 1 -RSB- , _ y -RRB- and clicks on the ? Evaluate ? button , 2 . The result of the query ? NO ? -LRB- indicating that the query has failed -RRB- and a TPM visualization are returned . 3 . Bill steps through the execution using the button in the control panel until he reaches the point shown in figure 4 He notices the suspicious failure of the arrowed qsort node and investigates this further by clicking on the node to obtain a Theseus style fine-grained view -LRB- also arrowed -RRB- . The finegrained view tells Bill that there is no match when the list to be sorted is empty . He fixes his code and tries the query again - this time it is successful . Some time later Bill tries his code with the query : qsort -LRB- -LSB- 4 , 2 , 3 , 1 -RSB- , _ y -RRB- and gets the result : YES _ y = -LSB- 2 , 1 , 3 , 4 -RSB- The Internet Software Visualization Laboratory Page 6 Text Area Graphics Area Control Panel 2 . Result of Prolog Query 1 . Prolog Query 3 . Fine-grained view of Figure 4 . The ISVL code developement environment . Even with the visualization Bill fails to track down the bug , so he calls up his tutor Ingrid . Ingrid agrees to start up a collaborative tutoring session . To start a tutoring session Ingrid needs to move to Bill?s work area and then start broadcasting . Tutors can connect to any of their student?s work areas by choosing their identifier from their current students page . Ingrid?s current student page is shown in figure 5 below . Using the mouse she chooses ? B. Hoad M251067 ? and her screen changes to one similar to figure 4 . She then starts broadcasting by clicking on the ? Broadcast ? button in the control panel . Bill puts his ISVL into receive mode by clicking on the ? Receive ? button . Ingrid?s interface actions are now replicated in Bill?s environment . Ingrid evaluates the query : qsort -LRB- -LSB- 4 , 1 , 2 , 3 -RSB- , _ y -RRB- turns on the fine-grained trace option and plays forward to the point shown in figure 6 . She then says into her microphone : ? Things have gone wrong in the highlighted region -LRB- Ingrid highlights the region by dragging the mouse -LRB- see figure 7 -RRB- -RRB- . Splitting the list -LSB- 3 , 1 -RSB- around 2 should yield the list -LSB- 1 -RSB- -LRB- numbers less than 2 -RRB- and the list -LSB- 3 -RSB- -LRB- numbers greater than 2 -RRB- . I can see that this is the first time that greater-than -LRB- > -RRB- has succeeded , here -LRB- Ingrid rings the white ? > ? node by clicking on the ? Trail ? button and then drawing a circle using the mouse -LRB- see figure 8a -RRB- -RRB- , and that the first split clause has been tried . The split here -LRB- Ingrid rings the The Internet Software Visualization Laboratory Page 7 top split in red -LRB- dark in greyscale -RRB- -LRB- see figure 8b -RRB- -RRB- has put 1 in the wrong place - this comes from this split -LRB- Ingrid rings the lower split in yellow -LRB- light in greyscale -RRB- -LRB- see figure 8c -RRB- -RRB- so there?s probably a data flow problem in your second split clause . Let me check your code . Yes , if we compare your two split clauses -LRB- Ingrid brings up Bill?s source code -RRB- you can see that you?ve not been consistent in the order your two variables ? Highs ? and ? Lows ? . ? Bill can now use this help to continue with the exercise unaided . ISVL automatically records and stores broadcast sessions allowing Bill to retrieve and replay the commented visualization at any time . Figure 5 . The ISVL interface for moving to a student work area . Figure 6 . The ? crucial point ? in the visualization which Ingrid plays to , so that she can explain Bill?s bug to him . The Internet Software Visualization Laboratory Page 8 Figure 7 . Ingrid highlights the problematic part of the execution on her machine and this is reflected on Bill?s interface . -LRB- a -RRB- -LRB- b -RRB- -LRB- c -RRB- Figure 8 . Three annotations Ingrid uses in her explanation of the buggy execution to Bill . ringing -LRB- a -RRB- the first successful ? > ? node , -LRB- b -RRB- the top split which returns an incorrect value -LRB- c -RRB- the lower split which has an incorrect call . DISCUSSION AND CONCLUSIONS The ISVL approach to integrating SV within the educational experience has a number of clear benefits : ? collaboration - either between students or with a tutor , ? synchronous and asynchronous communication - using live contact or archived material , ? platform independent - the client software is written in Java and will run on any Java aware browser , ? cost - no licensing costs for students , low distribution costs and no high specification machines required by students , ? detailed/quick evaluation - all student activity within ISVL can be stored and analysed centrally , ? reduce evaluation cycle - changes to ISVL in response to evaluation findings can be made quickly , ? enhanced tutor contact - a rich collaborative context supplements email and telephone contact , ? intelligent central server - provides options for semi-automated student support and data analysis . ISVL will allow us to continue our research into the evaluation of both SV and teaching over the internet . We have taught Prolog , Lisp , and Knowledge Engineering over the internet and have experimented with a virtual summer school for teaching psychology -LRB- Eisenstadt et al , 1996 -RRB- . Both the The Internet Software Visualization Laboratory Page 9 prolog and knowledge engineering have used visualizations -LRB- TPM and TRI -LRB- Domingue and Eisenstadt , 1989 -RRB- -RRB- as a key component , though never before delivered over the internet . This will extend our work to incorporate an enhanced level of collaboration . Similarly , Brown and Najork -LRB- 1996 -RRB- are working on an internet visualization called CAT -LRB- Collaborative Active Textbooks -RRB- which shows programs on an algorithmic level . Our own work is complementary to this , showing the execution at the code level , with the advantage that the user can more easily move from a tutoring to exploratory scenario , which is less possible with animations on the algorithmic level which require a working program . With the ISVL environment we will be able to undertake a more fine-grained evaluation , looking not only at usability issues but also deeper issues of external representations and how they impact on learning outcomes . The issues we intend our evaluation to address can be posed as three separate questions : 1 -RRB- What is the most effective use of SVs in education : cradle to grave or stage dependent ? 2 -RRB- How will students collaborate and share resources when working in a virtual teaching environment ? 3 -RRB- How will the tutor?s role differ when using ISVL compared to a traditional setting ? We hope our forthcoming evaluation of the course using ISVL will illuminate these areas . ISVL is based on a decade of empirical work and software development and will be used and tested on a real course . Due to the localised nature of the materials on the ISVL server , empirical findings using ISVL will be able to feed directly into the future presentation of the course , as well as informing us as to how SV technology can be most appropriately incorporated into a computer programming curriculum . The Internet Software Visualization Laboratory Page 10 REFERENCES Baecker , R. M. and Sherman , D. -LRB- 1981 -RRB- . Sorting Out Sorting . Narrated colour videotape , 30 minutes , presented at ACM SIGGRAPH ? 81 . Published by Morgan Kaufmann , Los Altos , CA . Brayshaw , M. and Eisenstadt , M. -LRB- 1991 -RRB- . A Practical Tracer for Prolog . International Journal of ManMachine Studies , 42 , 597-631 . Brown , M. H. -LRB- 1987 -RRB- . Algorithm Animation . Cambridge , MA : MIT Press . Brown , M. H. -LRB- 1991 -RRB- . Zeus : a system for algorithm animation and multi-view editing . In Proceedings of IEEE Workshop on Visual Languages . Kobe , Japan . Marc H. Brown and Marc A. Najork . -LRB- 1996 -RRB- . Collaborative Active Textbooks : A Web-Based Algorithm Animation System for an Electronic Classroom . Proceedings of the IEEE Symposium on Visual Languages -LRB- VL ' 96 -RRB- , Boulder , CO , Sept 3-6 , 1996 . Chi , M. T. H. , Feltovich , P.J. and Glaser , R. -LRB- 1981 -RRB- . Categorisation and representation of physics problems by experts and novices . Cognitive Science , 5 , 121-152 . Chi , M. T. H. , Bassok , M. , Lewis , M. W. , Reimann , P. and Glaser , R. -LRB- 1989 -RRB- . Self explanations : how students study and use examples in learning to solve problems . Cognitive Science , 13 , 145 - 182 . Domingue , J. and Eisenstadt , M. -LRB- 1989 -RRB- . A New Metaphor for the Graphical Explanation of Forward Chaining Rule Execution . In Proceedings of The Twelfth International Joint Conference on Artificial Intelligence -LRB- Detroit , MI -RRB- , 129-134 . Domingue , J. , Price , B. and Eisenstadt , M. -LRB- 1992 -RRB- . Viz : A Framework for Describing and Implementing Software Visualization Systems . In Proceedings of NATO Advanced Research Workshop : User-centred requirements for Software Engineering Environments , September , 1991 . Eisenstadt , M. , Breuker , J. , and Evertsz , R. -LRB- 1985 -RRB- . A cognitive account of ? natural ? looping constructs . In B. Shackel -LRB- Ed . -RRB- , Human-Computer Interaction . Amsterdam : Elsevier -LRB- NorthHolland -RRB- , 1985 . Eisenstadt , M. -LRB- 1993 -RRB- . Tales of Debugging from the Front Lines . Empirical Studies of Programmers V , Palo Alto , CA. , December , 1993 . Eisenstadt , M. and Brayshaw , M. -LRB- 1988 -RRB- . The Transparent Prolog Machine -LRB- TPM -RRB- : an execution model and graphical debugger for logic programming . Journal of Logic Programming , 5 -LRB- 4 -RRB- , 277-342 . Eisenstadt , M. , Bayshaw , M. , Hasemer , T. and Issroff , K. -LRB- 1996 -RRB- . Teaching , learning and collaborating at a virtual summer school . In A. Dix and R. Beale -LRB- Eds . -RRB- Remote cooperation : CSCW Issues for Mobile and Tele-workers . London : Springer . Gloor , P. A. -LRB- 1992 -RRB- . AACE - Algorithm animation for computer science education . In Proceedings of Visual Languages Workshop , Seattle , WA : IEEE Computer . Lieberman , H. and Fry , C. -LRB- 1995 -RRB- . Bridging the gap between code and behaviour in programming . In Proceedings of CHI ? 95 Human Factors in Computing Systems . New York : ACM Press . Mulholland , P. -LRB- 1995 -RRB- . A framework for describing and evaluating Software Visualization Systems : A case-study in Prolog . PhD Thesis , Knowledge Media Institute , Open University . Petre , M. and Green , T. R. G. -LRB- 1990 -RRB- . Where to draw the line with text : some claims by logic designers about graphics in notation . In Human-computer interaction INTERACT ? 90 . NorthHolland : Elsevier . Stasko , J. T. -LRB- 1990 -RRB- . Tango : A Framework and System for Algorithm Animation . IEEE Computer , 27-39 .

O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O B O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O B O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O

evaluation
prolog
software visualization


Using parallel simulated annealing in the mapping problem Borut Robi < = c and Jurij < = Silc Jo < = zef Stefan Institute , Computer Systems Department Jamova 39 , 61111 Ljubljana , Slovenia E-mail : borut.robic@ijs.si Technical Report CSD-94-4 April 1994 Using parallel simulated annealing in the mapping problem ? Borut Robi < = c and Jurij < = Silc Jo < = zef Stefan Institute , Computer Systems Department Jamova 39 , 61111 Ljubljana , Slovenia E-mail : borut.robic@ijs.si Abstract This paper presents a parallel simulated annealing algorithm for solving the problem of mapping irregular parallel programs onto homogeneous processor arrays with regular topology . The algorithm constructs and uses joint transformations . These transformations guarantee a high degree of parallelism that is bounded below by d jNpj deg -LRB- Gp -RRB- +1 e , where jNpj is the number of task nodes in the mapped program graph Gp and deg -LRB- Gp -RRB- is the maximal degree of a node in Gp . The mapping algorithm provides good program mappings -LRB- in terms of program execution time and the number of processors used -RRB- in a reasonable number of steps . ? This work has been supported by Ministry of Science and Technology of the Republic of Slovenia under Grant Numbers J2-5092-106 and Z2-5509-106 . This report will appear in the proceedings of the 6th International PARLE Conference , Athens , Greece , July 13-17 , 1994 . -LRB- Lecture Notes in Computer Science 817 -RRB- Technical Report CSD-94-4 April 1994 1 The mapping problem Let Gp = -LRB- Np ; Ap -RRB- be a program graph , where Np represents tasks and Ap represents data dependencies among them . Let a parallel computer be represented by a graph Gc = -LRB- Nc ; Ec -RRB- , where Nc is the set of processors and Ec is a set of physical interconnections . A mapping of the program graph into the parallel computer associates a processor to each task and a sequence of interconnections to each data dependency . Formally , a mapping ? of Gp into Gc is a pair -LRB- ? N ; ? A -RRB- of functions ? N : Np ?! Nc and ? A : Ap ?! Sk > = 1 Ekc , such that for each a = -LRB- u ; v -RRB- 2 Ap , ? A -LRB- a -RRB- is a path in Gc , starting in ? N -LRB- u -RRB- and ending in ? N -LRB- v -RRB- . A mapping must be feasible , i.e. , it must satisfy all the constraints imposed by the target computer . The fundamental constraint is imposed by the topology of the computer architecture , i.e. , the set Ec of processor interconnections . Another limiting factor may be the load of the mapping , which is the maximum number of tasks that may be mapped to any single processor . Mapping may also take into consideration the congestion , which is the maximum number of arcs that are routed through any single interconnection link . We define the cost of the mapping ? as follows . Let us define the dilation ` -LRB- a -RRB- of an arc a 2 Ap to be the number of intermediate processors along the associated communication path in Gc , i.e. , ` -LRB- a -RRB- = k ? 1 if ? A -LRB- a -RRB- 2 Ekc . Then we define the cost of the mapping to be Q -LRB- ? -RRB- = Pa2Np ` -LRB- a -RRB- . A mapping ? is optimal if it minimizes Q -LRB- ? -RRB- . The problem of minimizing Q -LRB- ? -RRB- is NP-complete . Thus , heuristic techniques must be applied for finding suboptimal solutions . There are two basic approaches one can undertake when solving the mapping problem . In the direct approach a -LRB- sophisticated -RRB- mapping algorithm is constructed yielding results which are as good as possible . This may be a difficult task since one must consider as many characteristics of the program graphs and target architecture as possible . The indirect approach , however , fixes some -LRB- simple -RRB- initial mapping algorithm and then improves its results using heuristic techniques . Both approaches have been considered . In the first case a mapping algorithm from -LSB- 2 -RSB- was adopted and enhanced by three nontrivial improvements -LSB- 5 -RSB- . However , these improvements involve solving several NP-complete problems . In the second case several heuristic algorithms were applied and considerably improved the initial mappings -LSB- 3,4 -RSB- . Next , we will consider the parallelization of one of these indirect improvements . 2 Suboptimal mapping with parallel SA Let Gp and Gc be a program graph and a r-regular processor array , respectively . The configuration space M of Gp is the set of all possible mappings of Gp onto Gc , i.e. , M = fM jM = ? -LRB- Gp -RRB- for some mapping ? : Gp ?! Gcg . If M 2 M for some ? , then the cost of the configuration M is defined to be equal to Q -LRB- ? -RRB- and is denoted by Q -LRB- M -RRB- . Furthermore , we say that a transformation ffl : M 7 ! M 2 M is elementary if it moves a single node v to one of r neighboring processors and reroutes all incident paths , i.e. , if there is some v 2 Np such that ? N -LRB- v -RRB- < > ? 0N -LRB- v -RRB- while ? N -LRB- u -RRB- = ? 0N -LRB- u -RRB- for all u ; u < > v . In this case the shifted node v will also be called ffl-node . The parallel SA algorithm will make use of joint transformations which are defined as follows . Given some configuration M 2 M , a joint transformation is a set fffl1 ; ffl2 ; : : : ; fflk g of elementary transformations of M whose order of execution is not important , i.e. , fflk ffi fflk ? 1 ffi : : : ffi ffl1 -LRB- M -RRB- is the same for any permutation ikik ? 1 : : : i1 of indices 1 ; 2 ; : : : ; k. Therefore , a joint transformation can be executed in parallel . Moreover , the cost variation ? Q = Q -LRB- -LRB- M -RRB- -RRB- ? Q -LRB- M -RRB- after the application of a joint transformation = fffl1 ; ffl2 ; : : : ; fflkg can easily be determined since ? Q = P 1 < = i < = k ? ffli Q , where ? ffli Q = Q -LRB- ffli -LRB- M -RRB- -RRB- ? Q -LRB- M -RRB- . Joint transformations can be constructed by the following simple algorithm JT ? . ffl-Nodes : = ; ; Destinations : = ; repeat choose u in Gp ffl-Nodes : = ffl-Nodes -LSB- fug pick a free q in Gc such that q = 2 Destinations Destinations : = Destinations -LSB- fqg Gp : = Gp ? -LRB- fug -LSB- fneighbors of u in Gpg -RRB- until Gp = Algorithm JT ? returns a set ffl-Nodes of nodes that can be moved by a joint transformation , and a set Destinations , which is the set of corresponding processors , where the nodes will be moved . Hence , the constructed joint transformation can be described as a set of pairs -LRB- u ; q -RRB- , with u 2 ffl-Nodes and q 2 Destinations . This transformation can be performed in parallel using k = jffl-Nodesj processing units , where each of the units takes one ffl-node u , performs the elementary transformation ffl represented by -LRB- u ; q -RRB- , and computes the corresponding ? fflQ . Note , that a processor is free if it does not host any of the nodes of Gp , or if it hosts an ffl-node . Joint transformations are constructed with Algorithm JT ? and used in the following parallel simulated annealing algorithm PSA . let M = ? -LRB- Gp -RRB- be initial configuration choose an initial temperature T > repeat construct joint transformation = fffl1 ; ffl2 ; : : : ; fflk g applying Algorithm JT ? Mold : = M forall ffli 2 in parallel ? ffli Q : = Q -LRB- ffli -LRB- M -RRB- -RRB- ? Q -LRB- M -RRB- if ? ffli Q < then apply ffli on M else if random -LRB- 0 ; 1 -RRB- < = e ?? ffliQ = T then apply ffli on M else : = ? ffflig endforall ? Q : = P ffl2 ? fflQ if random -LRB- 0 ; 1 -RRB- > e ?? Q = T then M : = Mold if a long time no decrease in Q or too many iterations then lower T until some time no change in Q After the construction of a candidate joint transformation , its elementary transformations are probabilistically accepted in parallel using Metropolis test . The resulting joint transformation is then probabilistically accepted with a global acceptance test . Performing global tests results in the improved convergence of the algorithm as it was shown in -LSB- 1 -RSB- . We use the following annealing schedule . Initially , the temperature T is set to aQ -LRB- ? -LRB- Gp -RRB- -RRB- , where a > is such that it guarantees high probability of accepting elementary transformations with ? fflQ > 0 . At each T changes are attempted until cjNpj acceptances occur , or djNpj attempts are performed , where < c < = d . After that , the temperature is lowered to bT , < b < 1 . To conclude , Algorithm PSA generates , tests and accepts many elementary transformations in parallel and is , therefore , a member of the class of \ error '' algorithms . It synchronizes after each joint transformation consisting of independent elementary transformations . Since there is no error accumulated it can also be termed as \ zeroerror '' algorithm . 3 Performance analysis The convergence of the Algorithm PSA depends on the degree of its parallelism -LRB- and , of course , on the annealing schedule -RRB- . The only parallelism in the algorithm is available in the forall : : : endforall loop , so one immediately observes that the number of elementary transformations in each is crucial . Let us concentrate on Algorithm JT ? since it is the task of this algorithm to construct joint transformation . The number of ffl-nodes is the same as the number of iterations of the repeat : : : until loop because , in each iteration , a node u is chosen in the current Gp and included in the set ffl-Nodes . Next , the node u and all its neighbors are excluded from current Gp . Since there are at most r neighbors , we find that d jNpj r +1 e < = jffl-Nodesj : Note , however , that this lower bound can be improved . Let deg -LRB- Gp -RRB- be the degree of the program graph Gp , i.e. , the maximal degree of a node in Gp . Then an improved lower bound on the number of ffl-nodes is given by the relation d jNpj deg -LRB- Gp -RRB- +1 e < = jffl-Nodesj : Generally , this lower bound is not tight -LRB- see example below -RRB- . For example , the number of ffl-nodes may be increased by choosing nodes with low degree first . 4 Conclusion In this paper we presented a parallel simulated annealing algorithm PSA for solving the problem of mapping parallel algorithms onto processor arrays with regular topol - ogy . The algorithm PSA takes an initial mapping and constructs a sequence of joint transformations to eventually produce an optimized mapping . Each joint transformation is a set of elementary transformations that can be applied in parallel to a given configuration . Hence , algorithm PSA is a member of the class of \ error '' parallel SA algorithms . Moreover , these elementary transformations are independent in the sense that they change independent parts of a given configuration . Hence , the erroneous assumption that there are no other simultaneously performed elementary transformations has no impact on computing the total cost variation of a joint transformation . Since there is no error caused by this assumption , our PSA algorithm can be termed \ zero-error '' parallel simulated annealing algorithm . References -LSB- 1 -RSB- N. Boissin and J.-L . Lutton . A parallel simulated annealing algorithm . Parallel Computing , 19 -LRB- 8 -RRB- :859 -872 , August 1993 . -LSB- 2 -RSB- I. Koren , B. Mendelson , I. Peled and G. M. Silberman . A data-driven VLSI array for arbitrary algorithms . IEEE Computer , 21 -LRB- 1 -RRB- :30 -43 , October 1988 . -LSB- 3 -RSB- B. Robi < = c , P. Kolbezen and J. < = Silc . Area optimization of dataflow-graph mappings . Parallel Computing , 18 -LRB- 3 -RRB- :297 -311 , March 1992 . -LSB- 4 -RSB- B. Robi < = c and J. < = Silc . High-performance computing on a honeycomb architecture . Lecture Notes in Computer Science 734 , pp . 1-13 , Springer-Verlag , 1993 . -LSB- 5 -RSB- B. Robi < = c. Optimization of parallel algorithm mappings onto regular processor arrays . PhD Thesis , University of Ljubljana , 1993 .

O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O

optimization
parallel simulated annealing