ing labels is a straightforward use of dot index of and dot substring in the private method process word we use these methods to ensure that punctuation or letters before and after the angle brackets are preserved printing the story will display the final result in the console window of blue jay or a different programming environment the private method print out has a parameter to specify the line width so you can create a story and use 80 or 40 characters or any other number you could modify the print out method to write the story to a file to using the edu dot duke dot file resource class changing the gladlib dot java program requires understanding how the ArrayList instance variables are used there is one ArrayList for each possible label in a story template like noun or color and instance variables should be named appropriately so that programmers will be able to easily understand their use in reading and modifying code as with all instance variables or fields they'll be created and initialized when the gladlib constructor is called either using new or from within blue jay when you create an object the program could use all the fields when replacing words as part of telling a story the field adjective list will hold replacements for the label adjective the field noun list will be for nouns and the instance variable color list will hold colors to be chosen at random each field holds replacements for the label that's part of its name this is a convention in the program not required by java but following the convention will make it simpler to create a new instance variable like verb list for a new label we'll look at each use of these fields one is finding a substitute for a label like color based on the word that's part of a label like color or noun the private method get substitute will access the appropriate instance variable to find a random replacement for example if the label is color then a replacement will be chosen from the field color list if the label is noun then noun list is used you can see in the method get substitute that adding a new label will require adding a new if statement to access the appropriate array list a value is chosen at random using a private method random from both random from and get substitute are private they're called as a result of calling the public make story method when get substitute calls random from get substitute will always pass one of the instance variables you made such as adjective list noun list etc as the value for the parameter source initializing the array list is straightforward but you'll need to understand it to create a new field for a new label all the array list must be created and initialized when the constructor is called the constructor will call a private helper method initialize from the source for the data for colors nouns and so on could be a url or a file calling initialize from will result in reading files or urls to store strings in each array list if the parameter to initialize from begins with http then eventually a url resource object will be used to read data otherwise a file resource object will be used you can't see that in the code here because the parameter source is simply passed to the helper method read it whether reading code is located let's summarize how the instance variables for replacing labels are used you'll need to understand this to enhance the program by adding a new label for example to create a label like verb you'll need a new instance variable you'll name it appropriately like verb list you'll need to modify code in two methods with the addition of verb list you'll modify the method initialize from a private method called from the constructor you'll modify the code in the method get substitute also a private method called by the public story via the private methods from template and process word the program documentation should include information like this to help you the software engineer make modifications and enhancements have fun telling stories and writing code we'll outline some of the design features of the gladlib class and talk about software design in general adding a new label like verb means modifying gladlib.java implementation in several different places and requires following a naming convention used in the class you'll need to create a new instance variable for the array list that stores examples of verbs like run think or swim you'll need to initialize the array list using the method initialize from called from the constructor you'll need to get a random verb when needed by modifying the code in the method get substitute you should follow the naming conventions used in the class where the label noun is associated with the instance variable noun list just as the label country is associated with country list this means you should use the name verb list for a field that's the array list that stores strings that are verbs as you modify and extend programs and classes you'll gain experience with many kinds of programming and design you'll gain experience that will help you make good decisions when programming and create good designs but you'll see that sometimes experience comes from bad judgment or bad designs that nevertheless allow you to reason about the tradeoffs in doing things in more than one way so you might realize that a choice that works can lead to code that's not easy to maintain some software designs are called brittle meaning that the software or design breaks when you try to extend it or use it in ways that is a little bit different than what's initially intended flexible designs on the other hand are able are better able to cope with changes in the software when you learn about object oriented design you might come across a principle that that says code should be open for extension but closed for modification the open close principle the idea is that you should be able to extend software without extensive modifications to the existing code that's more possible with object oriented design ideas that we won't be able to cover in this course but you can still create designs that are more open than others you'll be able to understand a better design after working with this design and implementation the glad lib class does have some good features it's relatively easy to understand each method and the code works it is possible to extend the code as you'll see even if the extension requires changing the code in several places you'll be able to create a better design after learning some new Java concepts will introduce soon but those concepts will be more clear because you'll have this experience with this code in class and you'll understand why the new Java features can make the code simpler the new Java features will let you keep changes in one place rather than being sprinkled across three parts of the class the new features will also minimize the duplicated code that's in the current implementation have fun making new stories hi we're going to walk through the process of adding a new label to the glad lib.java program glad lib class a new label might be something like verb which would go along with noun, color and the other labels we have to do that you've seen part of that process outlined in one of the previous lessons what I'm going to do here is show you that to add a new label we need a new template file so I've taken the standard template file that we had before which was called madtemplate.txt and I've replaced sing and dance the two things that a creature would do in the stories we generated with verb and verb so I'm adding the label verb to replace sing and dance in the original story I've still called this madtemplate.txt you'll use whatever text editing software you have which is usually something like text edit on a Mac or notepad plus plus on a Windows machine text editor now I'm going to take the glad lib program that we've used before and I'm going to walk through the locations where we need to make changes I need to create a new array list to store my verbs I'm going to go along with the same conventions that have been used before and make sure that I use verb list to go along with adjective list, noun list, etc as I scroll through my source code I see that these array lists are initialized from source method it's called from our classes constructor I need to initialize verb list and it's going to be initialized by calling readit passing the source which is either a place on my computer where it can access files or a URL this is verb.txt I'm going to also have to make one other addition because now my label might be an adjective or an animal or a name but I'm going to need to copy and paste these so that in addition to using time frame I'm going to use verb so I'll replace that make sure I get the syntax right if the label is verb I'm going to replace the label from verb list I'm going to compile my program I know that in this program it's reading the template from madtemplate.txt which I've modified to include the angle bracket verb angle bracket label I'm going to create a new gladlib on my object workbench and then I'm going to run the make story method it's so love to unknown and unknown there was a funny slippery line named Vivian that scared it so much so if we go back and look at our source code we'll see that it must have found verb but failed to replace the verb with the objects that it was replaced with in this case I've got verb list here my instance variable I've initialized it from verb here I've made sure that if oh look if label dot equals verb t I had a misprint when I spelled it now it's got verb so I'm going to go back and recreate that create my gladlib object open it up and make a story hey it's so love to think and contemplate but there was a gigantic gigantic tiger let me run that one more time and see if I get something besides think and contemplate it's so love to ride and surrender one more time and I'll be convinced that it's actually reading verbs it's so loved to surrender and surrender we've done nothing to make sure that it can use the same word or verb more than once so just to remind you what we've done I've used this location to change when the label was found I've made sure that the label was read from the source and I used verb list here which is the name of my instance variable here verb list using the same naming conventions we've used in all the other programs I compiled my program I tested it I created a new text file to hold verbs and a new text file for the template happy verbing hi it's time to learn about a new way of structuring data this also helps in structuring classes you've seen some of the concerns with the design of the gladlib class the design has flaws but you still were able to add a new label and create new stories so the design works larger programs with the same flaws might be harder to modify we've also seen code to count nucleotides in a strand of DNA and letter frequencies to break a Caesar cipher we extended this idea to counting word frequencies in any file with two parallel arrays to count nucleotides we used four int variables to count each time a c, g, t, or a occurred to count letter frequencies we used an array of 26 int values one for each letter a through z to count words you saw code that used two parallel array lists these contained as many values as there are different words being counted now you'll see how to use the concept of parallel array list to help understand the java.util. hash map class using a hash map will make it much easier to modify the gladlib class the hash map class is also very efficient compared to using two parallel array list we'll explain the hash map class using the parallel array list class to help you've had a chance to see this code that uses two array lists to count how many times each different word occurs in a file the method.index of finds the location of a word in the array list of strings named my words if the value returned by the .index of is minus 1 then the string read from the file hasn't been seen before and it's stored in my words with a corresponding count of 1 in my freaks if the string has been seen before the integer value of my freaks at the same index is incremented by 1 this code works but using a hash map will make the code much faster and the hash map class will make it easier to modify the gladlib class as well. let's work to understand the concepts in the hash map class a hash map object associates keys with values in many languages this is called a map you might think of the key or legend in a map as you can see here the key helps in understanding the map you can look up the color in the key or legend and understand what the color means in the map in programming the concept is more graphical than geographical the word map has meanings in both math and geography the key is an element in a domain that's being mapped to a value in the range in math a function is sometimes called a mapping and this expresses the ideas in the hash map class as well in a hash map you look up a key and you get the value associated with the key in an example illustrated here we are counting word frequencies the word rainbow occurs 41 times so the integer value 41 is associated with the string key rainbow in java the value returned by the call map.getRainbow is 41 the value returned by map.getTruth is 17 indicating the string truth occurs 17 times if football occurs 23 times then 23 is the value associated with the key football and is returned by the call map.getFootball wonderful occurs 23 times in our hypothetical example the keys in a map are unique but it's possible to have the same value associated with different keys to use a hash map you'll need to understand the operations it supports we'll use one hash map to replace the two parallel array list here's the code that uses one array list of strings and one array list of integers with the dot index of function used for associating integer values with strings the hash map code also associates an integer value with the string key the key in the hash map will be a string the value is an integer that's the number of times the string occurs in a file being processed to define a hash map variable you'll need to specify the type of both the key and the value when calling new you must specify the key and the value types as well the key is the first type and the value is the second type the code will determine if the key has never been seen before whether it's stored in the hash map or not the method dot contains key on the map object returns a bullion indicating whether that key is in the map if it's not in the map the value one is put in the map with the key using the map method dot put if the string is a key in the map meaning the string being processed has been seen before the value associated with the key is found using the dot get method the value is stored for the key by calling map dot put with an updated value of one more in addition to accessing individual map keys and values you'll also need to access all the keys and values in a map printing the strings and frequencies when parallel arrays are used requires a typical indexing for loop with the index used to access both strings and integers as shown here printing all the key value pairs in a map requires looping over all the keys and getting the value associated with each key the method dot keyset returns an interval you use to access each key in a loop this is similar to using dot words or dot lines with a file resource or url resource to access elements or the dot data method to access each string in a storage resource iterating over the keyset and calling get for each key allows you to print the contents of a map maps will allow you to modify the gladlib class more easily but maps are incredibly fast too when files are large efficiency matters more than when they're small this concept of using efficient structures or code is important computers are so fast that simple concepts lead to code that's fast enough given how fast computers are today as you can see in the first row of the table counting how many times each word occurs in Shakespeare's Julia Caesar is fast enough on a laptop computer even using the parallel array list with a bigger file like the sayings of Confucius the code for array list is still under half a second whereas the hash map code is incredibly fast for the novel the scarlet letter there are many more unique words and total words but the hash map code is still roughly 10 times faster than the array list code for a large file like the king james version of the bible with over 800,000 words and 32,000 different words the array list code takes more than 20 seconds while the hash map code is under half a second that's more than 40 times faster looking up keys in a map takes time that's independent of the number of keys roughly speaking this means getting the value for a key in a map with a million keys takes the same time as looking up the value associated with the key in a map of just 10 keys with an array list you might have to look at all million elements in the array list so hash map is incredibly fast in later courses you'll study how the hash map class can be so fast in this course you'll use hash maps and many examples happy coding hi today we're going to write some code to find out how many times every word in a file occurs so we'll find how many times the word the occurs how many times the word wonderful occurs and we're going to start with a working program let me show you very quickly what that looks like this count words function that we have here this method count words simply creates a file resource loops over all the words in it and counts how many total words there are if I take this program make an object on my workbench right click and call count words it will ask me for to enter the name of a file I will use confucius and it will tell me that there are 34,582 words in the works of confucius rather than knowing the total number of words I'd like to find out how many times each individual word occurs to do that I'm going to use a map so I'm going to need to add another local variable here a hash map I'm going to map strings to integers each string will be a word that occurs in my file that's the key in this map and the value in the map is how many times that word occurs so I need to create a new one I'm allowed to make a hash map because I've imported java.util that's the package in which we find hash map now as I read the words rather than incrementing the total I'm going to ask have we seen w in the map before so I'm going to ask whether the key set associated with the map contains w the word I'm looking for and if it does I've seen the word before I'm going to put the value w back in the map it's already there I'm going to get the number of times that word occurs and add once I'll go over again what I'm doing in a minute if I've never seen the word before then what I'm going to do is put it in my map with the number one because the word will have occurred one time let me compile and make sure I've got that right so let me go over very quickly what I've just done I've looked to see does this word w that I just read and converted to lowercase is it in my key set have I ever seen it before if I have seen it before let me get the integer value associated with it that's the number of times it's already occurred add one to that and put that in the map as a key value pair with the word and an incremented count if I've never seen it before this is the first time I've now added all the values to the map and I'd like to print them out the way I'm going to print them out is to loop over the key set that's all the words that are keys in my map and I'm going to get the value associated with it that's the number of occurrences that's the value associated with my word and if that value is big and here I'm going to say if it's bigger than 500 I'm going to print the words that occur a lot which is, so I'm going to print the number of occurrences a tap character and the word itself so I'm looping over the key set that allows me to find every key I'm getting the value associated with that key in the map and if that's big I'm going to print it big in this case is bigger than 500 so I'm going to right click and create a new object and I'm going to invoke the method count words which before used to count the total number of words and now it's going to count all the individual words in Confucius and here we can see that in the file Confucius there are not that many words that occur more than 500 times the occurs more than 2,000 times and occurs 762 if I want to see a little more in the way of words I'll say 200 just to make sure that I'm getting something more and I'll make a new version of this that sits on the object workbench I'll right click and call count words I'll once again count Confucius and you can see there are more words master occurs 484 times Confucius was the master of many things you are now going to be the master of programming as you use maps to map keys to values to solve many problems have fun coding in this lesson we'll look at how we can use the hash map class to make our glad lib class easier to extend have fewer lines of code and be a good example of how to become a more skilled and experienced software designer as a reminder extending the class to use a new label like angle bracket verb angle bracket requires modifying the code in three places you'll need to create an array list instance variable initialize it properly and use it as the source for your random replacements you should also follow a convention of using field names like verb list for the label verb this makes it difficult to use text files or urls for the source of word replacements unless all such sources follow the same conventions such as using a file name noun.txt for the label noun or the field noun list or color.txt for color list and the field color let's take a look at the concepts behind these requirements for extending the glad lib class to look at a new way of structuring data in classes each label is associated with an array list instance variable you see the label noun associated with noun list the label color with color list and so on these named instance variables make for a poor design adding a new label like verb requires defining an instance variable by name initializing it by name and using it by name that's three places in which the program must be modified instead we'll use a hash map to help create a better and more flexible design the hash map will allow us to label or align the label to an array list without ever having to name the array list itself given a label the code will look up or find the associated array list in the hash map structure as you've seen this is similar to how index of works for finding a value in an array list or a character in a string getting the value associated with a label will return an array list let's take a closer look at how using a hash map creates a more flexible design one hash map will replace seven or more instance variables the hash map will reference as many array lists as needed rather than us having to define separate instance variables and following a naming convention as you see here the code will use a single instance variable a hash map named my map this will associate an array list with each label so the keys in the map are strings the label in the original program the value associated with each key is the array list of replacement words for that label this means that to add a new label and a new array list we don't have to add a new instance variable we simply need to store new values in the single hash map instance named my map let's look at how the method get substitute works with a hash map in the original program a list of if statements was used to identify the instance variable associated with a particular label the naming convention of using country list for country allows a programmer to extend the code but there will always be as many if statements in the get substitute method as there are labels and instance variables the last if statement is different you can see that the label angle bracket number generates a random number instead of finding one in a list of numbers when using a hash map the get substitute method is much more simple the hash map associates a label with the array list of replacements the array list for a label is accessed using the hash map dot get method to get the array list associated with a string label like country or noun or color adding a new label doesn't require modifying this method at all and that's an example of the open close principle that we talked about in a previous video using a hash map makes for a more flexible class that's easier to extend but there's room for even more improvements using hash maps again the original program reads a file or URL to store information in each named instance variable the array list of replacement values for that label that was done in a sequence of statements that call the helper method read it the hash map version still associates a label with a file name and that file name must be specified in the program but the code is different because we use a loop to associate each label with a file name this wasn't possible in the original program note the private method help the private helper method read it is still called what changes if we want to add a new label like verb the program will still associate the name of the file of replacement values in verb.text with the new label we could store a new string like quote verb quote in the local string array of variable labels we could add that just after the string timeframe for example unfortunately we still have the limitation in that the code uses a naming convention for files like verb.text for the label verb we could use a hash map in a different way to associate file names with labels without modifying the program the program could be designed to read a file of information that specifies where to find the words to replace the labels rather than requiring the code to be modified compiled, tested and run to simply find nouns in a different file or website this kind of file is often called a dot properties or property file as shown here it simply associates a label with a source of replacements for that label hello let's summarize what you've had the opportunity to learn studying our gladlib example of creative storytelling the gladlib.java program read templates from files or URLs to create engaging stories about any topic you chose we use the program to motivate the need for and study of the ArrayList class ArrayLists are like arrays in that they support indexing to individual elements but an ArrayList object can grow as needed rather than being a fixed size we also use the program to motivate the study of the hash map class hash maps are very efficient structures that can be used to associate keys with values you saw two examples of this in the study of the gladlib class we also use the gladlib class as a small case study to understand that creating code and programs that are extensible is a good idea but requires thinking planning and experience we use the gladlib class to study the ArrayList class an ArrayList object is an indexable collection of elements ArrayLists store primitive types so you can store integer objects but not into values this means you often need to update ArrayList integer values in two steps first getting the value and updating it and then putting that value back in the map to use the ArrayList class you must import it from the java.util package in contrast you don't need to specify a package for arrays useful methods you've seen include add to add a new element to the end of the ArrayList size to determine the number of elements in an ArrayList by its index set to update an element using its index an index of to determine where an element is stored in an ArrayList by index you can write code to loop over all elements in an ArrayList by either using an ArrayList object as an iterable or using an int for loop starting at zero and looping up to but not including the size of the array to access each element by its index we also studied the HashMap class a HashMap object is a collection of key value pairs the keys serve as mappings for accessing the values hence the name HashMap both keys and values are objects so you would use integer rather than int just as you did for the ArrayList keys are best as immutable objects like string or integer keys must be unique values can be any object type including string or ArrayList as in the examples you saw you import HashMap from java.util just as you do for the ArrayList class in the examples you saw the methods put to add a key value pair to the map size to determine the number of pairs or keys in a map get to access a value by its key key set to iterate over all the elements and contains key to determine if a key is in the map looping over all elements requires an iterable over the key set of the map you can't access individual elements one at a time by an index as you can with the ArrayList two different collections with two different sets of strings we hope you'll have fun using them now you're going to write some programs to analyze web server log files most major web servers log each access to a file which records who made the request when the request was made what the request was and how the server responded so why would you want to analyze web server logs? a web server's log file lets you understand a lot about how your website is being used you might want to know how many people are visiting your site is it popular or not if you have many different pages including traffic or only a few understanding how your site is being used is particularly important if you are trying to make money off of it popular pages bring in revenue while pages that nobody looks at are not helping your business the log file can also be useful in diagnosing problems as it will tell you when your server is experiencing errors if one of the pages isn't getting traffic because a link to it is broken you want to know so you can fix it for the rest of this lesson you're going to work on code to read the contents of a log file being able to read the contents of a log file will set you up to solve a variety of problems such as figuring out how many different visitors have come to a website or how many times each visitor has visited the site welcome back now that you know about the importance of web server logs it's time for you to start thinking about writing code that deals with them the first thing you're going to want to do is to be able to read in a web server log file and represent the information in it in java objects to do this you need to think about two things the first is what does all of the information in the web server log mean and the second is how do you represent it in a java class here you can see one entry from a web server log file it has a lot of information and it's not readily apparent what each piece means you would want to read documentation about this web server this particular data came from the web server log for an apache 2.4 web server so to find out more you could go to google and search for apache 2.4 web server log file format if you do that you'll end up with a lot of hits and this first one here gives you a link to the apache documentation site if you scroll down a bit you'll find the information on the access log that is the log of accesses to the web server which is what this web log is and you'll see that it has information about each of the pieces of this entry the first is the IP address that is the address of the device on the internet which made the web request that is logged here the next two pieces are both dashes that indicates that they're missing information the first dash is for some information about who made this request which you'll see the documentation says is unreliable the user's computer could lie about who they are the second dash is for the username if they're logged in with HTTP authentication if they've typed a username and password on the website the next piece of information is the date and time when the request was made next you have the request itself including what type of request it was in this case get where they're asking for a particular web page and then what page they asked for next is the status here it says 200 which indicates success there are many other statuses which indicates success or failures you may be familiar with 404 which is the very well known status code that indicates that the requested page was not found finally is the number of bytes that the server replied with how much data it sent back to fulfill this request okay now that you've read the documentation and understand what each of these pieces of information mean it's time to think about how to represent them in a Java class the first thing you need to think about is what type of information each of these is for the IP address you could use a string since you're interested in just the text of that field Java does have a built in class for IP addresses which will give us some more features if we wanted to actually connect to that address for example but we don't need that functionality and we don't need to worry about the complexity that would introduce right now we don't need to represent the two fields that we've omitted that have no useful information we do however want to represent the date you could use a string for that in which case you would just have its text or you could use the built in Java date class which understands what dates and times are and how they relate to each other so you could check if one time is before or after another time for the request you can just use a string and for the status and number of bytes those are both numbers so you can use an end now that we've thought through these types it's time to turn this into some Java code you can see the start of a Java class for a log entry we've declared a public class log entry and written fields based on the types that we just discussed you should now think should each of these fields be public or private remember that if a field is public any piece of code can access it and if a field is private only code within this particular class can access it in this particular case it makes sense to have each of these be private and to design your class to be immutable remember from earlier when we learned about strings that immutable means you cannot modify an object once you create it so you're going to write this class so that each of these fields will be set in its constructor but can only be read to make anything able to read these fields you'll write a public getter or accessor method such as these which will just return the value of that field but there will be no way to set the value of the field once it's constructed speaking of constructing you need to write a constructor for this class there are two ways you could do it the first is to take in this entire string and then I have the constructor pull it apart into each of these individual pieces and fill in the fields or instance variables of the class the other is to have the constructor take each piece of information separately and simply initialize the fields of the object we're going to have you do the second one just making a constructor that looks like this which is going to fill in the fields based on the information passed in with each piece being passed separately why would you do it this way well this gives you a little more flexibility if you wanted to create one of these objects with other sources of information you could do so it turns out that pulling this line apart is actually a little bit tricky so we're going to give you the code for that it's a little bit ugly and we'll package it up into a nice method for you so you can just use it to read the file here is the final log entry class with all of the things you just learned about you would use this to represent one of these log entries as you work with it it's going to be to use this class the code we give you to split this up into each separate piece and put them together to make code that's going to read the entire log file so today what we're going to do is we have this log entry class and I want to show you two string so the log entry class notice it has these five fields here IP address and so on we've got our constructor and we've got all these methods written two string down here and you can see that it returns those five fields as one big long string and what I've done is I've got a tester over here that is creating two log entries LE and LE2 I've just kind of made up some information for each of them so they're different and then it prints out the actual object it prints out LE and it prints out LE2 so let's run it and see what happens so I'm going to come over here notice I need to compile both of these I'm going to compile tester and it actually compiles both of them for me I'm going to now run tester test log entry and there you can see it printed out the five pieces of information for each log entry, very nice so now what I'm going to do is I'm going to come back to my class and instead of calling it two string I'm going to change the name I'm going to call it get log info and compile again and run it and let's see what happens so I've got my object and now I'm running test log entry oh so I ran it but it just printed out something else let's see what it printed it actually printed out the memory location of each of those objects so let's go back to test our tester here and you can see when I print out an object it doesn't know how to print it out so it just prints out it's address location it's not calling the method we wrote which is called let's see here right here we called it get log info it's not calling that because we didn't specify it so let's come back over here and we'll specify to print out a log entry with that method we need to actually write log entry dot get log info and I'll just do that for the first object and I'll leave the other one le2 like that and let's see what happens we'll compile it and we'll come over here and run tester and so you can see what here for the first object I called get log info and it prints calls get log info and I'll specify pieces of information but for the second I just said print the object and so it just shows the memory location of it so what's going on here it turns out that every class has a two string method by default but it only knows to print out the memory address of an object unless you actually specify a two string class so I'm going to change this name back to tester and get rid of this because notice I didn't actually call two string here I just said print out the object and again if I show you what happens here you don't need the there we go so we're just going to print out the object we're not saying how or anything but it just knows go look in my class and if there's a two string method that's how I'm specifying how I want to print it out so again we call it here I didn't call two string it knows it says oh you've got a two string method so I'll use yours and it prints out the five pieces of information the way I specified I wanted the object printed out I want to show you one more thing I'm going to come over here that name two string is very important it has to be spelled exactly two string with a capital S because I'm going to show you if you change it to lowercase S so it says two string then when I come over here and I run it let's see what happens it goes back and writes the memory address because it says if there's a two string in there spelled with a capital S it's going to use that but that's the only thing it looks for since I don't have it spelled correctly it doesn't find it and so it says I'll just print out the memory address location so just remember that all objects have a default two string method it's just going to print out the object address then you can write your own and so when it runs it'll see if there is a two string method in there then it will call it that's it, that's for two string, thanks Hi, now that you've made your log entry class you need to parse the lines of the web server log to be able to create instances of the log entry class you'll do this by splitting the string into the appropriate fields to pass values to the constructor for the log entry class you can parse this task with many index of and substring calls although this task is not algorithmically hard the code for it's very cumbersome for example for the time portion of the entry you would need to turn the string into a date object the built-in Java class from the java.util package which represents a date and time even though both the date and time class as well as methods which parse strings are part of Java the use of the date class is complex especially since the date format in the server logs is not the default format in Java for these reasons we've provided code for you which will take a string from the web server logs parse it into appropriate fields and return a log entry record to use this call weblogParser.ParseEntry and pass the string you want to parse the method returns a log entry object with that in mind it's time to turn your attention to starting to write the log analyzer class for right now you're going to write code in the constructor to initialize the object and then write the read file method in later lessons you'll write additional methods that will perform the actual analysis of the log file that you've read in the first thing you would do to fill in the code for the constructor the constructor should initialize the record fields to an empty array list you've created array lists in the past so what you need to accomplish this task should be familiar the second thing you should do is fill in code for the read file method this method will determine the file name to read from and then add log entries to the records field to reflect the information from the file you opened to accomplish this task you'll want to make a file resource for the requested file you will then want to iterate over the file resources lines and for each line you will use the WeblogParser.ParseEntry method to convert the line of text into a log entry then you'll add that log entry to the records field which as you may recall is an array list when you've written the constructor and the read file method you'll want to test out your code we've provided a convenient method called print all log entries you've stored in the instance variable records remember the two string method that we taught about system.out.printlin will make use of that two string method to represent the log entry as a string once all this works it will be time to start analyzing the data you've read in happy coding in this lesson you learned a bit about web server logs which can give you a lot of information about your website you learned what is in them as the records are formatted as we looked at the Apache web server documentation together and then you learned how to make a log entry class based on the information in that file as part of that you learned about two string which is an important Java concept as you will want to write a two string method in many of the classes you create finally you wrote code to read in the log file using the parsing code that we provided now you are ready to write some code to analyze the data you've read in welcome back you have already written code to read a web server log file parsing each line into a log entry object and creating an array list of them now it is time for you to write some code to analyze the data you have read the first problem you're going to solve is finding out how many different people visited a website you don't want to just look at how many elements there are in your array list since some people may have visited your website multiple times so you need some way to distinguish requests from different places you can use the IP addresses recorded in the log file to tell where the request came from using the IP address is not perfect since you cannot distinguish between different people using the same computer but how many different IP addresses you see is a very good estimator for how many different people visited the site as you might recall from programming and the web for beginners an IP address is the address of a device on the internet whether that device is a traditional computer a mobile phone or something else so what you are going to need to do to solve this problem is take the array list that you have read the log entries into and find out how many distinct IP addresses are in it have fun! you have already written code to read in the entire contents of a web server log and now you want to find out how many unique IP addresses it contains as always you are going to want to approach this problem with the 7 steps you use for every problem we are going to walk through this problem starting at step 1 but we are going to use color names instead of IP addresses this problem is really the same problem there are unique values there are in an array list of strings but color names are easier to say and to see which ones we are talking about so when we look at this list of 10 values you might be able to tell just by looking at it that there are only 4 unique values but if this list had a million elements in it we would like to develop a method that would still work there are many ways of solving problems in general and in this case there are a couple that I will mention and I will summarize in code one method that we won't do is to as we look down the list cross out values that we have seen before crossing out values in a list is a problem in Java programming because if we replace the values in our parameter with other values we have caused a side effect something we have spoken about before as a thing to avoid if you can so instead we will use an idea of visiting every value in turn which is typical for array problems and if we haven't seen a value before we will copy it into a new list so first I am copying the value pink because I haven't seen it before in my new list then I am going to visit green I haven't seen it before so I will copy it over into my new list now I am going to visit pink I have already seen it so I don't copy it then green I have seen it so I don't copy it pink pink I have seen them all orange I haven't seen before so I will copy that into my new list the next value is blue I haven't seen so I am copying it into my list finally I get to pink the last value in my original list I have seen it before so I don't copy it over now I look at my list that I have created it has four values and so I am going to return the value four from my method that I am writing that determines the number of unique values in the array parameter developing this algorithm follows a lot of the same patterns you have seen before you will see that sometimes you add an element to the copy and sometimes you do not and you will want to think through the conditions under which you add once you have done that you can express the main portion of the algorithm in terms of steps to do for each element of the input you can work through steps two and three hopefully you came up with pseudocode that looks like this now for this particular problem you will want to do things just a little bit differently remember that you have log entries in the field records and you want to use the get IP address method to get the string out of the log entry object there are a couple of ways to deal with this difference the simplest is to just use the same algorithm just slightly to reflect the fact that you want to use records the fields in the class and you want to get the IP address out of each element of records check to see if that IP address is in copy and if not add the IP address to copy at this point you would want to test out your pseudocode then you are ready to turn it into java code alright now you have devised the algorithm to count the unique IP addresses in a web server log so it is time to turn it into code as usual we have here the outline of this method with the pseudocode that you have just developed the first thing we want to do is make unique IPs which starts as an empty list so we are going to have an array list of strings and we are going to call it unique IPs and it is going to be a new empty array list now we want to do something for each element we are going to call le in our records so as you are familiar with by now this is a for each style for loop each of these is going to be a log entry and remember that records even though we don't have a records variable in here is an instance variable in our class so we are going to take each log entry in records and then get the IP address out of it and then we want to know if IP address is or is not in unique IPs so we are going to say if unique IPs .contains IP address but we want not that that is the opposite so we are going to put a not in front then we want to add IP address to unique IPs unique IPs .and IP address close that close that and then at the end here it says we want to return unique IPs .size now my braces have ended up in slightly weird places that is probably just because I have some braces in my comments so I am going to delete those and then try to make sure my code lines up nicely so now I am going to hit compile it says class compiled, no syntax errors of course we want to test this I have already written a tester here which is going to create a new log analyzer read in short test log which is this log file here it has this IP address this same IP address appears three times we have only seen two unique IP addresses we have a third and we have a fourth then it uses the log analyzers count unique IPs to count the unique IPs like we just did and then it prints out how many there are so I am going to go over here to blue j I am going to um make a new unique IP tester and I am going to run test unique IPs and it prints out our IPs which is the result that we expected so we are more confident that our code is correct now that you have written code to find the unique IP addresses in a web server log let us take a brief look at something very close that would not have worked here is the code that you just wrote for the unique IP addresses problem remember that you have an array list of strings and that you put the string for each log entry's IP address into that array list what if you had written this code instead this code is the same as what you wrote but the array list holds log entries and the code checks if it has the current log entry not the log entries IP address likewise it adds the entire log entry object if it was not already there if you were to run this code it would give you the wrong answer in fact it would tell you how many total log entries there are not how many unique IP addresses there are in the log file why is that to understand this problem think for a moment about how contains would work in particular how does contains know if two objects are the same or different contains is going to check if they are equal what exactly do we mean by equal java has two different notions of equality to illustrate this consider the situation in which you have three string variables A B and C which refer to two different string objects A and B refer to the exact same string object so they are clearly equal A and C however refer to different string objects with the same logical contents on the one hand you could say they are equal because they talk about strings that mean the same thing on the other you could say they are not because they are talking about different objects these are the two different notions of equality that exist in java the notion of equality meaning the exact same object is what you get when you write equals equals if you write A equals equals B then java checks if A and B refer to the exact same object since they do this expression evaluates to true however if you write A equals equals C then java again checks if A and C refer to the exact same object but because they do not this expression evaluates to false the other notion of equality do they mean the same thing is done with the dot equals method if you wrote A dot equals C then java would call the dot equals method in the string class which checks if the two strings have the same sequence of characters because these two strings have the same sequence of characters A dot equals C would evaluate to true so how does java know whether two objects have the same logical meaning each class defines dot equals to specify what it means for objects of that type to be the same if you do not write one explicitly the default behavior will be to have the dot equals method check if the two objects are equals equals to each other that is if they are the exact same object so now that you know about equals equals and dot equals should you write a dot equals method for log entry well the first thing you should do is think about when two log entries are logically the same what about if they have the same IP address well that would fix the broken code for this particular problem it's not a good approach in general it does not actually match with the notion of two log entries meaning the same thing two different requests are not the same even if they came from the same computer so what if you checked more information what if you checked for the same IP address and the same request string even this would not really mean the two log entries are the same as the same computer could ask for the same page many times for log entries it makes sense to just say that they are logically the same only if they are in fact the exact same object because the behavior you want is the default for dot equals you do not need to explicitly write it since you do not need to write a dot equals method for this class we're not going to delve into how to do it yet fully understanding what goes into a dot equals method requires a little bit of knowledge but you will not get until the principle of the software design course however now that you understand the ideas of equality and that the contains method checks if two objects are the same you can understand why this code did not work and why this code where you use the IP addresses did work thanks great now you have completed your first analysis of the web server log file data that you read you wrote code to figure out how many unique IP addresses there were which gives you a good idea of how many people visited your site doing this gave you some great practice with array lists which are useful for solving a wide variety of problems and you also learned about Java's two different kinds of equality equals equals which tests if its operands refer to exactly the same object and the dot equals method which checks if two objects have the same logical meaning each class can define this method however it needs to so that it can provide the appropriate definition of having the same logical meaning now it's time for you to solve a new problem with web server logs for this problem you'll be writing a program to determine how many times each user visited their website this information can be quite useful for understanding how people use your website do the same people visit your site repeatedly for example this person keeps coming back to the site again and again her IP appears many times in the log file seeing the same visitor repeatedly coming to your site might indicate that they find the website useful because they keep coming back by contrast this person visited the site once but never came back maybe it wasn't what he was looking for or maybe he found the site hard to use if most users only visit your site once it might suggest that there are some issues you would want to look into of course as with all problems the techniques you're going to learn are useful in a wide variety of other problems not just those dealing with web server log analysis so let's dive right in as always you're going to apply the seven steps to solve this problem as with the previous problem we'll observe that the problem is fundamentally the same no matter what strings you use and use strings that are easier to talk about than IP addresses we'll work through this using some animals we want to count how many times each animal name appears in this list cat snake t-rex snake cat now you start by working through this example in a step-by-step fashion examining each animal and keeping count of how many times you have seen its name when you have finished looking at each animal in the list you have your answer cat and snake both appear twice and t-rex appears once now that you have worked an example it is time to think about and write down exactly what you just did the first thing was to make an empty table where you could keep track of each name and how many times you had seen it saying you made a table is fine but it is also good to think about what this means in terms of actual data types you can use in your program what type have you seen that is useful for representing this kind of information yes a hash map that maps strings names to integers counts once you realize that this is a hash map you may as well call the columns by their technical names key and value finally you would want to give this hash map a name so that you can refer to it easily we will call it counts next you looked at the first string in the list cat then you looked for cat in counts and saw it wasn't there so you put it in with a value of one you did a similar step for the second string snake which was also not in counts and for the third string t-rex for the fourth string snake things were a little different when you look in counts you see snake is already there with a count of one so you update the count to be two and similarly for the last string cat you find that it already has a count of one so you update it to have a count of two after all of that the entire hash map counts is your answer that leads to these steps for this particular instance of the problem so now it is time to find patterns and generalize to any instance of the problem notice that there are several steps that did not find the current string in the hash map in each of these cases you put it into the hash map with a value of one why one? do you always want one? or should you look for some other pattern? if you think about it for a moment you will realize that here you always want one it is the first occurrence of that name so you have seen it once there are also some cases where you already had that particular name in the hash map in these cases you want the old value to be two again you should ask yourself why did I use two? do I always want two or is there some other pattern? in this case you do not always want two instead you want the old value plus one that just happens to always be two here because of the specific example you worked with all of that in mind you should be able to generalize these steps and come up with an algorithm that looks like this hash map then iterates over each string in the input and checks if that string is already in the hash map if not the algorithm puts that string into the hash map with a value of one and if it is already there it updates the value to be one more than the old value after processing all the strings the answer is the hash map counts now you want to test this algorithm out test it on the input fish dog fish fish the algorithm got you the right answer so you can be more confident that it is correct before you turn this into code we are going to remember that our input is not actually a list of strings but a list of log entries whose IP addresses you want to process this slightly adjusted algorithm is basically the same but we have changed this variable name to le to stand for log entry and are iterating over the contents of the array list records in the log analyzer class then you want to get the IP address out of that log entry and use that as the string that you use to update the hash map now it is time to turn this into code we'd like to find out how many times somebody visits a website so we are going to look at a log file that has IP addresses and see for example for one IP address how many times did that IP address appear in the file that's how many times that person visited the website so we have a program here a class called log analyzer and we are going to write the method count visits per IP so how many times does each IP address visit a page and what we have in here is we have we're going to put log entries in an array list it's called records so we have a constructor that initializes that array list we then have read file which is going to have the file name of the log entries and that is going to allow you to select a log entry file and then it just goes through and reads through all the lines and puts them into records so we're going to focus on writing count visits per IP and we're going to use a hash map to do that so the first thing we're going to do is we need to make an empty hash map and we're going to be mapping a string to an integer so for each IP address which is a string we're going to map that to the account which is the number of times that IP address appears in the file so first let's make let's see a hash map of type string which is the type of the key and of type integer which is the type of the value for the hash map we need to give it a name so we're going to call it counts and then we have to create a new one so a new hash map of type string for the key and integer for the value there we go now that we have it we want to iterate over all the records that we have so we're going to use a for a for loop for each log entry we need a variable name so I'm just going to call that le we're going to iterate over the records where we put all the records from the file there's our for loop and now we're going to look at each log entry one at a time so first what we'll do from the log entry is we'll get the IP address okay so we need a variable for that and that is going to be of type string so we'll call that variable say IP which is of type string and we will use the log entry to get the IP address so if you remember we have get IP address and that should get us the IP address now that we have the IP address we have to see if it's already in our hash map or not so we'll have to ask a question about that so we're going to ask if that contains key okay so again counts is our hash map and we're using the contains key function and we will ask if that IP address is in there or not and I'm actually going to ask first if it's not in there so we'll put the not there and if it's not in there then that means we want to put it in there for the first time when we put it in there for the first time its count is going to be one so we'll add code for that so we get the name of the hash map which is counts and we'll use the put and we need to put in the IP address which is the variable IP and we put in one for the count one now if it's not in there then we'll want to sorry if it is in there then we'll know it's already in there we need to get the value out we need to add one to it and we need to put it back in so we'll do that now so essentially we're going to use counts.put but what we're going to put in there is we're going to put the IP address in there again or basically replace it with we have to get the old value out and add one to it so we'll have to use counts.get to get the old value of it and then we'll have to add one to that and then what else we have to do we come down here so if we get to the end of that and we've looked at all of that we put all of our log entries into our hash map and essentially they're each in there once with the count of how many times they appear in the file then we can return the answer which is just counts so we'll compile this and let's see we've got to get IP address correctly so we'll fix that and we've compiled it with no syntax errors now alright so now we want to test this out again let's just see count visits per IP is going to return the hash map that is all the mappings of the IP addresses to their counts so you get the whole thing we need to test it out now so in order to test it out we're going to create another class here which I've started called count tester and what we'll do in here is we'll first create a log analyzer the class that we just had so we'll create a log analyzer object we'll just call it LA and we have to create a new one okay so now we have a log analyzer object we'll need to pick a file to read from so I'm going to use the read file and we'll put in I happen to have a very small test file to make sure and convince myself that this actually works so it is called short test log and I'll show you that in a minute and then I want to now call count visits per IP and remember that's going to return a hash map so I need to have a hash map variable to put it in so I'll do that hash map of type string integer I'm going to call it counts and now that is going to be assigned the value returned by our log analyzer which is called LA and that's going to call the method we just wrote which is count visits per IP and then once we have that we can just print out the hash map that we created so I'll just have system.out.printline on counts and let's see if this compiles and we're missing something let's see we need to use we use the wrong kind of quotes so we'll put double quotes here and we'll try compiling it again and we forgot the colon at the end all right good we're good so that works and I want to show you over here my simple file and then we'll run it so I have this very short test file and you can see it's got one two three four seven log entries in there you can see the second entry which is <IP_ADDRESS> is in there three times now we're going to run it so we'll come over to blue j everything's compiled we'll run count tester we got to create the new object first and we'll run and there we go and you can see if I can put both of these up here at the same time there we go with one so we created a hash map and you can see that the one that starts with 152.3 is listed as being there three times that is we print out the hash map and it prints out each IP address equals its count that we came up with and so the second entry here is <IP_ADDRESS> with a three and you can see the 157.551 is only in there one time and we got one and you can see the 152.3.135 is in there twice and you can see we counted it twice so now I'm more convinced that what we wrote is correct alright happy coding think back to the problem where you found how many unique IP addresses there were in a web server log that problem was essentially finding out how many different strings there were if you look at what you just did finding how many times each string occurred you may realize that you have solved a larger problem of counting unique strings is already here you have each unique string in this hash map as a key and would just need a way to turn it into the answer you want your counting algorithm has already done the hard work you would just need to be able to extract the answer from the hash map this situation is common in programming you may write code to solve a more complex problem and then be able to solve a simpler problem easily by using the more complex algorithm to do the hard work recognizing such situations can be quite helpful to becoming a highly productive programmer in this case using the hash map from the second problem to solve the first problem is easy hash maps have a dot size method which tells how many key value pairs the hash map has as each key appears once in the hash map the result from dot size tells you exactly how many unique keys there were in the input if you had written count visits per IP first you could have written count unique IPs with just these two lines of code the first line uses count visits per IP to solve a larger problem and the second line uses the dot size method in hash map to turn the answer from that problem into the answer to this problem the size of the hash map is exactly the number of unique keys which is the answer to this problem whenever you are programming try to think of ways to use code you have already written and tested thank you congratulations you have solved the problem of counting how many times each user has visited a site you solved this problem using hash maps which you learned about previously and should be becoming quite proficient with by now that proficiency is great as hash maps are really useful data structure to solve a lot of different problems you also saw how you could use the solution to this problem to solve a different problem easily you could just take the size of the resulting hash map to figure out how many unique IP addresses there are in the log file being able to realize when the solution to one problem can be used to solve another is a very useful skill to work on to develop as a programmer if you think back to the start of this course you learned a little bit about cryptography and implemented the Caesar cipher now you are going to learn a bit about the vision air cipher which historically is quite important as it was thought to be unbreakable for hundreds of years however as you are going to see and do the cipher is quite easy to break with a computer now let's see how this cipher works the key in vision air was classically a word for example here we picked dice as our key you write down the word repeatedly to match the message length each letter represents a number for how much to shift by so dice means shift by 3, 8, 2 and 4 repeatedly well in your cipher program it would be quite convenient to represent the key as an array of ins now to encrypt you shift each letter by the amount written under it much like you did in a Caesar cipher but each letter gets shifted by a different amount the first letter is m which has 3 added to it so you get p the second letter is e which has 8 added to it so you get m then you repeat this process with a higher message as we did for Caesar we'll have to skip anything that's not a letter notice that conceptually this cipher is like 4 different Caesar ciphers one with a shift of 3 shown in blue one with a shift of 8 shown in red another with a shift of 2 shown in green and a fourth with a shift of 4 shown in purple a programmer who has already written an implementation of Caesar cipher could make use of it to help implement a visioner cipher in fact you could make an array of Caesar cipher objects one with each shift specified in the key and use them for your encryption if you did something like this you could use the mod operator to wrap account into the pattern 0123 0123 in any project we're going to give you the code for a visioner cipher and you are going to write the code to break it your goal is to take messages that we have encrypted with visioner and find the decrypted message without knowing the key we used you will start with breaking a message that you know is in English and then expand your program so that you can try to break encryption in a variety of languages why? the first step of your visioner breaker is going to be to write code which only handles the case when you know the key length and are working with a single language like English whenever you're developing software it's a great idea to implement one feature first to test it thoroughly before you build more features into your program and that's exactly what we're going to be doing here remember that the visioner cipher acts like several Caesar ciphers with a message here you see a message which was encrypted with a visioner cipher whose key length is 4 we've colored the letters based on the part of the key that was used if you take only the blue letters you could use the Caesar cipher cracker that you wrote previously to find that part of the key these letters are basically just a message encrypted with a Caesar cipher but they're spread out through the total message then you could do the same thing and similarly for the green and the purple letters this is the conceptual idea of how to break a visioner if you know the key length you slice the string up and you break each slice using a Caesar cipher so what should you write to implement this the first method you'll want to write is slice string which takes three parameters the message to slice up which slice you want and the total number of slices for example if you call this method with four total slices and which slice equals zero you would get the blue letters put together into a string like this similarly which slice equals one would get the red letters as you see here the new line character counts as a string and which slice equals two you'll get the green letters in the string and finally which slice equals three would give you the purple letters again you see the new line character we have some advice for you to help you write slice string first remember the string builder class you learned about earlier it will be useful as you build up the resulting string to return you'll want to append characters to a string builder object second you will likely want to make use of counting four loops in ways that are slightly different than you've seen before you've typically had four loops start at zero but they can start at any number here you see an example of a counting four loop that starts at four of course you could use a variable or a parameter to indicate where you start counting that will prove quite useful in this method you can also have counting four loops that count by something other than one this loop counts by seven so it would print the values four, eleven, eighteen and twenty-five of course you can count by a variable or a parameter instead of the constant seven you see here that may be very useful as you write slice string once you've written slice string you'll want to write the method try key length this method finds the visionary key for an encrypted message assuming that the key length is k length the parameter it also takes a parameter for the most common character in the language this parameters for later pass it e when you write this method you want to make use of slice string that we just discussed and you'll want to use the Caesar cracker class we've provided you with a version of Caesar cracker that's similar to the one you wrote before but with a few changes first we separated out the code that finds the key from the code that decrypts the message and second we've made a constructor which takes the most letter in the language that you're working with in this part of your program you'll just pass the most common parameter this method should return an interay of length key length which holds each of the shifts that the Caesar cracker found for each slice of the message after you've written try key length you have one more method to write for this part of your program break visionary this is the method you'll want to call blue j it will set everything up and will call the method try key length you'll want to use a file resource object to read in the file that you'll want to decrypt file resource has a useful useful method as string which reads the entire file into a string for you once you've read the entire file you'll want to call try key length passing the message you just read the key length which is given to you at this stage and the letter e which is the most common letter try key length will return the key as an array of ints you'll simply pass this to the constructor for visionary cipher and you'll make use of its dot decrypt method to decrypt the encrypted message finally you'll print out the result voila you are done now you've written the code to break visionary if you know the key length but what do you do if you don't know the key length well you could just try some different key lengths you could use the code you just wrote and pass it a key length then see what the result is maybe the key length is one hmm that output is incomprehensible one is probably not the correct key length what if you try to hmm that doesn't seem to be right either how about three oh hey this output looks like it could be English it has readable words it would seem this message was encrypted with a key length of three you could write a loop to try many different key lengths start at one and count up from there the computer can try one particular key length in a fraction of a second so even if a message which is thousands of lines long is encrypted with a key length of 92 the program can do it in no time but how do you tell if the key length is right do you really want to look at the output for each iteration to say that's what we did a moment ago looked at the output to see if it was meaningful text but maybe if we think carefully about what we just did we can find a way to automate it as you look at the incorrect decryption here think about how you knew it was not right this group of three letters forms a word but j o w is not a real word likewise y t isn't a real word nor is y o b none of the words in this message are actual English words contrast that with the correct decryption h o w is the word how d o is do and y o u is u all of the words in the correct decryption are actual words this observation leads to the idea of how to figure out which key length is right count the number of real words in the output you would start by reading a list of English words from a file you could store the list of words in an array list but we are going to recommend that you use a hash set which will give you the same functionality but work much faster then you try the decryption for various key lengths start at one and count up where do you end well you could just count up to the length although if the key is close to the length of the message you won't be able to break it because there won't be enough letters to meaningfully frequency count for this mini project just count up to 100 then for each key length you try see how many of the words in the decrypted text are actual words from the file you read in once you have done that choose the key length key and decrypted text which give you real words as we just mentioned using an array list would work just fine as you read in the file you would use the dot add method to put each word in the list when you want to see if a potential word is actually a real word from the list you would use the dot contains method a better option is to use the hash set class like array list and hash map you can use a hash set containing different types of data so you need to put string in angle brackets after hash set you want a hash set of strings for this problem you will use the hash set in much the same way as you would in array list you can call dot add and dot contains on it you cannot index into a hash set like you can in array list but if you wanted to iterate over one you could do that with a for each loop that is what has been happening behind the scenes when you iterate over a hash maps dot key set the main advantage is that dot contains will be much faster instead of searching through every word in the hash set it can look at only a few words to figure out if it contains the requested information or not the last thing that we will mention before you start coding is how to split a string up into individual words you have already seen that string has a dot split method you can use that passing in quote backslash capital W quote this asks the split method to divide the string up at every character that is not part of a word spaces punctuation or numbers once you have done that you can iterate over those words with a for each loop like this one okay great now you have cracked vision air for English messages of unknown key length but what if the messages were in other languages what if you did not know the language well you can use the same techniques and just try each language for each potential language you would need a list of words in that language and to know the most common character which is not E for all languages then you can try breaking the cipher for each language you would use the same code you already wrote to find the key length that gives you the most real words in each language this gives you the best choice for that particular language then you just see which language results in the most real words that is which languages best decryption is best overall to do this you're going to read the words in for each language you can make use of the read dictionary method that you already wrote as you read each dictionary you will want to put it into a hash map whose keys are strings and whose values are hash sets of strings in particular the key is the name of the language and the value is the dictionary that you read in with read dictionary such a hash map would conceptually look like this here here you have a table where the keys are the names of languages the values are then the sets of words in each language you might think that this type looks a little complex you have multiple sets of angle brackets nested together however it is a great example of an important programming principle do you remember the idea of composition if you took our course programming in the web for beginners you learned about it in the context of html the idea that programming languages allow you to put small pieces together to make larger pieces and that they obey the same rules when you put them together this principle lets you make and understand large and complex things in this case you can understand this complicated looking type by understanding the pieces so what exactly do you need to make your visionary breaker work for unknown languages you need to write two new methods one that counts the most common character in hash set of strings and one that tries the different languages you also need to modify two old methods the break visionary method which is what you run from blue j and the break for language method you need to write the most common car in method to account for the fact that e is not the most common letter in all languages so you will count the frequency of each letter in the hash set of strings which is the list of words for that language you have seen and done many problems with counting how often you find something and figuring out which of something occurs most often so hopefully you are getting proficient in these skills by now the other new method will try each language in the key set of the hash map language you will want to use break for language to do the work of trying to break that one particular language you will want to use dot get to get the word list out of the hash map to pass into break for language you will then want to see how many words you ended up with in the string that break for language returns fortunately you already wrote a method that does that you will then want to pick the best language the one with the most words and its decryption print them out so that you know what your program found you will want to make a few changes to break vision error the method that you call from blue j instead of just reading one languages word list you'll want to read many of them the other changes that you will want this method to call break for all langs instead of break for language so that it tries all of the language that you all of the languages that you read in you will also want to make one small change to break for language up to now you have just passed in e is the most common letter however now you will want to use the most common car in method that you just wrote to find the most common letter in the word list and pass that into try key length so now that you know the plan it's time for you to devise your algorithms and write your code