Thank you, York for the introduction and thanks to everyone for coming here for the first talk in the morning so So I'll be talking about new implementation for the spell utility that I have been working on for about one year so a Bit of an introduction about myself as York said I work About five years back for net BST in the Google summer of code program and So I worked on writing a new implementation of the apropos utility which had a full text search feature and And I got the net BST commit bit about last one year one year back and So this project started about at the same time. So I'll be talking about this So these are the five major topics. I would like to cover today. So I'll briefly talk about what are the shortcomings in the Current spell implementation that is there in the BSTs So from now onwards, I would like to call this the old spell implementation and what I have been working on I'll call that as a new implementation So based on the shortcomings I have come up with the set of features or requirements for that a modern spell implementation should have I'll talk about the Some implementation details that I have done in this project I'll compare the performance of some of the major in spell implementations and With the this implementation and I'll show one small demo of integration with other Shell utilities of how to use spells spell check so This idea for a new spell checker came to me when I was working on the Google summer for code project about five years back But at that time I realized that writing a spell checker is not trivial and I could not do it in the summer of code And I dropped the idea at that point of time and about two or three years back. David Holland Decided I created a bug that we should just remove the spell utility from net BST because It just doesn't work. So he gave cited several examples of some invalid spellings and spell will just Never complain that those are incorrect words So At that point of time I volunteered that and I said that I would like to work on this and I finally water on and I started working on this last year So I'll talk about some of the shortcomings that spell has so it's a very old implementation As per Wikipedia It was created around Unix version 7 from AT&T. So Douglas McClroy Created this implementation spell. He's also credited with creation of I think diff sort Tr Join and some of those utilities. He also created this spell implementation So the problem with this implementation is that it Uses to to decide whether a word is a correct spelling or not. It uses a set of rules called Infection rules. So essentially these rules are So when looking at it at a word if it it will first check whether this word exists as it is in the dictionary or not if it doesn't then it will It has a list of prefixes and suffixes So it will remove if it it will check that whether this word consists of any of these prefixes or not If it does it will remove all those prefixes Similarly, it will check for all the suffixes if it has any suffixes it will remove those and Finally, whatever is left it will check whether that word is in the dictionary or not So as it happens if you saw that So these are all popular suffixes. So fragment Fragmental frog wood if you add any suffix to a word Spell will think it's a valid word because if you remove the suffix, whatever he means is also a valid word So while It's a handy trick to do but It's quite inaccurate and another major drawback is that all these rules are strictly applied to the English language So you probably will not have all these prefixes and suffixes of being useful in other languages So if you want to use the same spell implementation with a different dictionary it will probably not work unless the language is English or English like and Another drawback that I consider missing feature is that it does not give spelling correction So it just checks whether this is correct or not, but it will not say what should be the correct spelling and Finally it lacks a library interface. So for example, when I was working on apropos and I wanted to integrate a spell checking functionality, but Even though there's a spell checker I could not use it because it doesn't have any library interface that I could just call some API's and do it so I would have to end up implementing the whole spell checking interface myself So based on these Shortcomings I have come up with these set of requirements so Just apart from doing a spell check it should also do spelling suggestions So it should come up with some possible correct spellings for the misspelled word It should also use algorithms which are not strictly tied to the English language So for example those inflection rules will not work with other languages. So if if we can come up with algorithms Which don't use such kind of rules that will be good So we can use the same implementation with a different language dictionary and it should work And it would also be nice to have a library interface So other applications for example the shell or apropos or similar Applications could also easily hook up and do spell checks So as part of this project These are the four major things that I've done so I've created a bigger dictionary because the existing dictionary that is there in net VST that is user shared dick words It's it doesn't have all the words. So I worked on in expanding it and then I Worked on writing a new spell implementation using some of these algorithms the edit distance double metaphone and that tries I compared the performance against some of the Other open source spell implementation such as a spell I spell and hunts spell and I Also tried to integrate with other for example with with the shell to see how easy it is to integrate so the existing dictionary that is there in the BST is that is the user shared dick words it is the Merriam Webster's dictionary, but the problem is that it does not have all the What forms so for example if there is a verb it will not have it's it's past and Other work forms or if there is a noun it will not have the plural forms. So That's why the oldest spell implementation use those rules because it could use those rules And even though dictionary doesn't have all those possible words it can remove the prefix and suffix it can still figure out If it is a correct spelling or not, but if you want to get rid of those rules We need to have the full dictionary with all the possible word forms so I expanded the dictionary and So this is the difference that I have currently so the old dictionary had around 235,000 words and the size was about 2.5 megabytes and The expanded dictionary has around 421,000 words and the size is around 4.5 megabytes now talking about the spell checking Issue so when we want to do spell checking there are two major kinds of Spell checking errors that can be made So one is a non-word error. So for example if you if use misspell Applied as Apple. So if you mean if you miss the I then it becomes Apple and Apple is not a dictionary word So that is not a real word. So it's a non-word error. We can easily figure this out and Then there are real word errors for example if you want to spell desert but if you add an extra s and becomes again desert but that's also a word if you want to write there But you exchange R and E it becomes 3 that is also a word if you want to write peace You use a different spelling that is also a word These are much harder to figure out. So how do we handle real word errors? It's a much harder problem we because all of these are actual real real word words So we can't just look up the dictionary and say this is not a correct spelling. So We can use Bigrams or trigrams to figure out based on the surrounding context like for example if the word is I'll be there as soon as possible But okay soon is misspelled But and now there are two possible corrections to that Sun and soon but Based on the pattern we know that as soon is much more likely to be there as compared to as Sun So Based on that based on these kind of heuristics We can figure out. Okay, this is not a correctly spelled word and we should replace it with this word but this is much expensive to do because if you want to do this you have to Scan the every word because you cannot just say you cannot depend on Just Doing a dictionary lookup you have to scan every consecutive pair of words in the file and check if This is a likely Bigram or not So this is a much harder problem and it has a performance impact. So I did not work on this during this project But I have worked on handling the non-word problem, which is a much simpler problem to start with So these are much simpler to detect You just scan through all the words in the file and you check the dictionary if it is not there in the dictionary it is most probably a spelling error and If you use an expanded dictionary, we don't need those inflection rules So how do we represent a dictionary in the memory? So the oldest spell implementation used a nice trick. So it used to M-map the whole file whole whole dictionary into the memory and it would do a binary search. So It would just whatever word is there it will do a binary search and figure out if it is there in the file or not Although it works and it's quite it doesn't require a lot of memory, but I believe I Did not use this. So my first instinct was that if I if I want to expose this as a library interface and If an application wants to do a lot of spell checks then Going through a file like that memory map file might not be Good point to do so I decided to read the whole dictionary into the memory Although I have not compared a performance of M-map version versus reading the whole dictionary into the memory but I have tried some of these things so We need a fast lookup data structure to represent this dictionary into the memory so We could use a hash table, but it does not guarantee a worst-case performance. So if the dictionary is huge is huge It might have performance impact. So even though in normally a hash table guarantees Overn performance, but in the worst case it might just become a linear But we could use a red-black tree also which guarantees order log-in performance, but So if you have a huge dictionary and you want to See whether a word is in the dictionary or not and you have it represented as RB tree you have to do string comparison with at every node in the tree You'll end up doing a lot of string comparisons Tries are a specialized with data structure Which avoid this problem of doing that many string comparisons. These are like binary search trees, but every node has Single character and you just traverse the tree based on matching the characters So you don't end up calling strCmp a lot of times So these I've used these ternary search tries So there's a much like a binary search tree, but instead of having two nodes every It has at every level it has three nodes and Every node stores one character of the word and In the next node if the next character is smaller than this character We go to the left and if the next character is greater than this character We go to the right and if it is the if it matches with the current character. We go to the middle node so That's how we travels so and This also has order log in performance and another Cherry on the top is that we can do prefix Matches so for example if I want to see how many words are there with a certain prefix I can easily do that with this kind of data structure in the login time So this is an example of a turn research try in memory. So for example if I want to Insert a word cat. So I'll start with C and then This is the first first word. We'll just go down the middle node If I want to insert another node in another word, let's say bug I'll go since be smaller than C will come to the left and since there is nothing here will go down the middle If I want to insert cup so C is already there you is greater than C So we'll come down to the right and then we'll come again to the middle and This way you can do again a binary search kind of look up and Insert and get the words in login time So for doing a spell correction, so that was about how do we represent the dictionary? So once we have the dictionary we can look up the dictionary and we can figure out if it is a correctly spelled word or not But now to do a spell correction. There are different set of algorithms So one of the popular techniques is the added distance techniques. So we figure out how many Edit distances this word is from another word and then there are sound based algorithms such as the metaphor and We use and gram models to Figure out which is the most correct Spelling of the all of all these possible corrections. So the added distance is basically If we have two words, so how many edits are needed in this word to convert it to into the other word So the possible edits are there The insertion so you insert you can insert any character in the word or you can delete any character in the word Or you can replace characters with another character and it has been Found that majority of spelling correction are just fun added distance away from the correct spelling Yeah, transposition is also there This is an example of for example if the word is I misspelled eyes So then these are the possible This is not a complete list. I just Shortened it but so for example if you delete one character at a time So if you remove the T it will become E H and if you remove the E then it becomes T H Remove the E then become H becomes then becomes T and so on So these are the possible Words from ta at one edit distance once we have this list again We can look up all these and see how many of these are there in the dictionary and then those are the possible corrections of a misspell word So these are all added distance one if you want to go to edit distance to we take all these words and from these words We go to the next how many words are there at edit distance one from here And those are edit distance to but as we go on the the list of words Increases very rapidly. So looking up dictionary becomes quite expensive Another popular technique is the metaphone algorithm. So this is a phonetic algorithm There was a very popular algorithm called sound x which was developed in the 50s of the 60s and So essentially based on how a word is pronounced or how a word sounds it generates a code so every word gets a code and Based on the misspell word we can generate its metaphone and we can see how many words are there Which have the same Metaphone as similar to this and then Those are the possible Corrections for this particular word. So the advantage of sound x or advantage of metaphone over sound x is that It not it covers not only English, but also Many other languages. So if you want to use it it for other languages, this works quite well and There have been three Metaphone algorithms for the first one was developed in 1990 and but it had some bucks and then the next version called double metaphone came in 2000 and The final version came in around I'm not sure but that is a Commercial version so we cannot use it, but the double metaphone algorithm is available as open so we can easily use it and Finally once we have these possible corrections for a misspelled word We have to figure out out of all these possible corrections, which are the most possible ones So as I described previously, we can use biograms or trigrams to see the surrounding words and see how many of these are I mean how many of these are the most possible corrections here So this is an example if I have a sentence like this I am not feeling very well and I misspelled very as W er y Now I'll have suggestion. I have I will have corrections such as Very W e a r y that is also valid word and I'll I'll have Correction as very V er y both are possible corrections, but if I compare biograms then V er y very well is much more likely than W er y very well So this is a technique to filter down this list of possible Corrections and once you have all these algorithms, we have to figure out how to combine all of these to Get the get the best possible accuracy There are various choices here So you can just take all the words at a distance one and just stop there So if a match is found at a distance one just take that but although That gets good accuracy around 70 to 80 percent, but it does not cover all possible spelling errors Another possibility is once you you go another distance. So you go to edit distance to but edit distance to gets Because a number of words at edit distance to is much higher The number of comparisons you have to see in the dictionary is much higher also. So it becomes lower So what I found is that? These three steps. So first Check all the words at a distance one if a match is found stop there if there is no match found We'll try to find all the words having the same metaphor or a metaphor at distance one or two And out of those we'll see if a match is found in the dictionary or not If no match is found and we'll go to edit distance to so this is an increasing odd Increasing order of runtime Complexity because this is a very fast. This is also fast, but this is very slow So with this you only want to do in the worst case then when we are not able to find anything and then in the research literature there have been some Tricks which have been found so it has been found that There is a much less chance that you someone will misspell the first character of a word So if when we are generating the possible correct spellings if The first word is being changed and we'll give it a lower weight because it is very much less likely that this is Person will make this kind of a mistake and Similarly when we have a set of corrections and Out of those the words which sound similar that is they have a similar metaphor code we can give those words a higher weight because Because they are sounding similar for most probably that could be the correct spelling so based on these heuristics I have done some comparison with the other major open source spell implementations So there is an openly available data set which a spell Uses to publish its benchmarks. So I have used the same data set and compare the performances So the a spell so there are two variations in a spell. There is a normal mode and there is a slow mode So in the slow mode and then there is So this is like if a match is Founded in the first place that is if a space generating just one suggestion that is the correct match and If a space generating five corrections and a match is found in any of them This is one to ten that is a match is found in any of the top ten corrections and this is the same as for the top 25 So for the first position a spell gets around 73 or 74 percent accuracy But if we go to top five a spell gets much better So it jumps to ninety six point one and in the top ten it goes to ninety seven and then ninety eight The slow version is slightly better. It has seventy four and then ninety six point six hunts spell Is better in the if we are doing the comparing the first match in the first position, so then it will have around eighty point five accuracy and Top 25 goes to ninety seven I spell is the oldest of all these all so It has Accuracy around seventy seven percent and in the top twenty five it goes to eighty five So I also have done two variations of comparison. So there is a slower mode and there is a fast mode so in the slow mode it gets around ninety one percent accuracy and then in the top five it goes to ninety five and then The improvement is very marginal. So it almost stops in ninety five in the fast version It is charged at eighty eighty eight point seven and goes to ninety three and ninety three point four so the difference between the fast and the slow mode is that this test data set contains about four thousand words and The slow mode takes around two minutes to compare to come up With spelling corrections for all those four thousand words the fast mode Takes around Around eight to ten seconds to do spell corrections for four thousand words So I've also created a small Demo to show how this works if you want to integrate in a shell So I wanted to integrate this into actual into an actual shell, but I didn't have the time So I just created a shell myself and Integrated it on the spell checking functionality there. So For example, if I type anything wrong It will figure out okay, I spelled the command wrong I mean this is an existing functionality that most of the shells already have but I just wanted to show how simple it is to do this then since I'm using a try data structure and It can do prefix searches. So it is very simple to do Auto completions as well. So for example, if I want to see This is also an existing functionality But since we already have all the words in the memory and we can do this kind of stuff also And another thing is if you want to do Context sensitive auto completion so for example if I'm doing I want to look up a man page and I type man and then I Type I do tab and then it was auto completed to The possible man page name Yeah, and similarly if Another example of context sensitive auto completion is is it visible? So another example of context sensitive look Auto completion is for example if I want to do install a package and I type package add and then I want to see all I want to install some go package So I just type go and it will show me all the packages that are there for go So I built individual dictionaries for man pages package names commands and I'm based on the Context that is if I have typed man. I'm doing use looking up the dictionary of man pages if I am Trying to install package then I'm looking up the dictionary of packages. I'm doing auto completion based on that so All of this is quite simple to do With a library like a interface So I would like to come conclude with saying that although it does not meet a spell or unspell in Their top 25 corrections, but it still is able to match in the in the first order top five comparisons and There's still much room for improving the accuracy and the performance So because I'm using an in-memory dictionary. There is an issue about using too much memory So for example if the dictionary is too big it will consume a lot of memory. So that is something to look into But it's still it would be nice to have a BST licensed Spell checker plus a library so that we can use in our own applications and all this code is currently available on GitHub so it is not part of night BST as of you know, and I'm not sure if it will be but I'll try to post it on the mailing list I I'm still working on this. So there might be a lot of bugs still any questions if you have So Are you looking at what keyboard type people are using so you cannot say what characters are possibly the replacement for a typo? Not yet. So that is a I guess that is a different set of algorithm probably I'll have to see because For example the edit distance one, so I'm using the normal ask I order But if I want to use the keyboard order, then that's a different kind of I need to figure out How to get that ordering of I mean how the keys are placed in the keyboard that is a possibility. Yes So in your demo in your in your demo you showed your special shell, which was expanding context wise That's one approach. The other approach would be to maybe modify package info manual to use your infrastructure Yeah, what would you propose because both of them are kind of intrusive? And somehow it has to be a model which you would use for your deployment of this spelling addition So for example, you're saying for example if I want to modify package add and that should do the auto completion or The question is would you build it into the shell or would you build it into each and every command? it is simpler to do in the shell because that is one place and We can modularize it so we can provide hooks into the shell that shell should call Should use this dictionary to look up when someone is typing this command Modifying each and every command will be much more time-taking. I can answer that because I've written this thing before for DC show So there are certain common Completions so depending position if you're a command or you're a file and or if you're expecting a directory or not a directory So all of these are common so you can put it in the language of the shell Putting it each command would require executing the command every time you have to ask it what the completions are that can be inefficient But one example where I would like to modify the command is for example in apropos So if someone types a query and that does not get any results But apropos can see oh you misspelled this thing as something else so I it can say did you mean this so someone Can know okay. I misspelled this thing. I need to add a query That is one place you I can modify the command itself for auto completion. It makes sense to modify the shell


Hello friends. My name is Tejas and welcome to Readers Notes India. Today we will have the exciting journey of indigenous car. The story of the car company which every middle class family, individual dream to own off. Yes, you guessed it right. It is Maruti Suzuki, the company that brought revolution to Indian automobile industry and open doors for the competition to other automobile companies. The book name is the Maruti story. How public sector company put India on wheels co-authored by R.C. Bhargava who was former CEO and current chairman of Maruti Suzuki India Limited. Padma Bhushan awardee and IIS officer of 1956 batch with Sita who currently writes for Swarajja and first post. Having 30 years of experience in reputed newspapers and business magazines, the story revolves around the rolling out of fully indigenous car Maruti 800. It is an inspirational story and perfect example of public sector undertaking. How it gave tough fight to existing competitors like Hindustan Motors and Standard Motors. The book has detailing of every managerial decision taken and the challenges faced to the convince the foreign partners for having partnership. The book familiarizes us with the Japanese business culture as well as European business standards which were new to Indian market. The book is perfect for any individual business experience, management students and automobile and mechanical engineers. Just grab the book. Link is in the description. Thanks for watching. Please do like, subscribe if you like the review.


I want to give you guys a special message. This won't be long, but hopefully this just resonates with you. Someone's going to ask, people have asked, what do I need to do to be saved? And then after that, how do I stay safe? But more importantly, I want people to think about just your salvation. Paul says in Romans 3, 1st 10, he says, there is none righteous, not one single person who's righteous. There's none who understands. There's none who seeks, meaning there's none who continues to seek on their own. And I'm putting this phrase this way on their own because there have been those who have sought and done so temporarily, but he's using the participle, meaning there's none who is seeking. God, all have turned aside. They have all become useless. There is no one who does good, not me, not you, no one that you ever know. There is not even one. The reason why I bring this up is because even as believers, even still this resonates with us because there are times where you do things, where you think things, you say things, you act in ways that just seems unbecoming of a believer. The Bible says there's none good. Even those of us that are saved, that have the Lord in us, that have the Holy Spirit, there's nothing good about us, as Paul says. The only good thing about us is the fact that the Holy Spirit dwells in us. Why that is important, why I want you to focus on that is because all the times that you have failed, all the times that you have made mistakes, all the times that you've intentionally stumbled. God still holds you in a regard, not because of yourself, not in high regard because of yourself, but because He loves you. How do I know? Well, the Bible tells us that He caused us to be saved. We didn't do so on our own. He caused us to be saved. 1 Peter 1, 3, Blessed be the God and Father of our Lord Jesus Christ, who caused us to be born again. Now, let me tell you what some good news is. Matter of fact, not just good news, but let me tell you what the great news is. Paul says this in Romans 828, and we know that God causes all things, this word there, that word again, all things to work together for the good of those who love God. So all of us who have been caused to be born again and we love God, He's going to cause all things, good, bad, what have you, to work for our benefit, to work for the good of those who love God. Whatever's happening, it is if you love Him, if you're His and He's yours, it is going to work out for the good. Now, it may not be the way that we would design it to be, but for our good to those who are called according to His purpose. For those He foretold, He also predestined to become conformed to the image of His Son, so that He would be the first born among men. Now, I want to focus on this right here, verse 30. And those whom He predestined, He also called. Notice the past tense verbs here. And these whom He called, He also justified. And those whom He justified, He also glorified. These are past tense verbs that are used here. This justification means to be declared right and treated as such. Think about that for a second. As bad as you are, as broken as you are, as often as you mess up, He still says He's going to cause those things to work for the benefit of your good. What's that mean? Well, in bringing you close to Him, because He wants you to be conformed to the image of His Son. And He says that in all of that, He has justified you. You have been justified. That is a statement to say that you've been declared right and treated as such. Declared right by who? By Jesus, by the Lord. And then if He has justified you, well, then the Bible says, Who can bring a charge against God's elect? Verse 33. Who will bring a charge against God's elect? God is the one who justifies. So no one else can say anything counter. No one else can say anything to the contrary. It doesn't matter what anyone says. That is, if indeed you are His. And so if you are His, the Bible is clear. You have been justified. There is now no longer a payment of sin that is required. The debt has been paid. Hebrews 9 tells us that since the debt has been paid, there is no longer a need for a sacrifice. Meaning there is no longer a need for you to go and get saved all over again. You are saved once and for all. And so the question is though, what must I do to be saved? Not what do I need to do to stay safe? Because once you're saved, you are saved. I don't care what anyone says. Let me just give you this. And those who disagree, fine. Let them disagree and work to stay safe. Let them on their own, lift a heavy boulder that they can't carry. They're entrusting themselves to carry this burden of staying safe, not sending too much, not thinking too much about seeing to cause themselves a loser salvation. I'll leave them there. But for the rest of you that want the really good news, the great news, you don't have to carry that burden. The burden is done. However, let's make sure that you are indeed saved. And how does the person become saved? Well, the question was asked of Paul in Acts 16 30. And he says, after he brought them out, he said, sirs, what must I do to be saved? Notice what he says. They said, believe in the Lord Jesus Christ and you will be saved. You and your household. So what's required of salvation? Believe, simply have faith. The Bible calls us believing ones, those that are believing. And it's his part of simple, this continual believing and how does it stay believing? How does it stay continual? Because he gives us his Holy Spirit. That's why he says, cause all of us who are in Christ, he calls us to be born again. And so with this new heart, we then do seek after him. With this new heart, we then do keep believing. With this new heart, we then do keep following and abiding. And so that's the good news.  But you can still glory and be thankful for what he's done, what he has done. Guys, that's the good news. Nothing else you need to hear for the rest of the day that should make your day feel better than what I've just said. Not that I said it, but the Lord said it in his word. And so the good news is not, as I say before, the good news is not that salvation has come to the Jews. The good news isn't even that salvation has come to the Jews and Gentiles or to the world. That's not the good news. The good news is that salvation has come to the world permanently to people who did not deserve it. You're not any good. He is good. And he decided to show goodness to you and save you. Think about that.


Welcome to All Hands Update. I'm Petty Officer Tyrell Morris. These are your headlines from around the fleet. Sailors and Marines aboard USS Theodore Roosevelt rescued two fishermen from a burning vessel off the east coast of Florida, September 29th. After spotting a rescue flare, Roosevelt watchstanders worked with Naval Air Warfare Center to assist the two fishermen. Both men were taken aboard the Roosevelt and later flown to Naval Air Station Jacksonville, Florida. The fishermen were unharmed. Team Navy is off to a good start at the 2014 Warrior Games. Sailors have claimed gold, silver, and bronze medals in the hand-cycling competitions. Team Navy also defeated the Air Force in wheelchair basketball and sitting volleyball. 39 sailors are competing in the fifth annual Warrior Games.


And welcome back, we are here at the DuckConf 2012 in beautiful Las Vegas at the Cosmopolitan Hotel. This of course is Splunk's annual user conference, I'm Jeff Kelly with wikibon.org, I'm joined here by my co-host Jeff Frick from SiliconANGLE. Hello everyone, welcome back, we've had a full day here at the conference with a lot of great speakers, we've had customers, we've had the CEO with the CMO, but now we're going to cut right to the chase, we've got the crazy founders, we're happy to have them on board. I think everybody's a little punchy at the end of the day, but that's okay, that's the way we like it.  Probably closer to ten. Closer to ten? Yeah, I mean seven and a half, ten because once we kind of came off our other job, we didn't know what we wanted to do, but we're sitting around trying to figure out what it was we were going to do for a long time before we even kind of honed in on anything. So I think it was closer to ten years. Closer to ten. Funded almost eight years. Funded eight years ago. Funded eight years ago. Yep. And how did you hone in on, well backup, how did you pick the word Splunk? Well, there's two parts to this, there's why Splunk and then how do we get it, the difference between the ten years and eight years that Rob's sort of alluding to is there's a two year period of time when we tried to figure it out. This isn't our first rodeo, we've done startups before and we thought this time around we'd spend a little more time up front doing the homework and I think I just honestly, one of the reasons that I think we're sitting here today with a healthy company that's doing well is because we did a bunch of homework up front and we can go into lengthy detail what that was. One of the things we did is we talked to lots of customers and we came apart upon the problem by talking to people and one of the things that we heard people talk about was they had this massive amount of data and that when they went looking through it, it was a lot like cave diving. Literally it was this big, ugly muck of data and you put on your hip waders and your headlamp and you sort of went into the cave looking blindly for stuff and they would, it's sort of like election they actually used the word Splunking my data. I don't know if that's true and we're post-rationalizing or whether we making that up but a couple of interesting things is Splunk then as a, we're very brand sensitive that Splunk is a little like Google. It's a misspelling of a slightly geekier weird term like Google and Google are misspellings and they can use it as an adjective or a verb or a noun. I guess you can't use it as a noun but you wouldn't want to look that up and figure out what a Splunk noun is. I'm not the guy you want to look at. By the way, our English grammar question spot hated the name and it took a lot of convincing to, I mean they're like enterprise software company called Splunk are you guys nuts? But when you think about it, it was kind of like, almost like the consumerization of enterprise software at this point in time. So it was entirely on purpose that we picked a name that was catchy. Slightly inappropriate. Yeah, I mean all, it's slightly inappropriate and everything just so that it was unforgettable and it didn't sound like other enterprise software companies because we really wanted to focus at end users, not C level folks and kind of a turning around of the enterprise sales model at that point in time and I think the name fit really well with that. Whether you like it or hate it, you're going to remember it. And when you're a two person company, one of the things you struggle with is just a brand that people remember and you know, you'll remember it. Yeah, absolutely. Picking a name is one of the worst things you can do. It's very hateful process.  I think you guys hit on a good one. So what's interesting is this is like almost 10 years ago. So this is before the term big data was popular or even coined probably. So, but nevertheless, you're still dealing with, I mean, big data is relative. So even then you were dealing with pretty large data volumes. And in a way, the industry is now kind of caught up with you. So tell us a little bit about kind of that big data evolution from the time you started and how that's kind of changed over the last 10 years and how Splunk's mission is likewise had to kind of adapt into this new world that we're calling, you know, the big data era. Well, I mean, so literally from the day that we wrote the first part of the code, it was really, really slow. So it couldn't consume big data if it tried. But it was obvious that you could see if you were using it what the potential was if you could stick a lot of data into it. And our fabulous engineering staff of people a lot smarter than Eric and I spent a lot of time just honing it like squeezing water out of a rock, release after release after release after release. And we never thought about big data. We never thought at first we didn't even think we had anything to do with analytics. And, you know, we came up with the reporting stuff, but it was, you know, I don't even think that I was interested in doing reporting early on. It didn't make sense to me. I was like, come on, everything's about search. It's all about search, not the reporting. It turns out the reporting was a very critical piece. But now it's big data. And I think we were very lucky that somebody came up with the term big data right at the time. Somebody put a name about this around this. And we had never even thought that it didn't even make sense to us. All of a sudden somebody put it into a category and we just fell right into it. I think that's kind of the way that it happened. It never really was a thought. Success is often luck. Right time, right place. We've done companies in minutes just off the wrong place at the wrong time. Also, you know, I don't think we ever thought of ourselves, and maybe today, even big data, we're a machine data company. We started with machine data. And it just sort of happens to be that machine data is often big data. But we started out as machine data. We've always been interested in, I mean, one of the reasons why we now have to sort of the Google search model was is early on we did to machine data what Google did to human-generated data. So the analogy has been that human-generated content's growing. I guess your cameras are like this, right? It's growing this fast. Machine-generated data, we had these graphs from early on. And now they're big data graphs, but early on they were machine data graphs. And you had tools like Google for dealing with human-generated content to find something. But you didn't have the analogous tool for the machine data. And what you're finding is this sort of maybe 40-year revolution going on where machine data is becoming extremely valuable just like human-generated content. And over the next period of years, you're going to see everything's growing up data, whether it's airplanes and cars and buses or whether it's medical devices, everything's going to be throwing off data. Yet there's no tool effectively like a Google or any kind of tool to deal with that massive amount of information. So we're sort of like Google looks up there. We look down here. We index it all and allow you to dig through it. And I think that's where the revolution's occurring, right?  Now, what's interesting, and you said you've done some startups. And I think a lot of us haven't. Probably a lot of our viewers have been at various startups. And you always hope it's your last startup and it works. But oftentimes it doesn't work out that way. So now that it worked out and sounds like you haven't really had a pivot point, you've basically been to grow off kind of the original idea, which is spectacular. But now you've got the new challenges of growing the company, maintaining a culture that you guys obviously, you've got some personality. You're excited about who you are and what companies are. But you've got to grow it with new leadership or extra leadership and just a lot more people. And now you kind of, by de facto, getting out are the leader in the space, right? There is a category now, big data. And you are kind of the leader. How has that kind of changed things? Or how does that kind of, do you feel in that responsibility to kind of lead the charge and take it places where no one's really thought of it going before? Or you just keep doing your thing and having fun and coming up with some of the greatest titles I've seen at any company that I've loved it. That's it. That was the answer to a very long question. Yeah. Now we, you know, I mean, as far as the culture is concerned, Eric and I and we, in previous companies that we have founded or been very early members of have created a very similar culture and it was really nice to see that culture sort of grow to a much larger. I think it's really difficult to keep culture like that because as you, when we first started, we were hiring people that were just like us, crazy, listening to the same kind of music, doing the same kind of thing and enjoying the same kind of stuff. And after a while you can't do that anymore because there's not a lot of those people around. And you hired them all. You're hiring different people, different cultures. Some of them are quieter, some of them aren't. And how do you maintain a company with all these diverse people? You have to find diversity in people. You can't get everybody that's all the same. And so how do you maintain the culture? It's hard. I mean, there are certain, as long as we keep people around from the old days that remember the way that the company was, I think that they can continue to hold that, to hold that as they keep hiring people. It's totally important to keep that culture around. And I think that's one of the, one of probably the largest reasons that Splunk is successful is because of that culture. We have people that are working here longer than they've ever worked anywhere in their lives. We have people that came out of school. This is, it was their first job and now they've been here six, seven years. I think that's a testament to the culture. And I think that's, you know, like I said, probably the most important thing about building a company that's like this. Yeah, absolutely. And you can tell that the employees here really feel connection. Absolutely. Really invested in this company. And I wonder, you know, we often hear about, it's very easy to understand what a company does, but not always why they do it. What is your mission? Why do you do what you do? Why is it so important? You know, we know it's machine generated data, getting insights from that data, but why is it important for you? And how do you keep everyone so engaged? Well, I think personally I think hiring people like Godfrey Sullivan and the rest of the staff is totally critical. It's not about, it's not me keeping people engaged. It's not Eric keeping people engaged. It's hiring the best of the best and then enjoying the success of that, right? So how many of these people that have been here seven years in the stick around if the company wasn't doing well? I don't know, that's a really good question, right? I mean, you can drink beer and have cocktails all day long and other companies too. A lot of people on the south of the market these days. Yeah. I think the initial idea still has a lot of legs. The customers that have not adopted this technology, there's so many of them still out there that don't even understand it yet. So even with what we have today, I think that we can grow this to a really large company. But that doesn't mean that you stand still, so. And then we used to have up on the walls that we're only at mile three of the marathon. It really feels like you're in the middle of the game, right? We're not done. It's not done. No, we're not done. You're not maybe even a quarter of the way done yet. And I think a lot of people feel that. Like we're just, you're not done. There's so much more we have to do. In fact, I was talking to one of our earliest clinical engineers. He's like, Eric, the roadmap has never been thicker, richer, deeper, more options, you know, and the team's executing well. And I think as long as there's customers who like the product that you're delivering new functionality, that the roadmap is thick and people want it and people are liking it, you're not done, right? I think the problem would be if the company ever started to, you know what I mean on the financial side, that users started losing enthusiasm or there was that lack of that really positive, our roadmap is almost extensively driven by customer input that just love the product. They love it and they say, damn, if it just, please Eric, can you make it do this? And we have so much to do. I think people just react to that, right? They come to work in the morning because there's just, there's so much stuff that people need done. And that's not always true in the companies I've been in. You sort of hit that stall point where you don't exactly know what to do. Maybe you need to pivot or maybe you're running out of stuff. In our case, it's just getting, the pipeline's getting bigger and thicker. I think also like, even from the days that we were doing research when it was just a few of us doing cold calls, I mean, just to figure out whether we even had an idea that made sense that we were starting to build, we wanted to build this company to last, not to be a flash in the pan, not to be something which was two years and then got acquired by Google, but really a company that was built to last, a new kind of company that had legs.  Yeah, and we're very fortunate that the way we built it turned out. I mean, it turned out the way that we wanted it to. So they're giving us the signal over there on the other side of the control desk. So just to kind of close, you've been living it for 10 years. What are some of your favorite customer success story? Your favorite customer uses of Splunk that you just would have no idea that either somebody would think they use it that way or that the relationships that they were able to query were just so fantastic that, wow, what a fantastic tool of discovery as we are on the data discovery. Yeah, I was doing a homebrew kit yesterday. That's cool. I haven't heard that one. I haven't heard that one. I'm monitoring a homebrew. That's monitoring a homebrew kit with Splunk. I like that. Anything you're doing with yeast is good. Rob makes wine, so. How many data sources does he have on the boiling keg back in the kitchen, hopefully? There was one person that we talked to who was using it to monitor and create a feedback loop inside of a greenhouse environment, monitoring things like moisture and humidity and then controlling the louvers that let the air in based on what it saw. So having the software actually make decisions on how to control the environment based on the feedback it was getting from the sensors. I thought that was really cool. I think what Nest Labs is doing with the thermostat data is incredibly cool. I think that the idea of taking, of putting sensors on everything, everything that you touch in 10 years from now will have sensors on it. We're working with car companies, airplane companies, power generation companies. You know, we started out in the data center and that was cool. But it's really, that was the first mountain. The next mountain is there's this whole world beyond servers that is just about everything else you touch, right? It's just, it's everything else on the planet that's somehow physically a device. And we're seeing everyday new use cases of people, whether they're people rolling tractors and they're doing, as Rob said, as the tractors rolling, it's collecting data, whether it's delivery trucks that have mounted sensor devices and they're capturing weather data, climate information, travel time, and they're just saving it. They don't even know why yet. They're just like, okay. I think the use case that Garfrey mentioned in the keynote today with the Japanese people that are doing the elevator is probably the most stunning use case I've heard of in the last year. It may turn out that that's a great economic indicator and no one ever looked at it. Lots of hospital beds that are now throwing off data that we're working with. So there's some really cool, it's not just servers and data centers, it's all the other stuff that we've got going on. Yeah, I mean this big data really touches every industry. It's obviously a great business to be in. As you mentioned, you've got more things to do and as more and more devices, devices you never thought before might be, you know, equipable sensors, that's another data source, just increases the possibilities of things you can do. Absolutely, absolutely, absolutely cool. Well guys, thank you so much for coming. We really appreciate you making time. I know you guys are probably very busy here at dot com 2012. Or not. Or not. Nevertheless, we appreciate it anyway. So this is Jeff Kelly with wikibon.org. We are at dot com 2012, Splunk's user conference. We want to thank Splunk of course for having us here. Really appreciate it. Bringing you two full days of live coverage of the event. Again, I'm Jeff Kelly with wikibon.org. My co-host Jeff Frick from Silicon Angle. And we're gonna wrap up the day shortly. We'll be back with one more segment, kind of giving you our final thoughts on the day and then previewing tomorrow and the interviews we'll have for tomorrow. So thanks a lot and please stick around.