Hello everyone, my name is Jenna Hood and I am here representing Earthdance Organic Farm School. I'm going to talk today about our polyculture orchard that we have set up on our farm. I'm going to sort of talk a little background initially about about Earthdance as a whole and before I get into the origins of our orchard, how it got, how it came to be laid out the way that it's laid out and sort of where we're at now with the orchard. So I started with Earthdance in 2018 after completing their apprenticeship program. I came on board as the assistant farm manager and subsequently in 2020 took over the role as a farm manager. So pretty much the the origin piece of the of the orchard happened prior to to my experience with Earthdance. So but I'll get into a little bit more of that. So we are a nonprofit educational farm. We combine food production, agroecology education, and community development. We manage a mixed vegetable, mixed fruit, and livestock production on 14 acres in Ferguson, Missouri, which is a suburb north of St. Louis, Missouri. In addition to food production, we're also operating a variety of programs aimed at educating adults and youth about organic farming and gardening, regenerative agriculture, and sustainability. We offer beginning gardener classes as well as full-time paid summer apprenticeship that's targeted into folks interested in farming as a career. In this program or in the the apprenticeship they receive 40 hours of hands-on education and training each week for a 10-week duration during the summer. We also have a partnership with our local school district where we engage with their vocational program to provide their students with hands-on education as well as some classroom-based education. We host numerous field trips and also collaborate on research projects with nearby universities and research institutions. And we manage roughly 10 to 15 volunteers each week, April through October. Our farm site is designed with permaculture principles at its core. We have six acres in cultivation currently, about roughly 8,000 square feet of high-tunnel space, a 36-by-40-foot pack house, 30-by-96 greenhouse, a farmhouse office, and a 200-plus, yeah a little over 200 trees at this point, mixed perennial orchard planted on berms and swells with our annual row crops or annual vegetable crops, alicrop in between, along with pastured poultry, mushrooms, herbs, cut flowers, pasture, prairie, and a mixed wood lot. So the origin story of our orchard all starts several years ago, back in 2014, we, with the help of us, Sarah from a rancher grant, we outlined a plan to plant a 60-tree pear orchard. And just prior to actually putting the trees in the ground, we sort of, upon some reflection and more research, we decided that planting a single area of the farm to only pear trees was really not quite in alignment with the principles of organic farming and permaculture that we were, we were, you know, really going after. And so with the ideas of integrate rather than segregate and valuing diversity, we set out to design a different kind of orchard that would integrate the pear trees into a much larger plan for establishing a diversity of tree crops that were planted throughout the farm. We grew some inspiration from a couple of different places. The permaculture orchard, which is a documentary in Mark Shepherd's book, Restoration Agriculture, really spawned a lot of our ideas and really set into motion what it was that we were trying to accomplish. So before laying out the trees, the Earth Dance farm team at the time established a system of swales and berms, and a key line design was used to site the swales. And if you're not familiar, key line design is a technique for maximizing the beneficial use of your water resources on a piece of land. In our case, the land slopes pretty dramatically west to east. And so we cited our swales in such a way, our swales and berms in such a way that the water flowing, you know, flowing down the hill is going to be captured in those swales in an effort to, you know, not just be totally running off the farm. The trees were planted into the berms, which were mounted on the lower side of each corresponding swell. So if you see in that picture there, the individual, she's standing actually in the swell and we created these using a rotary plow on a on the tractor. And essentially taking the soil from the swell and throwing it in one direction, creating the berm, where the trees were to be planted. The fruit trees were alicrop throughout the growing fields and parallel rows about 80 feet apart. And alicropping is ultimately the planting of rows at wide spacings with an annual crop grown in the alleyways between the rows. So we are, you know, we, we understand that over time as our trees get bigger over the years that we're going to eventually be losing a little less production area for our annual vegetables. And that was something that we knew going into this. So that's just something that we plan for each year as we're doing our crop planning. We know when we're going to have to take a particular bed out of production due to shading from the surrounding trees. So you can see on this hand-drawn map here how the berms and thus the orchard are situated in between our main vegetable production beds. West being at the, you know, the top part of that picture and the land sloping, sloping east and the berms are roughly set up on contour. They're not perfect, but they, they, they get the job done in terms of capturing the water when we do have rainfall. This is a drone dot that shows maybe a little bit better of stuff in action. You can see the main production areas. There's a portion of an area cover cropped in the lower left-hand side and the berms being, you know, the taller trees that are dividing those other, those main production areas. So I want to talk a little bit about a couple other specific things that went into the planting part of the orchard. So we did put down a heavy-duty landscape fabric on top of the berms and around the trees. This was done after the trees were planted. And we also selected several other tree species that were all adapted for organic production, ultimately to help us create a diverse polyculture of tree fruits. We then added to the understory lots of herbaceous shrubs, flowers, and herbs in the form of gill plantings. And these were a combination of mutually beneficial species that were selected to create a thriving mini ecosystem around the individual fruit trees. And all of these plants were selected to attract pollinators, suppress weeds, and to fix nitrogen. And we planted those, you know, just by punching holes in the landscape fabric. And through the years, we've mulched with wood chips around or on all of those berms as well on top of the landscape fabric. So in order to, because we're, you know, not putting all of the pears together, all of the apples together, we had to create a sequence that would allow or that would ultimately support the pollination needs of the individual fruit trees. And for us, also important was to mitigate the spread of pest and disease. And so by spacing the trees out, not planting all of one type of tree together, it was the hope was that it would mitigate the spread of pest and disease. So our sequence that we came up with was pear, two paw paws, another pear, tart cherry, pear, plum, two more paw paws, a Juneberry, and an apple. And that sequence repeats throughout all of the berms. There was a quite a lengthy process of selecting the specific varieties. A lot of research went into that. We chose Asian and European pear varieties for crop diversity and for market appeal. And all of the pear varieties were chosen for their superior, supposed superior resistance to fire blight and also for their marketability. We added a couple initially Shinco and Korean giant were the two Asian pairs that we had chosen. And we eventually added Shinziki and Chojuro to increase pollen diversity and for their unique flavor that those two types have.  And I would say our acro pears are most, I might as well just say, they sort of round out our polyculture orchard. We have planted 60, roughly 60 paw paws on the farm. If you're not familiar with paw paws, they are known for being the largest fruit native to North America. They have just the most delicious flavor. They've seen very little attention from the farming community. You might be familiar with them growing in the wild, which cultivars that you find, or excuse me, varieties, those that you find growing in the wild, very low yields, but there are numerous cultivars that produce reliably and heavily. And as being that they're a native tree, they've shown minimal pest pressure and minimal disease. So we were really excited to plant those as a specialty crop for farmers market and for restaurants. And I will say the buzz about paw paws right now is just crazy. And if you're considering fruit trees, I highly recommend you consider paw paws. Additional species of juneberry, tart cherry, plum, and apple were chosen for their promise as potential organic fruit tree options as well. A lot of research was put into each variety, also always going back to a focus on disease resistance. Junberries, they were selected also, they're a native with several named cultivars and they produce pretty sweet berries. Tart cherry and the plum varieties were chosen for their early fruiting traits, the hope being that they would produce fruit prior to any heavy disease pressure. And lastly, the apples varieties that we chose, apples in the Midwest, grown organically, are highly susceptible to numerous pest and diseases. So we did try and choose the most resistant varieties as possible. So that's how the orchard was sort of set up a little about why we selected the cultivars that we did. I want to talk a little bit about sort of what's happened since the fruit trees were planted. So the majority of the trees went in in 15, well 14 and 15, a few more were added in 2016. And during that time, the farm managers who sort of solved this project, they were the brain children behind this project. They have since left the farm and their knowledge and expertise has been, unfortunately, I came in with less knowledge and expertise about fruit trees than they had. So my priority and my focus was on vegetable production. And so needless to say, the orchard as a whole was not able to ultimately receive the attention that it deserved. And so anyway, let's just talk about what's happening now and where we are with our management of the orchard. So ultimately, it's not a huge orchard compared to many orchards that folks might be more familiar with. But even as such, the size of our orchard requires more attention than there's three of us full time. And you've got to remember that we are, our time is also divided up between not just production, but also with teaching. And so we were really unable to ultimately prune every winter as required. We lost some trees to disease, firebite being the primary disease. And with the nature of our orchard setup, with the trees being planted on the berms, it also required that we manage those berms. We couldn't just, you know, mow up next to the fruit trees like you could if it was set up as, you know, sort of a more traditional orchard might be. Many of the trees over time had been overtaken by a Japanese hop spine. And the understory was overgrown with Bermuda grass and other weeds. At one point, elderberries were added to the berms, which were not in the original plan. And we found that they're now actually competing with the fruit trees. In an agroforestry scenario, we would be able to, you know, we could potentially run livestock through the linear orchard, but with our vegetable crops being adjacent to the orchard the way that they are, it just doesn't quite work for our setup. So through the years, we were managing the weeds and whatnot with volunteer help. But even with volunteers, we were hard-pressed to really stay on top of everything. So even with the neglect that the orchard received, the trees were set in fruit. We started getting fruit off of trees in 2017. When we had our first pair of harvest, we had a very small pawpaw harvest as well. And we've seen slightly larger harvest happening in the subsequent years. And I mentioned, so last or in the fall of 2020, I took over the role of farm manager. And I was introduced to a gentleman named Wink Davis. Wink, he had recently retired and I learned that he had owned an apple and peach orchard in Colorado. He and his wife sold their orchard, moved to St. Louis to be closer to their adult, one of their adult children. And he was looking to, you know, get back working with his hands and was really eager to get involved with the orchard at Earth Dance. So with Wink leading the charge, we put together a mostly volunteer, but some staff led orchard committee that we worked together to outline, to actually, you know, line out some goals pertaining to the orchard. And so we'd have a clear direction and could work toward bringing those goals to fruition. During these conversations with the orchard committee, it became clear that we were really, really wanting to pay special attention to the soil microbiology and ensuring that we were providing the trees with the nutrients that they needed to remain healthy and less susceptible to the pests and diseases that we experience here in the Midwest. That orchard committee came up with some goals. I'm not going to read, you know, read these necessarily, but important goal, grow abundant, delicious nutrient dense fruit and berries. And we wanted to grow those with a commitment to achieving those through biological and ecological means and without the use of chemicals. And the health and diversity of the biology and the microbiology, we wanted to manage pest and disease pressure by improving the health and diversity of the biology and microbiology. And we also wanted to take what we were learning through our own sort of experimentation and use that as a part of our education. It's really, we're a teaching form. And so everything that we do, I like to say everything we do is an experiment. And even, you know, you learn oftentimes learn more from your mistakes than, than, than from the things that go well. So we're, we're constantly learning and adapting. And the orchard is a prime example of that.  Even with his, even with us having more attention on the orchard, it's still primarily a volunteer-based thing at this point. We last year, were able to host a pruning workshop, pruning workshop in partnership with a level three arboretum here in St. Louis. And through this workshop, we garnered some attention for the orchard and were able to recruit some additional volunteers to assist with our pruning efforts. So for the first time, since probably 2017, we were able to prune every single tree on the farm. We did have to apply some restorative pruning techniques to a lot of the trees, which meant bringing down the leaders and a lot of the tall vertical growth to make the trees more manageable from the ground or using short ladders. We also opened up the centers to bring in more light and airflow, eliminated any redundant and damaged limbs. We were, we knew that fruit production this past season after that more extensive pruning was probably going to, that we were just going to have a smaller harvest and that ended up being true. But we're confident that that we'll see improvements in yields this year and in subsequent years. We also began the season following a holistic spray regimen that was outlined in a book by Michael Phillips called The Holistic Orchard. And without going into all the technical details, the spray regimen begins at Budswell and continues as determined by the stage of bloom. One thing we realized, again, kind of because of the way the trees are set up and also the fact that there's so many or several different species of tree, the trees progress on very different schedules. And so that meant not every tree necessarily received sprays at the precise right time that would have been indicated by the stage of bloom. One thing that, an additional thing that we experienced this past year, the papas were in full bloom with the promise of just a wonderful harvest when we had two nights of hard freeze. And as a result, we had far fewer papas this year, which, you know, can't do anything about the weather. That's just kind of one of those things that you got to kind of cross your fingers and hope for the best. So we also suspect that cold also impacted other trees as well. So in addition to the pruning and that late freeze definitely took a hit on our fruit production this year. We also had an extended cool wet spring, and that was perfect for the propagation of fire blight. We spent more time than we ever have cutting out disease branches and even removing some trees, some pear trees where the bacterium from the fire blight was determined to be systemic and the trees could not be saved. We also saw evidence of coddling moths on the apples as well as some other pest damage that we were not able to positively identify. So and despite selecting varieties that are thought to be more disease resistant, you know, there's no guarantee that those varieties, they're not perfect, right? Once something takes hold, really all you can do is try to improve the overall biology. If you're looking the way we're taking, you know, the way we are approaching a holistic management of the orchard, we're recognizing that it's just going to take time and patience to really correct the imbalance that currently exists with the overall biology in the soil, in the soil with the trees. So a couple other things from this past year. Managing the berms was quite a task. We ended up, I'm looking at time, I'm running out so I'm going to speed through this. We ended up pulling up all that landscape fabric as a result of the just knowing more now about how ultimately the landscape fabric was preventing the microbial, the sprays, the beneficial microbes that we were applying to the trees was not able to penetrate at the soil level due to the landscape fabric. So we ended up removing a ton of landscape fabric this year, which was extremely time consuming. We're also having to take the extra step of dechlorinating our water because we use municipal water at this point and all of the beneficial microbes that we're putting into the orchard would be killed by chlorine in the tap water. So we did dechlorinate the water and we do that by aerating it and adding humic acid. We also recognize that the orchard could really benefit from consistent management. The majority of the pruning spraying fabric removal was all done by volunteers this year. It felt like we were playing catch up, not staying ahead of the demands of the orchard. And we're just still not at a place where the fruit production on its own justifies hiring someone to outright manage the orchard. So one day we hope that the orchard can be a self-sustaining enterprise. But until then, we're hoping that interest in what we're doing around biological and regenerative farming will attract interest via grants, donations, revenues from educational activities, and that those things will offset the fact that the fruit trees themselves will continue to take time to be profitable on their own. Yeah, so there's resources that I mentioned already specific names of those books and documentary and the tree sources where we sourced I think 95 or more percent of the trees that we planted in the orchard. I have not been looking at comments. I can look at that real quick and see if there's questions in the chat. Yeah, there is one question in the chat about what kind of sprayer you use or recommend. I don't, off the top of my head, we, if you saw in that one picture, it's a, I don't know, off the top of my head, Wink purchased it. I paid for it, but Wink purchased, you know, found it and purchased it. But I can try and find that out. I'm not sure how I would get that information to the person who posed the question, Nancy. Yeah, Nancy's waving at me. Yeah. Does anyone else have a quick question? We've just got like a minute or so.  If you had to do it over again, would you space out your trees differently for starters? I don't think, we wouldn't, I don't, I don't have concern necessarily with how the trees are spaced out or the sequence that the trees are spaced out. I think if we could do it over, what I would, would want to do over is, all right, we've got someone, someone that is dedicated to the orchard, that it is part of their sort of responsibilities. Maybe it's not their only responsibility, but it, someone who can take ownership of it. And we just were lacking that once the, the farm manager who established the orchard moved on.


Live from Orlando, Florida, it's theCUBE. Covering Pentaho World 2017. Brought to you by Hitachi Ventura. Welcome back to Orlando, everybody. This is Pentaho World, hashtag P-World17, and this is theCUBE, the leader in live tech coverage. My name is Dave Vellante, and I'm here with my co-host, Jim Kobielus. Donna Perlich is here. She's the Chief Product Officer of Pentaho and a many-time CUBE guest. It's great to see you again. Thanks for coming on. Thank you for all of them. Happy to be here. So, yeah, I'm thrilled that you guys decided to sort of reinitiate this event. You took a year off, but we were here in 2015 and learned a lot about Pentaho and especially about your customers and how they're applying this sort of end-to-end data pipeline platform that you guys have developed over a decade, you know, plus. But it was right after the acquisition by Hitachi. Let's start there. How has that gone? And so they brought you in, kind of left you alone for a while, but what's going on? What's up to date? Yeah, so it's funny because it was 2015, was Pentaho World, second one, and we were sort of like, wow, okay, we're part of this new company, which was great. And so for the first year, we were really just driving against our core big data integration, analytics business and capturing a lot of that early big data market. And then probably in the last six months with the initiation of Hitachi Vantara, which really is less about, you know, Pentaho being merged into a company. And I think Brian covered it in a keynote. We're going to become a brand new entity, and which Hitachi Vantara is now a new company focused around software.  And so now as part of that bigger organization, we're really kind of at the center of that in terms of moving from edge to outcomes as Brian talked about and kind of how do we focus on data, right? Digital transformation and then achieving the outcome. So that's sort of where we're at right now, which is exciting. So now we're part of this bigger portfolio of products that we have access to in some ways. And I should point out that Dave called you this CPO of Pentaho. But in fact, you're the CPO of Hitachi Vantara, right? No, so I am not. I am the CPO for the Pentaho product line. So it's a good point though, because Pentaho brand, the product brand stays the same because obviously we have 1,800 customers and a whole bunch of them are all around here. So yeah, so I cover that product line for Hitachi Vantara. Well, and there's a diverse set of products in the portfolio. So I'm actually not sure it makes sense to have a chief product officer for Hitachi Vantara, right? Maybe for different divisions, it makes sense, right? But so I got to ask you before the acquisition, how much were you guys thinking about IoT and industrial IoT?  Certainly it was a discussion point and GE was pushing all this stuff out there with ads and things like that. But how much was Pentaho thinking about it and how has that accelerated since the acquisition? Yeah, so you know, at that time in my role, I had product marketing, I think I'd just taken product management. And what we were seeing was all these customers that were starting to leverage machine generated data. And we were thinking, well, this is IoT, you know? This is, and I remember going to a couple of our friendly analysts folks and they were like, yeah, that's IoT. And so it was interesting, it was right before we were acquired. So we'd always focused on these blueprints, right? Of, you know, we got to find the repeatable patterns, whether it's customer 360 and big data. And we said, well, there's some kind of emerging pattern here of people leveraging sensor data to get a 360 of something, right? Whether it's a customer or a ship at sea. And so we started looking at that and going, we should start going after this opportunity.  And then the last year we've probably seen like more than 100% growth in customers that are purely from a Pentaho perspective, leveraging machine generated data with some other type of data for context to see the outcome. So yeah, we were seeing it then. And then when we were acquired, it was kind of like, oh, this is cool. Now we've been, now we're part of this bigger company that's going after IoT. So absolutely, we were looking at it and starting to see those early use cases. Now, but a decade or more ago, Pentaho at the time became very much a pioneer in open source analytics. You incorporated WECA, the open source code base for machine learning data mining and so forth into the core of your platform. Now today here at the conference, you've announced Pentaho 8.0, which I, from what I could see is an interesting release because it brings stronger integration with the way the open source analytics stack has evolved. There's some spark streaming integration. There's some Kafka, some Hadoop and so forth. Can you give us a sense of what are the main points of 8.0, the differentiators for that release and how it relates to where Pentaho has been and where you're going as a product group within Hitachi Vantara? Yeah, so starting with kind of where we've been and where we're going, as you said, Anthony Deschetser, our head of customer success said today, 13 years, right? 13 years, I think on Friday that Pentaho started with a bunch of guys who were like, hey, we can figure out this BI thing and solve all the data problems and deliver the analytics in an open source environment. So that's absolutely where we came from.  So with 8.0, it's a perfect spot for us to be in because if we look at IoT and the amount of data that's being generated, and then the need to sort of address streaming data, data that's moving faster, this is a great way for us to kind of pull in a lot of the capabilities needed to go after those types of opportunity and solve those kinds of challenges. So the first one is really all about how can we connect better to streaming data? And as you mentioned, it's Spark streaming, it's connecting to Kafka Streams, it's connecting to the Knox Gateway, all things that are about streaming data. And then in the scale up, scale out kind of how do we better maximize the processing resources? We announced in 7.1, I think we talked to you guys about it, the adaptive execution layer. So the idea that you could choose the execution layer or the execution engine you want based on the processing you need. So you can choose the PDI engine, you can choose Spark, you can choose hopefully over time we're going to see other engines emerge. So we made that easier, we added Hortonworks support to that. And then this concept of, so that's the scale up, but then when you think about the scale out, you've got to, sometimes you want to be able to distribute the processing across your nodes and maybe you run out of capacity in a Pentaho server, you can add nodes now and then you can kind of get rid of that capacity. And so this concept of worker nodes that we use some of the, to your point earlier about the Hitachi portfolio, we use some of the services in the foundry layer that Hitachi's been building as a platform. As a low balancer. As part of that, yes. And so we could leverage what they had done, which if you think about, you know, Hitachi they're really good at storage and a lot of things that Pentaho doesn't have experience in and infrastructure. So we said, well, why are we trying to do this? Why don't we see what these guys are doing? And we leveraged that as part of the Pentaho platform. So that's the first time we've kind of brought some of their technology into the mix with the Pentaho platform. And I think we're going to see more of that. And then lastly around the visual data prep. So how can we keep kind of building out on that experience to make data prep faster and easier? So can I ask you like a really Colombo question on that sort of load balancing capability? Nice looking trench coat you're wearing. I mean, cigar. So is that the equivalent of a resource negotiator? Do I think of that as sort of your own yarn? You know, I knew you were going to ask me about that. Is that unfair to position it that way? I mean, conceptually right. It's going to help you to better manage resources. But if you think about mesos and some of the capabilities that are out there that folks are using to do that, that's what we're leveraging. So it's really more about sometimes I just need more capacity for the Pentaho server. But I don't need it all the time. Not every customer is going to get to the scale that they need that. So it's a really easy way to just kind of keep bringing that bringing in as much capacity as you need in habit available. It's a really efficient sort of low level kind of stuff. Oh, okay, cool. When you talk about, you know, distributed load execution, there's the whole, you know, you're pushing more and more of the processing to the edge. And of course, Brian gave a great talk about edge to outcome. You and I were on a panel with Mark Hall and Ella Hillel about the so-called power of three. And you did a really good blog post on that the power of the IoT and big data and predictive. The third is either predictive analytics or machine learning. Can you give us a quick sense for our viewers about what you mean by the power of three and how it relates to pushing more workloads to the edge and where you really Hitachi Ventara is going in terms of your roadmap and that direction for customers? Well, it's interesting because one of the things we, I wish I could, you know, maybe we have a recording of it, but kind of shrink down that conversation because it's a great conversation, but we had, we covered a lot of ground, but essentially that power of three is a lot. But we started with big data, right? So as we could capture more data, we could store it. That gave us the ability to train and tune models much easier than we could before because it was always a challenge of how do I have that much data to get my model more accurate? And then, you know, over time, right? Everybody's kind of become a data scientist with the emergence of R and, you know, it's kind of becoming a little bit easier for people to take advantage of those kinds of tools. So we saw more of that. And then you think about IoT, IoT is now generating even more data. And so as you said, you're not going to be able to process all of that. You're not going to be able to bring all of that in and store it, it's not really efficient. So that's kind of creating this, we might need the machine learning there at the edge. We definitely need it in that data store to keep training and tuning those models. And so what it does is though, is if you think about IMS, is they've captured all that data. They can use the predictive algorithms to do some of the associations between, you know, customer information and the sensor data about driving habits, bring that together. And so it's sort of this perfect storm of the amount of data that's coming in from IoT, the availability of the machine learning and then the data is really what's driving all of that. And I think that Mark Hall on our panel who's a really well-known data mining excerpt was like, yeah, it all started because we had enough data to be able to do it. So I want to ask you just sort of a, again, a sort of product and maybe philosophy question. We've talked on theCUBE a lot about the cornucopia of tooling that's out there and the people who try to roll their own.  When we talk to your customers, they love the fact that there's an integrated data pipeline and you've made their life simple. I think in 80, I saw Spark, you're probably replacing MapReduce and making life simpler. So you've curated a lot of these tools. But at the same time, you don't own your own cloud and so your own database, et cetera. So what's the philosophy of how you future-proof your platform? When you know there are new projects in Apache and new tooling coming out there, what's the secret sauce behind that? Yeah, well, the first one is the open source core because that just gave us the ability to have APIs, to extend, to build plugins, all of that, right? And a community that does quite a bit of that when there's, in fact, Kafka started with a customer that built a step, initially, right? I mean, we've now brought that into a product and created it as part of the platform, but those are the kind of things that in an early market a customer can do at first. We can kind of see what emerges around that and then go, we will offer it to our customers as a step, but we can also say, okay, now we're ready to kind of productize it. So that's the first thing. And then I think the second one is really around when you see something like Spark emerge and we were also focused on MapReduce and how are we going to make it easier and let's create tools to do that. And we did that, but then it was sort of like, well, MapReduce is going to go away. Well, there's still a lot of MapReduce out there. We know that, so it was like, okay, well, we could see then that MapReduce is going to be here and I think the numbers are around 50-50. You guys, you probably know better than I do about where Spark is versus MapReduce. I might be off, but- We had George Gilbert, he'd know. Okay. It's about right. It's about right. It's about 50-50. So you can't just abandon that because there's MapReduce out there and so it was, well, what are we going to do? Well, what we did in the Hadoop distro days is we created a adaptive big data layer that said, let's abstract kind of a layer so that when we have to support a new distribution of Hadoop, we don't have to go back to the drawing board. So it was the same thing with the execution engines. It was, okay, let's build this adaptive execution layer so that we're prepared to deal with other types of engines. I can build the transformation once, execute it anywhere. And so that kind of philosophy of kind of stepping back, if you have that open platform, you can do those kinds of things, right? You can kind of create those layers to remove all that complexity. Because if you try to one off and take on each one of those technologies, whether it's Spark or Flink or whatever's coming, that's just as a product and a product management organization and a company, that's really difficult. So the community helps a ton on that too.  Where are they trying to take you? Or where are you trying to take them? Well, hopefully a little bit of both, right? I think it's being able to take advantage of the kinds of technologies like you mentioned that are emerging when they need them. But they also want us to make sure that all of that is really enterprise ready, right? You're making it solid because we know from history and big data, a lot of those technologies are early. Somebody has to get their knees skinned and all that with the first one. So they're really counting on us to really make it solid and quality and take care of all of those kind of intricacies of delivering it in a non-orchest open source way where you're making it a real commercial product. So I think that's one thing. And then the second piece that we're seeing a lot more of as part of Hitachi, we've moved up into the enterprise. We also need to think a lot more about monitoring, administration, security, all of the things that kind of go at the base of a pipeline. And so that's an area where they want us to focus. The great thing is, as part of Hitachi Vantara now, those aren't areas that we always had a lot of expertise in, right? But Hitachi does, because those are kind of infrastructure-ish type technology. So I think the push to do that is really strong and now we'll actually be able to do more of it because of we've got that access to the portfolio. And I don't know if there's a fair question for you, but I'm going to ask it anyway. Because you just talked about some of the things that Hitachi brings and that you can leverage. And it's obvious a lot of things that Pentaho brings to Hitachi, the family. But one of the things that's not talked about a lot is go to market. And Hitachi data systems traditionally have a lot of expertise in going to market with developers as the first step, which is kind of where in your world you start. Have you seen, has Pentaho been able to sort of bring that cultural aspect to the new entity? Yeah, so for us it's actually, even though we have the open source world, it's less of a developer and more of an architect or a CIO or somebody who's looking at that later. And more and more it's the Chief Data Officer and that type of a persona. I think that as, now that we are a new entity, a brand new entity that's a software oriented company, we're absolutely going to play a way bigger role in that because we brought software to market for 13 years. So I think we've had early wins, we've had places where we're able to help in an account for instance, if you're in the data center, if that's where Hitachi is, right? If you start to get that partnership and we could start to draw the lines from, okay, but who are the people who are now looking at what's the big data strategy? What's the IOT strategy? Where is the CDO? That's where we've had a much better opportunity to get to bigger sales in the enterprise in those global accounts. And so I think we'll see more of that. Also there's the whole transformation of Hitachi as well, right? So I think there'll just be a need to have much more of that software experience. And also Hitachi's hired two new executives, one on the sales side from SAP, and one who's now my boss, Brad Suric, who's from GE Digital. So I think there's a lot of good strong leadership around the software side, and then obviously all of the expertise that the folks at Pentaho have. So. That's interesting, that Chief Data Officer role is emerging as a target for you. We were at an event on Tuesday in Boston, about 200 Chief Data officers there. I think about 25% had a robotic process automation initiative going on. That's, they didn't ask about IoT, just this little piece of IoT. So, and then Jim, data scientists, and that whole world is now your world. Okay, great. Donna Perlich, thanks very much for coming on theCUBE. Always a pleasure to see you. Yeah, thank you. Okay, Dave Vellante for Jim Kobielas. Keep right there. But this is theCUBE we're live from Pentaho World 2017. Hashtag P-World 17, brought to you by Hitachi Ventara. We'll be right back.


This is stock number 9217. We're here at Summit Automotive in Fond du Lac, Wisconsin. This is a 2015 Ram 2500 Crew Cab Short Box Laramie. This vehicle has the 6.7 liter Cummins diesel. From this HD video, you'll be able to tell that this truck is extremely clean all the way around. No denser dings on the front bumper, has factory fog lights, has a projected lamp headlamps, the LED running lights, western brown metallic is the color. This truck has Firestone Trans Force AT's. These are LT28560R20's and they are on the factory 20 inch painted with chrome pockets, rims, no scuffs or scrapes on that rim. And these front tires look like they have about 50% to 60% of the tread left. Frame and underbody on this truck is extremely clean. We shoot all of our videos in 1080p, so if you have HD capabilities on your computer, tablet or smart device, turn them on now because it's like you're right here next to the truck with me. You can see the overall quality of the vehicle without even being here. Passenger rim, absolutely perfect. And as we go down this side of the truck, you can see just how clean the body is, how reflective and glass-like that paint is. We take these HD videos so if you are far away or even if you're close by and you just cannot make the trip down, you can still see the truck here, the truck, and have confidence in the vehicle that you're looking at before you even get here. So you can make that decision to buy that plane ticket or make that drive. We sell vehicles out of state every day so feel confident that you're with the right people and that this is the right truck. Back rims are in excellent condition as well and these back tires I noted in my pre-inspection here are a little bit more tread than the fronts. These are probably about 75-80% of the tread. Frame and underbody on this truck is absolutely perfect and it has all the remaining factory exhaust so it's never been chipped or had any different exhausts or anything like that. Lower rockers look really clean. No corrosion or anything like that. Has the bed rail covers. Rear bumper is an absolutely perfect condition. No dents, no dings, no scuffs. Very very clean. Has a full towing package which includes receiver hitch and wiring. Four pin and seven pin. Can really see that metal flake in the paint here. Western brown. Really good looking color. Kind of like a root beer or chocolate. Good looking color. Has a factory spray and bed liner and I know that because on the window sticker which we have it says spray and bed liner so this is a factory one. Bed's really clean. No major dents or dings and down this side of the truck you can see is just as clean as a passenger side. I didn't see any dents or dings down this side of the truck. It has fold out tow mirrors. These are also power fold in mirrors so it's a pretty neat feature. Just press this button. There's power fold in and then they power fold out. Power windows, power locks, power mirrors, has auto headlamps. The Laramie package gives you the tan leather interior. There are no rips or tears on these seats. Both of these seats are heated and cooled. Memory driver seat, lumbar. Factory floor mats throughout can see the carpeting is very clean underneath and you can see that this truck has 24,733 miles. You get the leather wrapped heated steering wheel. Has bluetooth audio controls on that side. Cruise controls on that side. We have the 8.4 touchscreen radio with the factory navigation system. You can also do your climate controls, your heated seats, your cooled seats, and your heated steering wheel right there. You got the nav. Also the backup camera shows up right there. This one has the alpine sound system so it does have the factory subwoofer. You get 911 and assist buttons. Those are features of the vehicle. You never have to pay for them. Home link system and power sliding rear window. This one also has the power sunroof. You can see the passenger seat is very clean as well. I don't think this truck's ever been smoked in. Smells very, very clean inside.  You have your trailer brake controller right there. You've got more heated and cooled seat buns down here.  You can also turn off your backup sensors.  We'll take a quick look under the hood but first we'll check the back seats. Back seats are very clean as well. No rips or tears back here. Has the latch child safety system. So you've got your anchors on the bottom. These back seats are heated. They fold up for extra storage and then there's also a load floor here. So that's kind of a nice feature. If you have pets or boots or tools, don't want to get your seats dirty. Wreck them, put that down. Pets have a nice flat place to lay and your boots don't get the seats all dirty. You've got a sleeve in here for your hitch. Also you've got storage bin on this side. There's a storage bin on the other side. You can put ice and stuff in here. These come out for easy cleaning. So you can basically have two little mini-coolers with you. There's also storage under here. That's your winter front. But just a lot of stuff on these rams that make them pretty awesome. They went to the coil springs in 2014 for the 2500s. These things ride like a dream on the highway. 6.7 liter Cummins diesel. Engine bay is very clean, runs very smoothly. You can see everything's stock on here. This truck has been fully safety and inspected by our service shop. Has a fresh oil and filter change. All the fluids have been checked and topped off. Truck has been gone through mechanically 100% and is 100% ready to go. To see more pictures of this truck or one of our other 400 new and used cars, trucks, SUVs, minivans, Wranglers, you name it, we got it.  Videos of every single used vehicle that we have all at summitauto.com. Thanks for checking out my video. If you want to make this truck yours, give us a call right now,<PHONE_NUMBER>. Ask for one of our sales associates to make this truck yours today. Once again, that number is<PHONE_NUMBER>. Thanks for checking out my video. If you like it, want to check out more, go to our YouTube channel which is youtube.com slash summitauto. Remember to like, subscribe, and share. In fact, in a second, you'll see a subscribe button on your left. A link to more heavy duty truck videos like this one on your right. If you haven't been to our website, on the bottom, a link to this truck on our website. Click those, check us out, see what we're all about, and we look forward to helping you with the Super Clean 2015 Ram 2500 Crew Cab Short Box Laramie. Thanks again.


Artificial intelligence, AI, is a broad term referring to the use of technology to mimic human intelligence. It can be divided into two main categories, machine learning, ML, which involves computers drawing conclusions from data without being explicitly programmed, and natural language processing, NLP, which uses algorithms to interpret and understand text. ML has been applied to many areas of healthcare, including triage, diagnosis, treatment, and discharge. For example, ML models have been developed to help identify patients who are likely to develop sepsis or to predict the time until a patient will experience a severe form of pneumonia. Additionally, ML has been used to classify patients' COVID-19 severity levels and to determine the best course of action for their care. The increasing availability of EHR data and computing power provide opportunities for ML to improve patient safety, streamline clinical workflows, and reduce healthcare costs. This article was authored by Alisa J. Hamilton, Alexandra T. Strauss, Diego A. Martinez, and others.


Hello Boilermakers, I'm Roan Waller, a senior studying food science, and here's a day in the life with me. I started my day with some lovely breakfast in front of our amazing Christmas decorations. Then I headed off to my one class of the day food processing in Nelson Hall of Food Science where we learned about sustainability and food processing. I then headed to a sensory evaluation test where I got to try plant-based chicken. Stop for a midday coffee break at Copper Moon in the ABE building and then headed over to the third floor of Nelson for my favorite study spot. I spent part of my evening conducting interviews for Purdue Ag Council and then finished the night cheering on the Boilermakers in Macchi Arena. Thanks for following.


Two conference weekends down, two weekend splits for the BSU women's hockey team. For the first time this season, the Beavers kicked off a conference weekend on the road as they traveled to Duluth to take on the fourth ranked Bulldogs. First period Beavers up, pick up a loose puck in the offensive zone. Lisa Lighty takes the shot, but Maddie Rooney with the glove save and a beauty score stays at zero. A little later, just seconds left in the first. Ashley Brickalick trying to break the stalemate, but Brittany Moat makes one of her 39 saves to keep it scoreless. Then minutes into the second, Bulldogs good on the board. Katerina Mozova lights the lamp and UMD takes the 1-0 lead, but the Beavers were quick to answer.  That's her first collegiate goal. BSU ties it at one, but UMD would get one in the third. They take down the Midget State tonight two to one. A tax deductible contribution to Lakeland Public Television.


All righty, welcome back to Operating Systems Lite today. So what's going on today? All right, well, this is the last lecture of actual content. Pass that. Next one will be virtual machines and then just review sessions. So this one may is valid for the final. I forget. I wrote it a while ago, so I forget if there's a question on it, but yeah, this is the last lecture that is actually covered for the final. And guess what? The final is next week. So that's fun. All right, so we left off last lecture talking about how the kernel would allocate memory. And it wouldn't just use a free list of any old size. It would do something different. And today is that different thing. And it kind of relates a bit to Lab 6. So the allocators that the kernel uses are called buddy and slab allocators. Why would it be called the buddy allocator? Well, because it uses the idea that memory can be buddies with another piece of memory. So how it actually works is it restricts the problem a little bit. So in the general case, we have to deal with allocations of any size. And we make our blocks the same size as the request. But the buddy allocator does not do that. The buddy allocator restricts it a bit and only does allocations internally, at least in powers of 2. So every allocation that deals with internally will be something 2 to the n. So like 2, 4, 8, 16, 32, maybe 128. That's like an iNode. All the way up to page size. Maybe it's more than that and manages a bunch of memory. So why it restricts powers of 2? It's to have a more efficient implementation. And the idea behind it is it will just split blocks in two. And you know that they are contiguous because one block is a buddy. With another block, they came from the same parent. So if you need to merge them back together, merging is really, really quick because you can check because everything's aligned. Check if you and your buddy are free. If you and your buddy are free, you can merge them back into the next power of 2. And with this, with most memory allocators, once you start implementing it, the problem becomes searching to find a free block of the right size. And merging is also difficult if you implement the just most general strategy. So because we restricted it to be powers of 2 only, your implementation for a buddy allocator can use multiple lists. So since we restrict all of the allocations to be to the k, where that k could be 0 all the way up to n. And we round up, if needed, to a power of 2 if someone requests a weird size. So how the implementation would work is you have a free list for every power of 2. So there's a free list that just keeps track of all the free blocks that are size 2, a separate free list that keeps track of all of them of size 4, a separate one that size 8, so on and so forth. So if I went up to n powers of 2, well, then I need n free lists. So how this works is if you get a request for size 2 the k, so say it's like 32 bytes or something like that. So you search the free list, so you find a block big enough. So you search the free list for exactly 32. If that is empty, then you search for the next size up, so 64. And if that, there's no free blocks there. Then you search up for the next power of 2 up, which would be 128 or whatever it was, until you actually find one. If you do find one, you just keep breaking them apart, keep splitting them in two, until you create a block that is the same size of the request. And when you break them apart into two, you create a lot of buddies, so you would create more entries for every power of 2 you had to create. You would add it to the free list. And for deallocations, what you do is that coalescing. So if you free an allocation, and its buddy is also free, you coalesce or you merge them together to create a bigger free block. And you could do this recursively if you needed to to create the maximum size free block. So how would that look graphically? So say I have a buddy allocator, and it is managing 256 bytes of memory. So anything with the dark background there or the filled in background, that means this block has been split into two, and I'm not actually using it for allocations. So in total, I am managing 256 bytes, and I split that into two entries of 128. And then for each of the 128 byte entries, well, I split them each into two 64 byte entries. And for this 64 byte entry, I split it into two 32 byte entries, so this entry and its buddy. So anything that has a colored background means that it is used for an allocation. So there's a red allocation for 32 bytes, and then there would be a free block of size 32 bytes. So that would be stored in its own link list that keeps track of all of the free blocks of size 32 for the 64 byte ones. Well, there's two entries here, so there would be two entries in the free list. One's as good as any other one if you have to search for it, so it doesn't really matter. And then I have this blue or teal or cyan, I guess it's not really cyan, blue or teal, allocation of size 64 bytes. So what would happen to the buddy allocator if a request comes in for 28 bytes? How would this, what would it do? Yeah, so check the list of 32 size. So 28 is not a power of two. Got to round up to the next highest power of two, which is 32. It would check the 32 byte free list. Hey, there's a 32 byte one. So it would use that and just do the allocation right there. So we would fill in the green allocation and we'd have some internal fragmentation, right? We'd have four bytes of internal fragmentation because while we're only using 28 bytes of that 32 byte block. So what happens now if we get a new request for a 32 byte block of memory? Yep. Yeah, so we would search for exactly 32. Right now there are no free 32 entries so we go to the next power of two up, search for 64. This one doesn't really matter what you pick. One is good as any other. So we would take this 64 byte block, split it into two 32, and then we would use a 32 for the allocation and then put another 32 back onto the free list. So it should look like this. So now we have our purple allocation and we have one 32 byte block free and one 64 byte free. So what's going to happen now if I free that 64 byte block? So this one becomes free. Is this buddy also free? Yeah, so it's buddy is also free. The one that was allocated beside it, they both have the same parent. So we could create an even bigger free block. So we could create a 128 byte free block out of those two and we can't go the next level up because while we're using part, we're using the other buddy. So at the end of the day, we would be left with this after all the allocations. So any questions about that? That is the buddy allocator in a nutshell. All right, probably be quick today. So yeah, the buddy allocator, that is one of the primary allocators that are actually used internally by the Linux kernel. Advantages, well, it's fast and simple compared to just general dynamic memory allocation where we just have a free list and we have to search it every single time and there's just one free list and coalescing is really hard. Searching it is really slow. So this is a much better approach for that. The only disadvantage is that we have powers of two and we can only do that. It also kind of avoids external fragmentation because it keeps things contiguous because while it only has, you have buddies, so it's a lot easier to keep addresses contiguous if you're doing that approach. You can get into the same case where you do have fragmentation but it's at least a lot better with the buddy algorithm. The disadvantages is with this, if you don't have powers of two allocation, there's gonna be some internal fragmentation you just can't handle because while you only deal with powers of two internally because you always round up. So the last one, which is kind of related to lab six, so this lecture is built to be really short. So I will be here and I suggest you work on lab six if you haven't done it already and I will be here for any questions. So the last option is a slab allocator and that further restricts it. So for a slab allocator, all your allocations are all the same size. So how that works is you just allocate essentially a giant array, giant pool of objects and since everything is of the same type and of the same size, well then one is as good as any other and you can just keep track of whether it's allocated or not by just having essentially a giant array of bits that will keep track of each slot. It will be zero if it's unallocated and one if it's allocated or used. So this wouldn't have any internal fragmentation because you assume that everything is of the same size. So you don't have any of the fragmentation issues. One slot is as good as any other slot. So you will find this with lab six because you essentially have a giant array of inodes and then your inode bitmap keeps track of what inodes you are actually using. So it is actually an example of a slab allocator for your file system. So you can think of the slab as just a cache of slots or elements in an array if you want. So each allocation has a corresponding slab like a slot or an index, however you want to think of it and one slot is just one element of the array. It's one allocation. So instead of having a link list, we're essentially just using arrays and that's what a bitmap is. So there's just a bitmap. So for each slot, you just have one bit, keep track if it's allocated or it's not. And if it's trying to allocate memory generally or allocate one of the elements of the array, for allocations it's really quick. All you do is search the bitmap until you find some byte that has a zero in it. If you find one that has a zero in it that means that slot is free. You can go ahead, use that slot and then you just flip the bit from a zero to a one to indicate you are now using that slot. And then for deallocations it's also super fast. You just figure out what bit you need to change from a one to a zero to indicate that it is now free. And then you may or may not actually clear the memory for that and set it all to zero. Typically computers will not do that and kernels won't do that because it's slow to do that. It's faster just to mark it as unused because next time you use it you'll probably just overwrite the data anyways. So what's the point of wasting time setting it all to zero? And the slab, well it can be allocated on top of a buddy allocator on top of any general one. So typically if you really care about performance and you are creating thousands and thousands of objects of all the same type, typically what people will do for performance is essentially create a slab allocator, just ask malloc for a bunch of memory and then implement a slab on top of that and just have a bunch of, just treat it as a giant array where they just keep track of if a slot's allocated or not. So it could be on top of a buddy allocator, it could be on top of malloc. Typically when you implement this sometimes it's called an arena, sometimes people will call it an arena if they're implementing their own memory allocators. But once you start caring about performance you will probably implement something like this or use something like this. So if I have like two object sizes A and B what I might do is, well I get slab one or A one from the buddy allocator and I might put like four objects of type A there and then I ran out so I need to ask the buddy allocator for more memory. So maybe I get this block and I call it slab A two and I put four more A's on that and I'm wasting some space within the memory I got from the buddy allocator. So say each of these like I got, I don't know, 128 bytes from the buddy allocator and each of these is some weird number like 30 bytes or something like that, 28. Some number that doesn't quite fit. Then maybe I have bigger objects of size B that fit exactly within a block. So this one holds two B objects and B one. This slab for B two also holds two objects. So in this case, well you might notice that instead of having slab A one and A two I could have just used that entire space of memory and just have one giant slab of A objects and maybe I could have reduced my internal fragmentation instead of fitting in eight elements there I could have fit in nine. So you might want to do something like that as well if your sizes don't match up but typically you probably wouldn't wanna do that because that's again moving memory so you might make some addresses invalid and need to make sure that you update all the addresses and use all the newly updated addresses which we already have enough memory problems and if you're doing that in a multi-threaded application well guess what then you have to think about data races and all the fun stuff we know and love. So any questions about these memory allocator stuff? Yeah, so it's a bit weird. So typically you have like talk about external fragmentation probably from the buddy allocator that allocated these slabs and for that it wouldn't have any external fragmentation because it only deals with powers of two and just has internal fragmentation. So that's probably what you would consider in this case. All right, so rewinding to lecture 28 we ran out of time for this slide so now we have ample time so we can talk about journaling file systems. So remember back when we were trying to delete files there wasn't actually a delete. It was just RM is just an unlink so it just gets rid of the name to the inode association. So if you run out of, if there are no more hard links to an inode then you can actually delete it. If you start doing lab six you'll see the steps in order for the kernel to actually delete and get rid of a file there would be a three step process so it would have to remove the last directory entry so that gets rid of the name to the inode which case no one's using this inode anymore. So as the second step you can essentially free this inode back to the pool of free inode so that would require changing the bitmap that keeps track of whether an inode's allocated or not so in this case it would change that inode from a one to a zero since it's not being used anymore and then we would also need to free any of the disk blocks that that file uses because well we're not using it anymore. So if it used physical block six, eight and 10 for instance we would have to set those as freed as well but because this is all on a disk that is persistent well all of these operations I could yank out the power cord at any time assuming it was a desktop or I could lose power or run out of battery or something like that so between step one and two if I run out of battery then my file system is inconsistent so I would have nothing pointing to this inode but in the file system it would still be marked as used and it would be kind of lost forever so you might get in that situation and you want to be able to recover from it but in this case if you don't do anything it's really really difficult to recover from it unless you check the entire file system you walk every name and see oh nothing actually points to it so that inode is actually free I should probably fix it up or there's some inconsistency. So that is where journaling file systems come in so what a journaling file system will do is it will record in the log on a file like on the physical drive itself before it starts doing these three steps it will make sure it writes out what it's about to do so it will probably write that hey I'm about to delete this inode and then after making sure that that is on the disk then it would start doing these steps and if it crashed somewhere in between one of these steps makes it a lot easier to recover because you would just check the journal you would see oh you started deleting this file and you did not complete it yet so I need to see if just this is consistent and I can walk through just that instead of the entire file system and go ahead and clean it up. If however you know the three steps actually happen they're successful you would just remove that entry from the journal you have successfully completed it so you don't need to check it anymore so any questions about journaling file systems? So yep so if it crashes in the process of journaling then it would be like you never started deleting the file so it wouldn't have started anything so that file it would just look like you just never accessed the file or you haven't deleted it yet so it would be at some consistent state before anything happened so it just journals like the whole step so it just says I'm going to journal I'm going to delete these three things which implicitly means it's going to follow these three steps so in this case like deleting a file is these three steps so the journal would just say what's the bet to do so right to the journal I'm going to delete this file and then before it even did step one it would make sure that I'm going to delete these files in the journal in the middle of which step? So in the middle of doing this? Yeah so it would just say okay well the journal it's kind of like a to-do list so it's like I'm going to delete this file right so it would write that in the journal and it would be undone then it would start trying to remove this directory entry and maybe it removes the entry and then power gets lost oh if power gets lost like it just journals and then power is lost then well the file system would probably still be consistent but has a pending operation so it would probably just do that operation so if the journal said I need to delete a file and it's not done yet that's essentially a to-do list so as soon as you have power back again it would check the journal it would say oh I need to delete this file oh I didn't even do step one well I'll start at step one and go all the way to three and actually delete it so you might notice that when you format stuff especially on macOS will be like the file system and then the journal file system and the default option is a journal file system so this is what they're talking about when it says journal file system so it just means your file system will be consistent but it's also a little bit slower because you have to write to a journal before you do anything but typically it's fairly fast and it doesn't really matter all right any other questions before I guess we work on lab six all right so today we saw even more memory allocations so the buddy allocator is what most kernels use for most memory allocations of course it can get complicated for other things but buddy allocator is one of the most commonly used one it's a real world restricted implementation so it only deals with powers of two and uses the implementation when you use multiple link lists and then in the other case if we know everything is of the same size then you use the slab allocator slab allocators are used all over the place in fact it will be used in lab six if you haven't started yet because everything's of the fixed size so you don't have any fragmentation so with that just remember pulling for you