A warm welcome to the fifteenth lecture on the subject of wavelets and multirate digital signal processing. We had raised a few questions in the previous lecture which we set out to answer in this lecture. The questions pertain to the fact that we were building more than one multi resolution analysis and specifically the question was what are we looking for when we go from one multi resolution analysis to the other. Why cannot we be content with the hard multi resolution analysis? We have been singing the praises of the hard multi resolution analysis so frequently in some of the previous lectures. We have been saying that it tells us most of the things that a multi resolution analysis does. Why then should we look for others? What is inadequate in the hard? Well, if you remember when we began this course we set out to do something which a basic course on signal theory or signals and systems does not namely look at two domains simultaneously. We have been trying to do this for a while. So, when we took the hard we did at some point talk about the filtering aspect of it. We also talked about the ideal to which we strive in the filter bank and if you recall that ideal had to do with the frequency domain not the time domain. So, somewhere we have not been giving as much importance to the other domain so to speak as we should. We have been working in the natural domain of time, but we have not quite been doing justice to the frequency domain. When we take a hard multi resolution analysis what does it do in the frequency domain? We have not quite understood this as yet and the first thing that we would like to do is to answer this question. What does the hard multi resolution analysis do in the frequency domain and then how do these filter banks that comprise the hard build up to that frequency domain behavior? So, let us set out first to answer a more basic question. What is the Fourier transform of the scaling function and the wavelet function in the hard multi resolution analysis? So, we shall call today's lecture a joint perspective time and frequency and we shall set out first to look at the Fourier transform of phi t and psi t in hard. So, the hard m r a let us begin with the phi t and let us look at its Fourier transform. Remember how phi t looked this was phi t let us obtain its Fourier transform. We shall use this notation for the Fourier transform a cap on top and capital omega to denote the angular frequency and a log angular frequency radian or angular frequency and this is easy to calculate. Well, I am saying 0 to 1 actually I should write minus if you like I can write minus infinity to plus infinity, but the non-zero part is only between 0 and 1. So, it is also alright to write between 0 and 1 only for this specific function and this is of course, equal to integral 0 to 1 e raise the power minus j omega t dt which evaluates to e raise the power minus j omega t by minus j omega from 0 to 1 and that is 1 minus e raise the power minus j omega by j omega and we can simplify this. We can take an e raise the power minus j omega by 2 common in the numerator and that is easy to interpret. In fact, if you wish we can even make this by j omega alright 2 and then this becomes e raise the power minus j omega by 2 if I take the 2 j and this together I get sin omega by 2 there and omega by 2 here. So, this is the Fourier transform. Now, let us look at the magnitude of this Fourier transform. So, in fact, I could straight away sketch this it is essentially the magnitude of sin omega by 2 by omega by 2 a sketch would look like this. This is a very familiar function to most electrical engineers we call it the so called sink function the sink pattern people have different names for it they call it the sampling function the sink function and whatever other names. Anyway, this is the point where omega by 2 is equal to pi or omega is 2 pi and this of course is the point where omega is 4 pi and this where it is minus 2 pi this where it is minus 4 pi and so on so forth. At this point the magnitude takes the value 1. In fact, recall that the magnitude of the Fourier transform in fact, the value of the Fourier transform at omega equal to 0 is indicative of the area under phi t. So, the integral of phi t over all t anyway so much so for the magnitude and now we know what to do when we compress and expand. Let us look in general at the Fourier transform of phi 2 t minus m so we should interpret that you know we are talking about dyadic dilates and translates. So, let us consider phi in fact 2 raise the power of m t minus n if you please for m and n belonging to the set of integers. Now, you will recall that there is a very simple result in the Fourier transform which says that if phi t has the Fourier transform phi cap omega as we do in general then phi alpha t has the Fourier transform 1 by mod alpha phi cap omega by alpha for all alpha belonging to the real numbers other than 0. This notation says all real numbers except 0 of course alpha cannot be 0. So, for example, even if alpha is negative we can use this in particular for example, if alpha is minus 1 we have a deflection of the Fourier transform as well. So, using this we will now take care of the two so called distortions or modifications that we made in phi the translation and the dilation. In fact, the translation does not affect the magnitude the translation only affects the phase or the angle of the Fourier transform. So, I can even forget about the translation I need only look at the 2 raise the power of m term there. So, here if I restrict myself to magnitude only then phi t has the Fourier transform phi cap omega with the magnitude of mod of this phi 2 raise the power of m t minus n would then have a Fourier transform mod phi cap omega divide by 2 raise the power of m of course with a constant the same 2 raise the power of m, but in the denominator here. So, of course let us take an example suppose m is equal to 1 and minus 1 to fix our ideals and n equal to minus 1. Now, notice that the n is entirely absent here or here the n is irrelevant as far as the magnitude goes the n only contributes to the phase. Let us catch both of these how would these look. So, let me take the m equal to 1 case let me sketch this. So, omega by 2 so if I focus my attention on the main lobe and the principle side lobes that I have here this was originally 2 pi now it is become 4 pi this was originally 4 pi now it is become 8 pi and so to on this side this is minus 4 pi and so on. So, you are stretched by a factor of 2 let me put them together the original Fourier transform and this Fourier transform with m equal to 1 and so on this is I am just sketching mod phi cap omega and here I sketch mod phi cap omega by 2 multiplied by half of course this should be smooth here all these are smooth. So, as expected when we squeeze in time we have stretched in frequency and now let us interpret what we are doing when we take a dot product to calculate the coefficients in a V m to calculate coefficients in a subspace V m what are we effectively doing we are taking an inner product namely the inner product of an x with phi 2 raise the power of m t minus n remember now you know there is of course a normalization here. So, if you want to work with an orthonormal basis then it should not quite be phi 2 raise the power of m t minus n one must normalize it to make it unit norm. So, I will take an instance let us take any arbitrary m and let us look at the norm. So, if we consider phi 2 raise the power of m t and again the minus n does not affect the norm I am talking about the L 2 norm and therefore, we can as well take without loss of generality we could take n equal to 0 let us find out the norm therefore. So, I am putting 2 raise the power of m and then a dot dot denotes the argument of the function but we are treating the function as an entity. So, I do not use the explicit argument here the norm of this is essentially integral now you know when you go over phi 2 raise the power of m dot you are talking about 0 to 2 raise the power minus m here 1 square d t and that is obviously 2 raise the power minus m and therefore, if you want to make this unit norm then you must divide by this is of course, the square of the norm. So, if you want to make this unit norm you must divide the function by the square root of this we must consider 1 by square root of 2 raise the power minus m times phi 2 raise the power of m t minus n and that is easily seen to be 2 raise the power minus m by 2 phi 2 raise the power m t minus n. So, this is the unit norm this is now an orthonormal basis.  So, it is that m th subspace in the ladder and now we have an orthonormal basis for it anyway. Now, let us interpret what happens when we take the dot product of any function in l 2 r with an element of this orthonormal basis. So, consider x t belonging to l 2 r or if you please x with the argument belonging to l 2 r as is the correct way to write it and then consider the inner product of this x with this orthonormal basis element 2 raise the power minus m by 2 phi 2 raise the power m dot minus n. Now, we are going to invoke the passables theorem you will remember that we had discussed the passables theorem a while ago in one of the earlier lectures. When I had talked about the relationship of functions and vectors I had mentioned the significance of passables theorem there are different ways of stating it passables theorem on one hand says that the inner product is preserved as we go from time to frequency. Now, here if we use angular frequency a factor of 2 pi is needed if we use Hertz frequency that 2 pi factor is not required, but since we are working with angular frequency it would be safer to retain that factor of 2 pi, but that factor of 2 pi apart what passables theorem says is that after all when you go from the function to its Fourier transform in effect you are representing the same function in a different basis. You are representing the function with respect to the basis formed by the rotating complex exponentials or the phasors for different frequencies omega. The Fourier transform is essentially a projection of a function in this case a function in L 2 r on a particular element of that basis a particular rotating phase. The inverse Fourier transform reconstructs the original function from its components. It is worth recalling some of these points finer points because it helps for them to be firmly embedded in our consciousness in a course like this. So, even if it means a little bit of repetition let us emphasize those points again. What we said was if I took the Fourier transform so if I have x t and its Fourier transform x cap omega so to speak then x cap omega is essentially a projection x t e raised to the power minus j omega t d t a component of x t along e raised to the power j omega t. So, this is the Fourier transform and the inverse Fourier transform this is. So, let us write that down this is the Fourier transform and we have the inverse Fourier transform which reconstructs x t from its components with a factor of 2 pi here and you will recall the interpretation that we gave this. We had said that essentially this is the component along a particular omega and this is the so called unit vector with the factor of 2 pi. So, if I take this and this together it is like a unit vector here and what we are saying here is the original function is essentially the component multiplied by the unit vector integrated over all the components reconstruction of a vector reconstruction from components that is the interpretation. This background what we have said impassable theorem is the following the inner product of x t and y t. So, you know you are talking about two different domains is equal to the inner product of x cap omega and y cap omega, but with a factor of 1 by 2 pi in the language of components what is the interpretation? The interpretation is that it in calculating the inner product it does not matter whether one is using one orthonormal basis or another the result is the same. The inner product has nothing to do with the choice of basis the inner product between two vectors remains the same whatever basis we choose to express the vectors that is the statement being made in passable theorem here. We are saying represent the functions in the natural basis of impulses or represent them in its Fourier basis the inner product is the same of course, to within a factor of constant this constant appears because of the angular frequency radians radians per second I mean otherwise if you were to take Hertz frequency this factor would be absent as well. Anyway this was an important result that we had seen when we looked at functions from a perspective of vectors and now we shall use this passable theorem to interpret this idea of projection onto V m. So, when we consider the inner product that we were doing a few minutes ago the inner product of x with phi 2 raise the power m t minus n and now if you wish to make it orthonormal then introduce the orthonormalizing factor to raise the power m by 2 this is going to be equal to 1 by 2 pi times the Fourier transform of each of them. So, the Fourier transform of x of course, with its argument the argument remember is going to be omega here and I shall write here the Fourier transform of this whole thing. So, please do not misunderstand what I am writing to be the expression itself, but understand it to be the Fourier transform of this. So, you have 2 raise the power of m t minus n, but take the Fourier transform. So, I am saying Fourier transform of this whole thing here I would like to interpret this graphically first what are we going to do when we take the inner product in the Fourier domain. So, what is this going to look like we have just seen the magnitude we have just seen the magnitude of this quantity and that looks like this. This is the place where you have 2 pi multiplied by 2 raise the power of m. So, in effect you have expanded that main lobe and all the side lobes by a factor of 2 raise the power of m in particular with m equal to plus 1 you have expanded by 2 if m is equal to minus 1 you would have contracted it by a factor of 2 and so on. So, when we take the inner product in the inner product x cap and the cap of this what are we going to do we have the Fourier transform so let us let us understand it graphically as I said. Let us say this is the Fourier transform of x whatever it be I mean let us understand in the magnitude sense first. So, let this be the magnitude of the Fourier transform of x this is 0 let us say here and this is the magnitude of the Fourier transform of the other argument I am showing only the main lobe and a part of the other side lobes when you multiply them their magnitudes are going to get multiplied and therefore, essentially you are going to extract this band. So, to speak in a notional sense we are going to emphasize most the area of the Fourier transform around the main lobe of phi you see after all it is the magnitude which plays a significant role here when we take the dot product we are going to multiply the two Fourier transforms and integrate overall frequencies where the magnitude is larger the contribution will be larger where the magnitude is smaller the contribution will be smaller. So, the side lobes would kind of suppress that part of the Fourier transform and the main lobe would emphasize the corresponding part of the Fourier transform contained under the main lobe what we are saying is essentially that part of the Fourier transform of the original function x which is contained in the main lobe is emphasized as against all the rest in calculating the area. Now, this also gives us an interpretation of what happens when we increase or decrease m in frequency. In fact, let us look at this drawing once again this point is 2 pi multiplied by 2 raise the power of m. So, take for example, m equal to 0 you are essentially this dotted line here is the magnitude of the Fourier transform of the appropriate dilate of phi and of course, we do not care so much about the translate the effect of the translation is only to change the angle and it does not reflect in the magnitude. So, for m equal to 0 we are essentially emphasizing a region of the frequency axis broadly speaking between minus 2 pi and plus 2 pi when we take m equal to 1. We are emphasizing a region between minus 4 pi and plus 4 pi when we take m equal to minus 1 we are emphasizing a region between minus pi and plus pi and so on so forth add infinitum. What are we saying when we increase m m 1 2 3 and so on we are effectively keeping more and more information around the 0 frequency we are broadening it of course, we are narrowing in time, but we are broadening in frequency. So, we are keeping a larger band of frequencies, but all around the 0 frequency. Now, what happens when we consider psi t that is an equally interesting interpretation let us do that. In other words here we have analyzed the implications of taking the projection on one of those subspaces in the ladder. What happens when we project on one of the incremental subspaces that is also an interesting question. So, for that we must first consider the Fourier transform of psi t and that is easy to do psi t as you know is phi of 2 t minus phi of 2 t minus 1 and you can easily find its Fourier transform psi cap omega is therefore, going to be equal to well when you multiply by 2 here you would be dividing by 2 in the other domain and of course, here you need to take care of the minus 1. So, e raise the power minus j omega times the same expression we can evaluate this. So, you know it is I think maybe we should it would be better to come to this little more systematically. So, phi 2 t would have the Fourier transform phi cap omega by 2 multiplied by half. Now, phi of t minus n would in general have the Fourier transform e raise the power minus j omega n times phi cap omega and when we consider phi of 2 t minus n in general. So, here we are going to replace t by 2 t we should take this replace omega by omega by 2 and then multiply by half. So, this replacement must be in the whole thing and therefore, phi of 2 t minus n would have the Fourier transform half e raise the power minus j omega by 2 times n phi cap omega by 2 and therefore, we need to make a little correction here. The correction is we need to replace omega by omega by 2 here. So, in fact, let me rewrite that part of the expression it is more convenient. Phi t which is phi 2 t minus phi 2 t minus 1 has the Fourier transform phi cap omega by 2 minus e raise the power minus j omega by 2 times phi cap omega by 2 and if we aggregate terms we have phi cap omega by 2 and this is easy to write this is half well we can play the same trick we can extract and e raise the power minus j omega by 4 common from here and get 2 j I am skipping a couple of steps sin omega by 4 and we know the expression for this. This is essentially 1 by 2 e raise the power minus j omega by 4 times well I will continue on the next page it is a little complicated, but not too difficult sin omega by 4 divided by omega by 4. In fact, let us multiply all this and put it together. So, I have e raise the power minus j omega by 2 coming together j there sin squared omega by 4 divided by omega by 4 and a factor of 2 remains outside. What we need to focus on here is essentially this part this is important the rest of it is not because it is this that really affects the magnitude seriously the rest of it is essentially a constant magnitude. So, let us look at the magnitude of sin squared omega by 4 by omega by 4 first let us look at sin omega by 4 by omega by 4 that of course, would look like this you see this is the point 4 pi minus 4 pi and so on and then on the same graph I will draw. So, this is mod sin omega by 4 by omega by 4 and on the same graph I will show in dotted mod sin omega by 4 once again. So, that is going to look like this it is going to have a maximum at pi by 2 and then it is going to see this omega by 4. So, one cycle will be completed when omega by 4 equal to pi that is omega equal to 4 pi. So, one cycle being completed here mod sin is going to look like this. So, in each span of 4 pi you have one half cycle being completed right. So, this is one half cycle now in the next span of 4 pi again you have one more half cycle being completed like this this is the situation. Now, now look at the situation this is this was the 0 frequency here the solid line has a maximum at 0 and it tapers off up to 4 pi on both sides the dotted line reaches a maximum in between at 2 pi when omega by 4 is equal to pi by 2 all right. So, omega is 2 pi this point the maximum here occurs at 2 pi and therefore, when you multiply this decreasing function by an increasing function here there is going to be some point of maximum in magnitude somewhere in between 0 and 4 pi and it is going to taper off to 0 at 4 pi again. So, let me focus in you see in the other side lobes it is easier to understand. For example, between 4 pi and 8 pi it is very easy to understand you are multiplying essentially two similar looking functions and. So, you can see there is going to be a maximum somewhere it is going to taper off and it is also clear that the maxima in the other side lobes are going to be weaker in the than the maxima in the main lobe here all this. So, when you multiply the dotted function and the solid function you are going to get a pattern something like this which I now sketch something like this is going to of course, be mirrored on this side and then this way and this way and so on. So, this is mod sin squared omega by 4 by omega by 4 as a function of capital omega. Let us take a minute to look at this when we take an inner product and inner product of a function x t now here I am writing t to explain that the inner product is in time with of which say psi of t minus n in the magnitude sense what are we doing we are first multiplying the Fourier transform of x t with the Fourier transform of psi t minus n this should be understood to mean the Fourier transform of psi t minus n a cap on the whole function. So, when you multiply them and then integrate that part of the Fourier transform of psi t minus n which is significant in magnitude is going to be extracted out of the Fourier transform of the original function x and what is that significant part let me put it back here for you it is therefore, going to extract give emphasis to a band now not a band around 0 a band around some other frequency here of course, this symmetry is always symmetry in frequencies. So, you could focus on the positive side of the frequency axis, but what is going to be done is to emphasize a band here and instead of psi of t if you take psi of 2 t for example, so if you take say instead of w 0 if you go to instead of w 0 go to w 1 in which case you would have psi of 2 t minus n and how would the Fourier transform of psi of 2 t minus n look. So, if you take the magnitude if you take the Fourier transform of psi of 2 t minus n and plot its magnitude its appearance would be something like this it would be stretched. So, instead of going from 0 to 4 pi now you would have a band between 0 and 8 pi of course, symmetrically on the negative side now there are 2 things you see you must keep in mind that unlike the case of phi in psi we have 2 changes taking place. One is that the band expands the main lobe expands and each of the side lobes expand, but the second is that the center frequency the point where this was a maximum here you see the center frequency also doubles both the band and the center frequency double. So, that means now you are emphasizing a different band the center frequency is different each time you go to a different may different w you take w 0 you are emphasizing one band when you take w 1 you are doubling the center frequency. So, you are emphasizing a different band and of course, the band itself has doubled also when you take w 2 you are again doubling the center frequency. So, you are emphasizing a different band and again the band is doubled. So, each time you are doubling the band and you are doubling the center frequency. So, in a certain sense that idea of complementarity you know each time you put one incremental layer you are putting one more band and the band size is doubled here. Now, where in all this is our discontent with the half why are we discontent the discontent is because even if we say we are emphasizing a band it is only true to a certain extent. Let me put back this Fourier transform and illustrate we might say to an approximation that we are emphasizing this band, but that is only approximate. We are also to some extent keeping this band and this and there of course, the negative corresponding pieces and that is where we are not content with the half. We want to keep a certain band focus our attention on it we do not want interferences from the other side lobes that are there and in going to other multi resolution analysis we are essentially trying to reduce that unwanted presence of the side lobes as much as we can. We have given a feel of what our discontent is like we shall build on this further in subsequent lectures. Thank you.


Lori Netherow with Buckeye Valley Beef and today I'm going to talk to you all a little bit about marketing project that we took on for our SAIR grant and I'll even introduce to you if you're not familiar with the SAIR grant a little bit about what it is and how to apply yourself if you find interest in this topic. So we're going to go through a little bit about who Buckeye Valley Beef is, our North Central SAIR grant in the process to apply the study that we got our funding for, the findings of our study, the lessons that we learned, and then what you can do in your individual operations or at your family homesteads to implement some of the strategies that we talk about today. So as I mentioned, I'm with Buckeye Valley Beef and Buckeye Valley Beef is an agricultural cooperative that Ohio's Extension Office helped us create back in 2016. One of our members had an idea to collaborate with other small farms so that we can provide a year-long product to customers instead of just you know once a year or a couple times a year and that maybe we can even out be a resource to larger operations like restaurants and grocery stores. We do provide our beef to some local butcher shops. We've collaborated with some restaurants including one in Columbus and so I'm excited about what the future holds for the cooperative. And here is a short introductory video that we had created for our co-op to give you a better idea of who we are. Buckeye Valley Beef was started in 2016 by a few local farmers here in southern Ohio. We wanted to find a way to sustain our living on the farm as well as provide a product which we thought was as a high quality to consumers. The owners of Parish's family about forming a cooperative is a way to provide a consistent product year-round and the only way that we can do that is together. We feel like we have a great product that stands out because we're taking those core values that our parents and grandparents taught us and trying to perfect them to adapt to today's food market. One way we're able to adapt to today's food market is through direct marketing which allows us to connect with each customer and build relationship. Our dream of Buckeye Valley Beef is to not only grow our individual operations but to grow the cooperative in a way that strengthens our local food community. So as I mentioned we are an agricultural cooperative and you know as with any business venture a cooperative doesn't come with any shortfallings. We have minimal working capital because most of the money that we make in the cooperative goes back to the farmers. We take a small fee from each of the cattle sold to pay for advertising costs working things like supplies and shipping, utilities, any of those types of things. So we don't have a whole lot of money to waste on anything and when we first started nobody knew who we were and we didn't know how to get our name out there so we wanted to find what was most productive for small farms and cooperatives and as you'll find if you start doing research in this area you won't see a lot of research done by small operations and small family farms or agricultural cooperatives because there's just not a lot of us. So we wanted to look into what advertising methods worked for our types of businesses. You know we're different than you know a carpet outlet or a Walmart McDonald's. We may not work the same way or attract customers in the same manner so we wanted to do an experiment and find out what works so that going forward we can make sure that our very limited resources are put to good use and that was the point of this grant proposal that we submitted to North Central Sare. And I see a comment about the video. One of the things that's outside of the scope of this presentation but one of the things we've also learned since the implementation of this grant is that video advertising, YouTube and those types of outlets are another great resource and with the invention of TikTok now to reach out to or building a customer base. Maybe somebody else can do a Sare grant to look into that. So the North Central Sare grant was actually presented to us by one of the South Center members for Ohio Extension and they had mentioned that there was a grant process out there that that gives money to farmers and ranchers who are looking to do research and education on sustainable agricultural products and so one of the areas that you can research in is marketing and that's the area that we chose to go and we are a group so we went for the grant that's up to 27,000 but if you're an individual operation you can apply for up to 9,000 or a team of two is 18 and that data may be a little different this year that was taken from the 2021 information so it may increase each year when you go to check it out just looked into that. They fund about 50 projects a year and I will tell you this was not our first attempt. We applied the previous year before we were actually accepted and we went through the peer review process and took their notes seriously and then we revamped our proposal the following year and that's when we were awarded the grant and the timeline they do a call for proposals in August and by early December the proposals are due and by February they've made their decision and in the spring you can start your projects and receive funding for those projects and then this was a two-year project for us so we got some money up front to start the project and then as we continued forward and submitted progress reports we continued to receive more funds and then at the end a reimbursement so you know think about those things if you're interested or if you have an idea yourself they welcome any and all types of research or educational proposals so look into their website to get more information so our funded grant project like I have mentioned is to look at advertising channels for small family farms or agricultural cooperatives and we introduced four advertising methods we did bill boards radio ads social media and google ads and as we get to those slides I'll talk to you a little bit about why we chose those methods and then we measured the impact that those advertising methods had on our sales our customer reach and our self-reported exposure and then we share the information with you all so here's the timeline that we have so we started off with google ads and we did that from April to June of 2019 and then we wanted a break between each implementation so that we could account for any lagging sales that might be attributed to a previous implementation so we took a break in june in July we reset accounted for any lingering sales and then collected baseline data for august and then in august we implemented our billboard display until october took a break in november reset and you know created or gathered our baseline data and then in december we did our facebook ads up until march and as you'll see from march of 2020 to june of 2020 we started the pandemic so um some of that data may be a little skewed in terms of covid-19 um impacting our sales growth but we also will talk a little bit about why we think that might be a perfect natural experiment for us and then in april we implemented radio advertisements so google ads we chose google ads because we've been to prior trainings we went to the beef 509 course at ohio state university and one of the beef checkoff members that does their marketing there talked about how they've kind of drifted a little bit away from facebook advertisements to more youtube and google ads instead of you know social media we thought that was interesting because it seems like a lot of small businesses focus a lot on their social media accounts making sure that they've got a public image out there so that people see them and know who they are uh so we also hadn't really worked with google ads much we did a little bit and it did seem to have an impact on our sales so we decided why not throw that in there and see if we can compare contrast the two advertising methods so we have a budget of 3200 dollars approximately um and if you're not familiar with google ads we weren't we there are two types uh and they're there may be more now but when we did our google ad campaign there were two types there's display advertisements which are those that pop up on your newsfeed when you're scrolling through or if you're on a blog and you see advertising on the side of it those are display ads and they're there they're created from the the cash from your computer um looking at prior browsing history things that might interest you so let's say I am on a food blog you're going to see advertisements for like Hello Fresh or those types of companies that are targeting people who are foodies and then there are search campaigns which are the campaigns that we actually did and that's when you're bidding on keywords for um your opportunity to pull up at the top of the list on google so let's say you get on google and you type in local beef near me that is um keywords that people are bidding for and if you are at the top of that list if you have a high bid you're at the top of the list on google and people will find you when they type that in so we used two search campaigns we targeted our small bundles and then our direct to door deliveries and then the other we used to target our bulk beef sales so if you implemented these you just kind of look at what it is that you're trying to sell to the customer and maybe do a search campaign for each individual type product so each month that we did this we would reevaluate and adjust our keywords almost always you have to do this because nobody unless you're an expert have a degree in marketing and even then I still don't think you get it right the first time around get your words right at first we we talk about our product in a more professional we use more professional terminology whereas our customers may not so you know we might search for beef shares or um I'm not even sure what else to say for that piece but our customers will search like whole cow for sale um or you know you might be searching for cars for sale but your uh you salesman would think that you're searching for you know more specific terminology for that so you might need to readjust your your words that you're targeting to match what your customer is actually using and we kind of listened when we went to our we had craft shows that we would have our product at for sampling to customers and we would listen to the words that our customers were using to describe our product and use that to figure out what types of wording we would need to to bid on on google and then we used google analytics to track the progress and it's a beast in and of itself and it took a lot of time for us to learn how to use both google ads and google analytics so our google ad results were not like we'd hoped them to be this is our first advertising method we were extremely excited to get started and then we looked at our data and we had 15 customers self report exposure to the google ads so not a whole lot we had um about $4,800 in sales attributed to those customers which is a 50 return on investment um you know it's not horrible but it's not what we had hoped when it it takes us a lot to get the $3,700 that we invested to you know only return 50% to us so we weren't we weren't very excited about those results and and also all the time that we had to put in to get those results so that was our first implementation so we reset we gathered our new baseline data and then we went about doing billboard displays and the reason we chose those is because we had actually done yard signs prior to this and we had them in various customers yards any time somebody bought from us we'd asked them if they wanted to put a yard sign in their in their yard and it works for some types of companies you have black top companies ripping companies you know house clean companies they utilize those yard signs and and they work but do it does it work for an agricultural based company and does it work on a large scale we didn't know so we wanted to test this out and see how it would work for us um we chose Lamar companies because it's a big advertising billboard company out here in our area in Cincinnati and they were great to work with they helped us to design this billboard we're farmers we're not marketing gurus so we are you know graphic design artists we we were not able to do this on our own so they were able to do this for us and we had a general idea of where we wanted to put our billboard but we weren't confident and so they had their marketing experts look at our data and then and kind of solidify for us where it needed to be placed so we ultimately decided to put our billboard above a major artery highway in Loveland, Ohio which was a good area for us because it targeted a lot of the people who are kind of already starting to hear about Bucca Valley beef and buy from us so we had a six thousand dollar budget and prior to our proposal to get this grant we looked into all of these advertising methods and kind of looked at how much money did we realistically need for each of these I would have um severely underestimated the amount of money we needed for a billboard prior to doing research on how much they cost um this was one billboard um and it was six thousand dollars so a very expensive advertising method but if it works it works so and it wasn't our most expensive advertising method that we used in this in this grant and we also used our order form to track self exposure to the billboard because it's not really something that's easily trackable other than asking the customer if they could tell us where they heard about us so that's the method that we used to track we also did use Google analytics for this type of method to see if there was any increase in website traffic to from the Loveland area and there was so that was that was nice to see but can we convert those website traffic people to sales that was the question so um when we finished our three month period of time we had no customers self-report exposure to our billboard advertisement and so that was a big disappointment but after COVID-19 hit we did have four more orders that were placed and they did select billboard as the method that in which they heard about us and so we were able to eventually attribute 3,700 in sales to our billboard advertisement which was still a negative 37 percent return on investment so if you ask me our billboard advertisement was the worst method of before that we implemented but there are other other areas that you can look at for billboard advertisement and we'll talk about that towards the end of the presentation so we took a month break and we reset and we gathered our baseline data and then we implemented our Facebook ads now this is an area that we had used for advertisement prior to our implementation of the Sarah Grant we would play around with the boosted posts because you can add a dollar or two or five here and there and see how it worked and we did notice the traffic to our post increased and then the sales would often increase around those times so we attributed this to a smaller budget because we didn't need a whole lot to make this work and also we wanted to look at it from a perspective of how much would we realistically use if we were doing this without any kind of grant funding and we have some teachers that are in our cooperative so they're very organized and they helped us create a 12 week timeline for our posts which I highly recommend if you're going to use social media as a means to advertise to have a plan in place maybe once a month sit down write out a calendar of ideas on what you're going to talk about this month so that your posts kind of tell a story and in all aspects of your business are highlighted for the customer to get to know you again we used Google analytics to track our progress once you learn that beast it is so helpful and so many ways to kind of look at what your customers are doing when they get to your website and and tracking sale progress so it was helpful for us to learn how to use that and then we also used our order form to track self reported exposure so at the end of the day we had 78 customers self report exposure to our Facebook posts and they attributed to 66,000 in sales for us which is a 10,900 return on investment which we were flabbergasted by because as I mentioned previously they had some marketing experts had decided that their companies were going to kind of stray away from Facebook advertising and go towards other means and this worked fantastic for us so it's one of the few advertising methods that we studied that we still implement on a daily basis and then finally radio ads so we chose radio ads because we have a local radio company out here C103 I don't know if anybody on the call is from southwest Ohio but we have advertised periodically on the radio company at local games, football or baseball or basketball games and we had some success with that so we wanted to look at it from a larger scale and we took a big risk with this one because the amount of money that we had to put towards this ad or towards this marketing piece was quite a lot especially for a small farm and so $14,000 is nothing to sneeze at when you've got a small agricultural business that you're trying to get off the ground we did some good little hard look at our data and our customer base and we decided that NPR was the best place for us to advertise on we looked at 700 WLW and 105 and several others and NPR had similar demographics to us we reached out to all these companies got budgets from each of them and then we also they will also provide you with a customer demographic that tends to listen to their their radio station and then we compared it to ours to see which was the best fit it again we're not marketing experts so we were very appreciative when NPR said that they would create our 22nd ad for us and they use their own required language we approved it and then we were on our way we had 22 slots per week where the customer would hear our name over and over again and we had 348 slots in total now if you are listening closely at the beginning of the presentation you heard that our radio ads happened during the start of the COVID-19 pandemic and so we are aware that our data could be skewed from that but we also believe that COVID-19 was a great natural experiment because our our experiment was to see what advertising methods brought in the most amount of customers but COVID-19 might have been the reason that the customer bought but we they heard about us somehow or another and then when they were forced to buy they reached out to us and then we were able to capture very quickly the advertising method in which they heard about us from so if you had to force all of your customers or potential customers out there to buy from you all at once so that you could capture the best mechanism to reach out to them COVID-19 did that for us so we had 30 customers self-report exposure to the radio ads which attributed to 27,000 in sales and that was an 85 percent return on investment so for a marketing advertising method that cost so much up front we were able to bring that money back in and then some so you know maybe if later down the road as your companies get a little bigger you might look into doing something like this NPR told us we were not the first company that was associated with agriculture that used them and we they have several that do it on a yearly basis so something to look into for your individual operations so here is a summary snapshot of what I've talked to you about so far and as you can see Facebook was our most successful implementation followed by radio and then Google ads and then billboard but I did want to mention some other notable findings at the bottom of the page you're going to see that 153 of our customers during the time that we asked for self-reported exposure said that word of mouth was how they heard about us or through a Google search both of those things can only occur if you have brand awareness with your customers and where did they get that brand awareness well we may not know for certain on the word of mouth or the Google search recipients that that marked those on their order form but I can tell you that these were the only four advertising methods that we were utilizing during this time period so those can also be attributed in some fashion to one of the four implementations that have been previously discussed and it also goes to show that although we captured 101,000 in sales from our our marketing advertisements we continued to capture twice that amount or three times that amount later down the road because those customers then told their aunts and uncles and cousins and brothers about us and then we built our database so marketing is not an area that you should take for granted in your businesses because it's extremely important to the success of your company and building brand awareness so for those of you on the call who are more visual here is a visual aid for you to kind of illustrate the areas that have the most impact on our sales all right so the part that you're here for the lessons learned from our study and then what can I do for you so as I've mentioned and as you might have been able to tell our findings surprised us we had been told that Google ads was the way of the future and for us it was not and it was very difficult for us to navigate and we feel like the only way for us to maybe gotten better results is if we had hired somebody who is familiar with this type of advertisement and let's be honest most of us don't have the working capital to pay somebody to come in and do that for us so it's just not practical for our business and then bill boards was also not profitable it was the only one that we lost money on and we were disappointed in that but we weren't surprised we kind of thought that would be the least effective of the four advertising methods now some future studies if any of you on a call want to take on your own marketing grant proposal could reevaluate brand recognition by maybe placing several billboards in a targeted area and kind of infiltrate the area with your name to see if that has any different effects and then Facebook was very profitable to us that surprised us as we mentioned because we weren't expecting to get results that we got given the advice we'd received and then our broadcast media advertisements was also surprisingly effective for us especially with the amount of money that's needed up front to to do this so looking forward from our study we recognize our data might be skewed from COVID-19 and that maybe one of you ambitious folks on the call might be interested in taking on your own marketing Sarah Grant to try and replicate our findings so we can make sure that everybody else out there can trust that the data that we we received from our product project is valid and reliable like I mentioned COVID-19 was a beautiful natural experiment for us because it forced people to buy during the end of our project and we were able to capture those sales and then brand recognition word of mouth and Google search engine optimization is very important if you've never heard of Google SEOs before I highly recommend that you look into them it's out of the scope of this project although I can answer questions for you towards the end if you have any but it's extremely important for your customers to be able to find you when they need you and so building brand awareness and word of mouth and having good SEOs can do that for you so your next steps we highly recommend that if you don't already have one that you create a website we use the Wix.com to do that and it's very user-friendly we have no marketing or IT backgrounds and we were able to do that ourselves and a very cost-effective if I remember correctly it's about $300 a year so extremely cost-effective I also worked with Ohio South Centers and they created a video to the public on different advertising or different websites that people use so you could look for that information out there some people use Shopify and other types of website WordPress those types of things but Wix was worked for us the other thing that we recommend is that you create a business page and such as Google being and Yahoo you just go to like my Google business and set up your profile that's what you use when you go and you search Kroger uptown to see when their hours are that is the Google page that they've created somebody maintains so make sure you set one up for your own business because people rely on that to know how to call you how to get your website to see pictures and reviews of your business so make sure you're going to all those popular search engines and creating that I took a class on my Google business and didn't realize how important it was to your your business but people look at that for reviews and the more you post on it the higher you get on Google's list for when they search your company name so when we first started out if you search for Buckeye Valley beef hardly anything came up but now if you search for us we've got a couple pages worth of data that point back to our website from various newspaper or magazine articles that have wrote things on us that bring back the customer to our website my Google business also helps you with that and there's courses out there that you can take on how to utilize that the most in the most effective manner we also started a blog and the reason for that is that we want to keep traffic going to our website as much as possible it refreshes our page and and you know let's Google know that we're still present and people still want to know who we are and again it helps with your search engine optimization make sure you keep a strong social media presence this is the hardest for us because it takes the most amount of time you have to go in and think about taking and illustrating all of the things you're doing when you're outside in the rain it's you know 20 degrees out and you're in your muck boots and you're feeding the cattle you don't think about pulling your cell phone out and taking a video of what you're doing but it's important to the customers to see what you're doing and how you're doing it and keeping that transparency with them so it's the hardest for us but it's the most effective and so this study kind of helps solidify that and and make us a little bit less disgruntled when we have to go out and take videos of the work that we're doing and then use those boosted Facebook posts those are the the easiest advertising method I think to step into and the most cost effective and then as you get a little bit more comfortable with that you may transition into radio advertisements which cost a bit more money but we're still successful successful for us but start small do we have any questions I've allowed ample time and we can you know talk about anything and everything that you guys have interest in Facebook's okay so Donnie it looks like you said that somebody had told you that um LinkedIn to use LinkedIn instead of Facebook that you know people who use Facebook successfully um you know any of the social media profiles work we are a little bit older so we we went for the Facebook platform but there's also Twitter there's also YouTube there's also Instagram um LinkedIn um I haven't heard a whole lot about that in terms of like your businesses because it's more of people who are professionally networking um but you there's no harm in trying but YouTube is also a very successful area to utilize and there's a lot of successful businesses that have very strong Instagram profiles um but Facebook was the um and and also you need to utilize them the platform that your customer base is using so for us we had a lot of families um that you know came to us for their meets um and so Facebook was the most beneficial platform for us um but if you are you know trying if you have a small coffee shop and you're targeting maybe college students or um high school students or you know younger inner city folks you may look into Instagram um or Twitter one of those mechanisms might be more successful for you how often did you use your Facebook ads we or how long did they run to um we did um three posts a week that we boosted but we posted every day for 12 weeks so it was uh $600 in total so first off I want to say thank you for this presentation the the topic is very interesting and very much needed and the research methods and everything was great I really enjoyed your presentation the visual so thank you for a job well done um so I do have and I don't know if those are like questions that you can answer whether you were able to capture that in the study um I was wondering if there were any type of demographic differences in terms of where people were finding things out uh if you were able to capture that be be it like Facebook or Billboard like whether any type of demographic characteristics that was different in terms of um what type of um advertisement was capturing people's attention and another thing that I was wondering and that might be difficult to capture was how long did it take for people to reach out um based on exposure so how long must they have been exposed uh to the ad before they decide oh I've been seeing this for a few times like I want to reach out and I don't know if that's something that maybe anecdotally you may have had a conversation with a customer about well I've been seeing this and it may be after the second or third time I decided to reach out or whatever yeah um so demographically we um we didn't capture as much of that as we wish we had um looking back we probably would have a little bit um would have captured that differently but Google Analytics was also helpful in getting the demographics from the customers who um reached out in certain advertising methods but not in all of them um now we were able to talk with our customers when they would come and kind of just you know hey we heard that you heard about us through the billboard or through you know Google ads um you know tell us a little bit more about that and then when we were out and about prior to the pandemic um with the um craft shows and such where we were meeting with customers on the face to face basis um they would say hey I saw your billboard ad you know I didn't buy back then but um I keep here in your name or I keep seeing your name on the here in your name on the radio and um when I needed to buy meat the stores were low um I reached out and that was very um that was nice for us and and it took them a little while um like I said with the billboard advertisements they didn't buy during that time period when we analyzed it but later um after we finished out our our experiment so our experiment lasted for two years we were able to capture some lingering sales later down the road um that had said that they heard about us um but they were forced to buy because of the pandemic um so did that enter your question yeah I did thank you you're welcome how many members are in my cooperative and how do you meet and how often and describe how you function as a co-op group so we have four farm families currently um we have about 300 head of cattle that we push through the cooperative so about nine every two weeks go out um to customers um I think last year we finished out with a um about 750,000 in sales um so um it's it's growing each year we continue to get bigger and bigger I think the first year we had like 150,000 in sales so that tells you over a you know six year time period um how much we've grown um we are starting to look into adding additional members to our cooperatives um we were able to service all of our customers with our current farms and the number of cattle that we had on our farms but we are quickly approaching our limit um but one of the benefits to a cooperative is that you can be a small footprint so um you we have lots of small farms in our area um but service larger populations of people so if you have 10 small farms that come together to provide a product for the population of people you don't need these massive feedlots um you can have small farms that have um cattle that are um well taken care of providing for our customers um and on a consistent and reliable basis so um that was that's what works for our co-op um and we meet monthly um we have a slack chat and a um boxer chat that we you know talk about things on the daily basis when we all get time because several of us have off the farm jobs um but we we meet on a monthly basis to take care of our business um decisions did you vary your facebook posts and if so did you notice any narratives that performed best um we did so we started off talking a little bit about who we were um and then we would talk we would kind of go gravitate towards the educational piece on you know how we grind feed and how we move cattle and how we birth cattle and um you know what we feed our cattle and why um and then you know we would go into what kind of products did we have available and um those types of areas so um we didn't have anything we the videos seemed to be the most um effective on social media for us um that video that we created it was um extremely popular on our facebook um and anything even the ones that we filmed on our own iPhones um seemed to grab our customer's attention so videos were very um uh effective for us um and anytime that we talked about our products and um what what how you can like the 10 different ways to cook a steak or um five different crock pot recipes those types of things those were very um effective for us nobody else does i've got one more quick one could you go back over the um class that you took about google or or whatever that class was it told you about your pages i missed that and uh this has been super helpful by the way i'm gonna i'm working with you cut out at the end but um at the beginning you asked what class um we took to help us with google ads is that correct yeah that's right sorry i would just say thank you okay um so i had um i used linkedin learning it was it was called linda at the time um but now i think it's called linkedin learning and they have um lots of courses out there on google analytics as well as google ads um to teach myself how to do it um and then i think you can use courses r a i think they have some marketing um classes out there on on google analytics and um google ads as well but linda dot com which is now linkedin learning um is my go to for that um so uh i would definitely take advantage of that when you can and also look into your local chambers um oftentimes they have classes on marketing strategies um we do in brown county so i'd say a lot of counties in ohio probably do as well um our ad cap helps us with that and offers classes on you know my google business or facebook or social media advertising so look into those as well because um they're very helpful get on their new their newsletter um and um you never know when something's going to pop up i'll get something you know once a week and they'll have some training on there that might be interesting to me and oftentimes are virtual now so um i can hop in on my lunch break or from anywhere um to learn a little bit about the topic that they're talking about oh that's fantastic thank you so much you're welcome and if any of you are interested in in a cooperative yourself um i know there are some people who would ask questions about how we function um ohio state university south centers is a great resource for that we didn't know where to start and we reached out and they were able to help us to even create our articles of incorporation and our bylaws um they taught us how to do our financial planning um and what types of um you know stuff that we've to provide to the irs on a yearly basis um and they were able to um points in the direction of cpas that can help us make sure that we're staying um on task with the irs as well so um utilize them because uh they are a great resource oh ohio state university south centers um so north south um they have their own website i believe uh so look that up


started. Let's get started if we have a long agenda. Okay, is Alyssa here? I think you're first on the agenda list. Oh, I am. Am I? I'm still pulling up the agenda. Are we talking about release stuff? Any versus struck deprecation. Oh, I didn't know that we added it to the agenda. I didn't add it there. Who who who added it? I think I suggested that Michael Payne. Oh, okay. Well, then why don't you lead our discussion? Well, I've got a PR in that Harvey's reviewing. I've got another one that converts all of the tests, yamls. It's very painstaking to convert the the conflicts because they're all pretty much out of date. And then I plan on starting on the test code changes. So there's a fair bit that needs to be done there. And lastly, the learn site, which Chris posted the source for in the chat is all out of date. And I think that's driving a lot of our sort of chatter in the slack about all conflicts. Well, yeah, there's that. And then this will come up in the next thing and maybe I maybe I'll move it and we can talk about it in parallel. But I think those Docker image changes that I made are also confusing people. It's like you can't win here. But I think we're actually moving towards a better place. But on the I guess on the any versus struck thing, I have one fundamental question. And maybe this is for Harvey. I'm not sure. Is there some like I thought there was some problem with using any? Yeah, so the problem is it comes down to what we do is we're hashing protobufs. And I see the GRPT team maybe getting excited there. Yeah, hashing protobufs is probably never a good idea. I mean, because they don't have any canonical hash, stable hash value. And we're trying to actually make it such that I think there's a bunch of issues filed to at least the individual languages have stable hashes so that when you have the same protobuf, you get the same hash, which is a desirable property. And this is this manifests itself as a problem when we switch to any because it would cause those config updates with embedded any fields. They would have unstable hashes essentially in that will force draining high CPU utilization and Istio. So I think some of this has been fixed in different ways. Like I know, I think they've actually fixed this on the go side. I think they've on the issue actually switched back to struct for their last release because of this issue. And they didn't want to see the CPU spike. And I think it still needs to be fixed at the protobuf level. So the yeah, you know, I guess my my high level question is given that it seems that any is not like fully production ready. Yeah, should we table these documentation PRs until it's ready? Or do we think that we should have all the documentation point to any? I think that's my concern. I don't think adding documentation for how to use it is bad. I think as long as there's a warning somewhere saying like, Hey, by the way, there's still this hashing thing. If you have concerns about that, be aware of it. Well, but what but what Michael's doing is he's going through and like he's fixing all the configs to translate it over and yeah, but Matt, I think it's fine for all the configs. I think the only place that any is a problem is when we have a dynamic control plane. And people are concerned about draining behavior specifically. So those are the ones we shouldn't convert over and we should put a big slap a big warning label up there. The rest I think are perfectly ready for primetime use. Okay. Sounds good. So so I guess, Michael, the other question that I had is for one of the configs. Don't we run all the configs in the repo through config tests? Like we we run config gen and then we run those configs through the config test. So shouldn't they be broken if they're out of date? Or is it just that you're trying to get rid of all the deprecations? Yeah, just trying to get rid of the deprecation warning. So and so I've got some some test stuff coming through as well. Yeah, I've running running everything through the config and stuff as well. So assist all configs that are not hard deprecated yet. Yeah, one one thought there and this is a question for Alyssa is, I think for config test, anything that runs through config underscore test dot CC, we should make it fail by default right away. And that would catch this at PR time. So is that is that possible, Alyssa? Sorry, you're wanting to basically catch anyone using fields that we haven't marked as deprecated or once we mark them. Sorry. So basically, in the repo, we have a bunch of example configs like we have our Docker containers. We have our like config gen examples. And what I'm suggesting is that those configs today will continue to run even using deprecated fields. And what I'm suggesting is that if we somehow make it so that when we run those configs through config tests, we make them fail by default. And then when people do the integration test. Sorry, what's that? It's config test, a unit test or on my integration test, it's unit. It's it's it's its own thing. It's a it's a unit test. So but I think we could do similar things to what you did. And I would guess 80% likely it's gonna fail. But let me actually take a look. I could take a lot in for that.  No, right. Yeah. And I had to actively override integration tests, but I'll take a look. Okay, let me let me point you at or if you just search in the repo, it's config underscore test.cc. Okay, one other thing though, on this is, does anyone if Harvey thinks that fixing the stable hashing problem requires upstream proto changes? Has anyone taken on reaching out to the party team and seeing the bugs a file? The question is, who's going to actually take action on them? That's what I'm asking. Yes. So have you filed a bug with a photo? I know Istio has a bug. Yes. I would hope that is I mean Istio or the people that cared about this, like, I mean, I've always felt honestly that this seems like a lot of churn for for not a huge reason. But if Istio is pushing on this, right, yeah, contact person, I can lean on them to actually do it. And say, look, you know, we need to make progress on this or, or, you know, see my blog post on the strike versus any, it's wrong. And it's the conclusions are you can basically it doesn't give you that much. I mean, it's nice. It's, it's more efficient. But that's all Okay. Just in the interest of time, we should probably move along. I did want to mention that for everyone out there, I did it. So there's been a constant source of confusion, which is that we have one Docker image repo, and we mix in the tagged images and the master build images. We constantly get questions about, you know, where's the version release or where the daily releases. So what I did is I split them so that we're going to put the tag releases in the envoy Docker hub repo and master releases and envoy dash def. This has exposed a bunch of similar issues where there are people out there who were just running latest, which is probably not great. If you're out there, please don't do that. Who I think have gotten confused because now latest snapback to version 1.9.0. So I have a PR up right now, which actually fixes the docs, try to snap stable docs and stable stable images so that when you're looking at the 1.9.0 docs, you'll actually point you at the 1.9.0 image or point you at the shop or the image that the docs are actually snapped to. So that PR is up. There's still some work which I was hoping Michael might be willing to help with at some point, which is we need to do the same thing for our Docker files. And again, in the interest of the time, I'll stop. I think we can do some stuff with environment variable injection in the Docker files to make it run from the right place. But you could track that work on GitHub if you're interested. But the goal, from my perspective is we should get to a place where people that are running master and are looking at master docs, they see docs that are consistent with all of the configs and the image that actually runs those docs. And for people that are running tag releases, if they look at the 1.9.0 or the 1.10.0 docs, they will actually see docs that point to the correct image. And I think if we can achieve that, it's going to fix a bunch of our support issues. So any questions or comments about that? Yeah, I'll definitely help. I'm on vacation this week, but I'll help when I can. This sounds like a great vacation activity. That's that's what I've been doing. My daughter has been, can't snow ski, so I've been doing config updates. This is really painful. Some people like to take vacation on their vacation. No, I agree. I just yeah. Okay. Anyway. All right. So are we on to? Yeah. API versioning? So how do we want to approach this? Do we want to give an overview? Do our folks in the GIPC team want to sort of give their pain points that they're experiencing? I'm not sure. Like, It would be, I think really helpful for me and everyone else if either you or the GIPC team, yeah, could just in a couple of minutes just describe to everyone, you know, what you're trying to achieve, why the current thing is broken, and then we can start brainstorming or discussing different solutions. Yeah, I think a special dog does that. Yeah, I can take a stab at that. So let me let me see if I can get my screen share working. But while I'm doing that, just a quick introduction. Yeah. So I'm Doug. I work on gRPC in Go. And where's the thing here? Sorry.  And let's see how this is going to work. Can you guys see my screen? Cool. Okay, cool. So I put together a quick one page or to sort of cover this. But yeah, so gRPC has been looking at moving to Envoy XDS protocol as our load balancing and name resolving a built in solution to replace something that we designed and developed in house. And so just load balancing. Oh, sorry. Okay, just for load balancing. Okay. Great. And right. So while we were in the process of developing this at least for Go, we got broken a couple times by some changes that were made in the proto files. These were both I believe instances where a field got moved from outside of a one of to inside of a one of and that that breaks the Go API but not in other languages or at least not in C. So so yeah, we started sort of digging into the breaking change policy and Envoy. And we were a little bit concerned, I guess, by what we read in the form of discovering that features could actually be removed during a major version of the XDS protocol. And this is going to cause some problems for us. So I don't know, do you guys have any questions yet? Or should I go into like sort of a diagram that sort of shows why this breaks that I can run through in a minute? No, thanks. That's super helpful. I think that makes sense to me. But yeah, I guess if other people have questions or would would love to see those diagrams that would be useful. I was actually going to say if this is designed for actual consumption, and is it shareable, if you can share it to me, I can stick it on my Chromium account and then actually stick at the meeting notes. Okay, sure. I can do that. And I can do that right now. Yeah, I mean, I think that's good for posterity. This is only a one page or such. But as long as as long as it's okay for me to look externally, this one. Yeah, I had I had another version that had some more Google specific concerns in it. So this is this is just removing it's actually just boring stuff. But so so yeah, so let me go through these pictures then. So so this sort of just shows like a, you know, happy deployment here with several versions of gRPC running in, you know, a couple cloud environment years, we got a Kubernetes deployment and a GCP. And in GCP, we have, we have a traffic director, which is the envoy management server for that environment. And in Kubernetes, you know, a customer is probably running some random version of Istio pilot for that purpose. And so here we'll show like these are future versions of gRPC that, you know, hypothetically support XDS. So so everything's working happily. So if in the future, let's say, Envoy looks at the protocol and decides, hey, you know, this feature, sorry. Okay, so so, you know, feature foo here is, you know, not really what we needed. What we needed is feature bar, which has, you know, the same capabilities as foo, but it also lets us do some other things. So for instance, a boolean being changed into like a floater and into represent a percentage instead of on off. So so Envoy decides we're going to deprecate this old feature. It's superseded by a new feature that has more functionality. And so based on our reading of the breaking change policy, this is allowed. But this is fine for now because even the traffic director here has updated to the latest version of the XDS protocol. And Envoy everything still works because foo is still declared and it's still supported by traffic director. So everything keeps working. Everything's good. And now let's so gRPC has a decision to make either we keep using foo or we switch to using bar, because we either want to work with the old versions or the new versions. And so here's here's where things go wrong. Step one. So gRPC in this case, we decide hey, we want to keep up with the latest and greatest Envoy version. So we're going to switch over to using bar foo is going away. So we can't keep using that obviously. But here now if a customer has an application that's using gRPC and they update the gRPC version in that to use this bar version, then the customer's Kubernetes deployment now has a problem where gRPC can't talk to pilot because they haven't upgraded their pilot version yet.  But anyway, so so far maybe but let's let's go to the next picture. So now when Envoy releases version one dot 11. Actually, the foo field entirely will be deleted. And it would be impossible to even do version negotiation, except maybe to negotiate that you can't work together. Because that field isn't even present. So you can't support that old feature anymore. So essentially traffic director and sorry, the colors here, I forgot to mention yellow was things that work with foo and purple are things that work with bar. And before we had a gradient on traffic director because it was in that intermediate version. But now traffic director is updated to the latest latest version that eliminated foo entirely. And now you see all of the existing gRPC deployments that we're working using who can no longer talk to this new version of traffic director. Meanwhile, you know, the new version that uses bar still can't talk to the Istio deployment in the customers Kubernetes environment, because that hasn't been updated either. So I think the issue here is when when foo gets deleted from the proto, we have we have problems because we can't set it any more in gRPC and no on boy management server can even read it. But okay, I'm I'm a little lost though here because I I definitely understand some of the concerns around versioning. I mean, that that makes sense. And I thought this conversation and maybe I was incorrect. I thought this conversation was mostly around the fact that we don't want like a Shaw or version based API, we want something that's tagged. But if I understand it now, I mean, you're you're suggesting that we can't deprecate fields. Like, I don't mean that doesn't that doesn't make a lot of sense to me. Right. So so deprecate, I guess has different meanings to different people. So so we're okay with deprecation, it just needs to remain available and supported until at least by the protocol, maybe not by the latest versions of envoy, if that's, you know, how envoy works. But it needs to remain in the protocol because, you know, it can be marked deprecated, but it needs to stay there because otherwise you can't even read the fields. And so you can't support legacy applications once it's been removed. And so, so there's actually like best practices for managing proto's have widely been agreed upon as if you break backward compatibility, then you really should create a new proto package that contains all the same messages, but with the things removed that you were intending to remove. So the once you've added something to a namespace, you can never remove it from that namespace. So it's essentially semantic versioning for proto's. And you do a major release by changing the proto package. So so that part makes total sense to me. And we can talk about a variety of different proposals by which we can make that better. I mean, I think we'll end up splitting hairs because I think you can do a lot of this from a shop based perspective or namespace. But yes, I totally agree that we can do better there. I still want to come back to requirements though, because that's where I think I'm getting a little confused, which is to me, there's there's two separate things here, right? Previously, we've been talking about, you know, essentially XDS as an API for envoy and management servers, there's some deprecation window of which we can move things around. You're obviously going to use the XDS API for a different purpose. But I guess what I'm still trying to understand is that, like, let me let me just suggest something. If we kept the same deprecation policy, but we basically just kept incrementing the namespaces so that you could go back in time to a particular shot or version that you support, but we're still removing things every three or six months. Does that work or do you actually need the fields to live for a longer period of time? Like, that's what I'm trying to understand here. I guess I didn't follow what you're saying. So you're saying you will still remove the fields, but what I'm saying is like, let's just say for the sake of discussion, that every three months, I'm going to go from V2 to V3 to V4 to V5. I'm going to change. I'm just, yeah, just for the sake of discussion, like I'm going to change that every every three months. Right. So this would, I think this would help, right? As long as V2 kept all of the features that V2 started with plus whatever was added in the middle. No, but it, but it, but it would be deleted from the repo, right? So, like, you would have to go back to it. We still need some long-term stability here, right? There needs to be. But that's what I'm asking. We need to keep working for a while and we need some policy that sort of says, okay, this version will live once it's created for such and such a great time before it stops. So, but that will, that will solve one of our problems, right? I mean, so, so the problem is backward compatibility and being able to tell GDRPC users versions of XDS it works with, right? So we can say we'll work with you soon. That'll solve a tactical problem. It doesn't solve. Right. So, so folks, there's, there's, there's two, hey, let's just, just, just time out for a second. There's two separate things here. And that's what I just want to clarify. So I understand what the problems are. One of them is how we version things. And again, I was saying before we can split hairs about this, because technically you could say that the proto is a version under a particular shot and that's what they are. I'm not saying that's what we should do, but that would technically work. But then there's the other concern, which I think I'm hearing. And that's what I really want to understand that to you, our deprecation policy is too short and it needs to be longer. Like, is that, is that accurate? It might be a combination of both. I mean, realistically, like it depends on how often you're deprecating features that we need, right?  Right. Sure. So again, I'm just doing this for discussion, I'm not saying that we should do that. I'm just trying to understand the different problems here. I understand. So, so, but that's, that's sort of like on one end of the spectrum, right? And then if we move to, okay, we work with V2. And then meanwhile, the Envoy project has moved on to V17 by then and it's only, you know, a couple months later, then they're also going to look at us really confused and upset because we're not able to give them something that works with, you know, what they might be running in their environment. Plus, it makes our job a lot harder because we have to support, you know, version two, version three, version four, version five, or, you know, we decide which ones we support. But if we're constantly having to change, then that becomes a problem too, right? Sure. There's, I think both are an issue here is what I'm trying to say, right? And then also the third issue of like essentially what does the API repository look like? Do we actually have, let's say, 70 versions in the repository or the latest version in the repository? So, I would say, again, given that we're changing in this deprecation policy where we have, we have this a six-month window now, right? We plan at some point, TBD, to go to a year. Envoy cuts releases quarterly, so that basically, you know, I think you could say we support every fourth version and you would be able to say, okay, now you migrate from the last version to this version and you're upping your versions once a year. Well, and with what Harvey was saying before, I do think that version negotiation actually helps us a lot. And again, I realize, you know, we have four minutes, like we're not going to figure out the final proposal here, but I think just for framing the discussion, I still think we need to be a little bit crisper about what the actual requirements are, because before we even start talking about solutions, because again, I think what I'm hearing is that the length of our deprecation policy is not long enough, but even there, I'm still a little confused because Envoy, the proxy, as I understand it, may actually not be involved in this problem at all, right? Because you have a GRPC client, you have Pilot or you have, you know, Traffic Director or something like that, and Envoy, the proxy may not be involved, it's just the protocol, right? Right, that's absolutely right. And I think one of the key points here that isn't sort of the problem we're talking about, but is the underlying philosophical issue is, if we intend the XDS protocol to really be an industry standard thing that is not Envoy specific, then tying its versioning to Envoy doesn't really make sense, right? There are all of us here, and it needs to be an independent thing. Sure, and again, I'm confident that we can find a proposal that will work for everyone, like I think the solutions are pretty well understood, and I think with the right tooling, we can figure something out here. So like, I'm not concerned about that. Mostly per my comment on the issue, I just want to make sure that we make all of you happy, but we don't decrease velocity, and I'm actually pretty confident that we can do that. But with that said, I think that actually laying out what the different scenarios are, like you did in the doc and what the requirements are, would be very helpful for helping us build the final solution. And I think that would be just a framing of the different scenarios of using the protocol without Envoy, using it with Envoy. These are the different deprecation dances. These are the timelines that you would be looking at. Is it possible to put something like that together, like not even prescribing solutions, but just talking about the world that you would want to live in? Is that doable? Yeah, I can take a stab at that. Sure. And yeah, I mean just... We'll put it in my trap. Sorry, yeah, we'll need to... Yeah, we'll need to sort of, I guess, work together to put together a doc for that. Yeah, so we can take a stab at that. I guess this meeting's only half an hour, right? Yeah, I mean, we will definitely need to meet again on this and talk more. And again, I want, in the two minutes left, I'll stop talking. I want other people to ask questions. Yeah, I think basically what we do is we get that doc, we stick it to the bug, we discuss back and forth with the doc, and then we schedule a one-off meeting where people can attend that one to discuss it. And that'll be at least an hour long, probably. Yeah, yeah, because again, like I am actually quite confident that with the right tooling, there's, I can think of four different ways that we could solve this problem and they're probably all fine. But I'm still concerned that if we still deprecate things too fast or do things in the wrong way, we're still going to end up making you angry. So I feel like we have to look at this a little more holistically. Okay, sounds good, yeah. And then just a couple of points that I'm going to make is, you know, the proto-fields really have to stick around in order for even version negotiation to work. And that's where sort of the technical aspects of backward compatibility kick in. And then, and then secondarily, you know, we're certainly open to, you know, being able to mark new features as experimental when they're introduced. And those would be available to be removed just like, you know, you do today with anything except, you know, that could even be at a moment's notice because we just wouldn't use those features. And then that would be a clear contract where you say, these are the things that we pledge to support and these are the things that, well, we're still messing around with them. No, and that doesn't make sense. Yeah. And we don't want to get better at that because we do have support for that today, but it feels like there's some communication gap between that and the users. So we should definitely fix that. The other thing that I think that we could do that will likely come out of this, sorry, just super quick, is I think that we can enhance the existing deprecated attributes on the protos to say things like deprecated for Envoy, but not deprecated for the protocol, for example, and like we can do things like that. So anyway, I'm confident that we can figure something out here, but I think that if you could put that initial doc together, that would be really helpful. Okay, we'll take a look at that. Thanks. We have to go. Okay.