Live from San Francisco, California, the Cube, covering MarkLogicWorld 2015. Brought to you by MarkLogic. Now, here are your hosts, Jeff Frick and Jeff Kelly. Welcome back everybody, we're live here on the Cube at MarkLogicWorld 2015. I'm Jeff Kelly, I'm with my co-host, Jeff Frick. Hey Jeff. We've got Jeff and Jeff for this segment. Keeps it easy for the guests. Absolutely, and speaking of guests, we're joined by Derek Tapp, who's the head of IT Solutions at CABBY in the UK. Hello. Welcome Derek, thanks for joining us on the Cube. Welcome. So CABBY, I'm guessing most of our US based audience won't know what that organization's all about. Why don't you tell us a little bit about CABBY and your role, and then we'll kind of get into some of the things you're doing with MarkLogic and on Structure Day. Okay, yeah, so CABBY's a non-profit organization. We aim to improve people's lives through solving problems in an agricultural environment. So we do things like help farmers in developing countries by providing information to them. So all about long-term solutions, so not just a point solution, but something that's going to help people over a long period of time. You're all. Yeah, so like I said, I'm the head of IT Solutions, so my area's quite a small area, but we work a bit like the glue between what the business needs and what IT can deliver. So we help with some of the larger scales of architectural solutions, provide technical direction and some expertise around that. So I have some business analysts and project managers, so we just sort of try and make sure everything fits together. Interesting.  What does it look like? Are they applications that a farmer might use or what do they actually look like? There's a big range, because we try, like our end goal is trying to help people like farmers in developing countries. So they have very different requirements to what we use for some of, providing some of our revenue which helps support our projects, which is providing information to academics and researchers. So tools that you'd want to provide to researchers is a very scientific and very heavy information, whereas when you go to farmers developing countries, you're going to want a lot less information, a lot less scientific, a lot more practical advice. And how we get that out to them varies by country as well. So some countries have good smartphone penetration, they have good mobile coverage. Others don't, but we still need to reach those people. So in quite a lot of countries who work, but one of our programs is called PlantWise. So as part of that, we've been training up a network of plant doctors around the world. Now what they do is they'll set up a little store, a marketplace or somewhere else central that farmers come to and they can ask those guys for advice. They can take along samples of their plant, they're having problems with and that plant doctor will help to diagnose it. Now, we can either try and provide information to those farmers themselves or, and this is what is often a better approach at the moment, we just train those plant doctors as well. So we're supporting the plant doctors to support the farmers. And even those farmers go back to their communities, they then help support the farmers that they then work with their families. And that sort of, it's that knock on effect.  So you kind of taking that and delivering a little different scale. Yeah, so it's mainly the small holder farmers we target. So it's practice that makes sense to them. So there's no point saying you can take this practice if it's something they couldn't possibly do. So it could be something that's incredibly labor intensive about a machine. They can't afford the machine. So don't offer that suggestion to them. Okay. So it's looking at what practical information we can give them. So it can be around sort of things that would help with the climate. It could be things about using seed, which is more resistant to a disease that's known to be in that area. So it's not a one size fits all. It's what's appropriate to that area. Excellent. So you're a relatively new to Mark logic. Yeah, I understand. So we've had a number of customers on today that have been on with Mark logic for a while. Let's talk with a customer who's fairly new. It's a good opportunity to talk about what was the specific business challenge that you were dealing with. That kind of led you to Mark logic. It's kind of walked us through that journey. But the one main thing is that cabbie is over a hundred years old. So we have a big variety of data internals through the organization. Some of that we can use very easily. Others, not so much, but even between the difference of silos of information we could use easily trying to merge those together is labor intensive. So we've been doing it as a plant wise we have to do that quite a few times. But what we identified is that if we could take away some of those hurdles then we could produce new products easier. So it's more about thinking about how could we use information A of information B rather than say, oh, have we got time to join those things together? So is that ability of Mark logic to ingest lots of different types of data and then minimize the effort to link those things together? So talk about when you were looking for a solution. Obviously you came to Mark logic, but I'm wondering, did you look at some of the other no SQL databases out there? I mean, there's this kind of slew of open source with SQL databases. And then you've got more of the traditional databases that are trying to add no SQL like capabilities. What was it like when you were looking at that landscape and what did you use to kind of evaluate what was the right solution for you? Well, we come from a SQL server background mainly. We have a few other database technologies inside Kaby but mainly a SQL server shop. What we identified fairly quickly is it was a document database flavor of no SQL that was the best match for our use cases. So it was really within that area. We put most of our attention but we were also interested in the graph databases too. So that instantly puts Mark logic sort of up near the top of the heap as it's got both. But even within just the document side, although I liked some of the open source solutions I had concerns about like betting the business on those being available for the next five years and even within short term periods is a disaster recovery good enough for us. So we let's have a number of solutions but Mark logic came out as the clear winner. We'll talk about those enterprise grade capabilities. We hear a lot about that from Mark logic. They kind of market themselves and enterprise no SQL. Thinking about a little bit more about what were the key quote unquote enterprise grade features that were important to you? Well, the two main ones for me is coming from the SQL service shop. When we first started looking around some of the open source ones we were, I wouldn't say disheartened but it was more of a compromise than we thought we were going to have to make. So we knew we were going to get the flexibility to get the document model but we didn't realize up front that we're going to lose things like ACID which to people growing up with SQL server is you take it for granted. So we'd be able to get that back again but we're still having the benefits of the document database and having the graph technology in there as well. That was a big plus point for us. The government grade security that my logic is very hot on to us isn't as a big an issue. Like security is important to us like it's for any organization. That wasn't like a, it wasn't a killer feature for us but being able to reliably recover from any disasters is. We have a relatively small IT team we're supporting products which are important to people. We need those to be available. We don't want to have to have our DVA spending weeks trying to manually rebuild a database. We want things to work and when they don't work to have some realistic way of restoring that data. So let's talk a little bit about the use of semantics. We hear a lot about that at the show today. And we talked to a number of practitioners, customers that are really touting the semantic capabilities to bring together data from a lot of different sources. So kind of getting back into the core use case. Can you provide a little bit of detail around those capabilities and how they're ADU and the building applications? There's a few things we're looking at in that area. There's one application we built in the past as part of PlantWise which we achieved what we needed to via a relational model but it was hellishly complicated. The SQL code to join everything together to get the answer we needed was very, very complicated, very laborious so lots of places there could be problems. But it works, it does a good job. But looking at, if we try to recreate that now using a semantic model it would be a much, much easier thing to do. Because basically what the tool does is say if you've got this problem on this part of this plant in this country, what could it be?  And that's just a perfect use case for semantic. So let's say that's linked to this, linked to that, linked to this. And it's just a perfect use case for it. So one of our earliest proof of concepts we did was to replicate that in my logic using Sparkle. And as we expected it was a much easier thing to achieve. Going to the future for things that we haven't already got. We track metadata across lots of our data silos already. But the idea of using semantics to link that together is a really interesting thing for us. So in one of our larger description products we have a lot of search capability in there but not so much around browse. And that's something we're looking to improve. And having good quality of metadata and the way you can richly link things together will make that a much better experience for people. So I think the semantics could be strong for us in that place as well. So you talked about using MarkLogic to build products with the data assets that you already had. Now that you've been at it for a little while, I mean is this opening up a lot of new opportunities? Do you see a really robust roadmap of places that you can go where you couldn't go before? Yeah, well we've only just started working on our first formal projects who became customers at the end of January. Okay. Where I'm seeing we've got really exciting avenues we can look at is by combining what we're doing with MarkLogic with our new API-centered design. So we're getting into open link data as well. So by having our internal data in a good state and also linking out to other people's data allowing other people to link to us is those two things together we think is going to be really valuable for us going into the future. Right, because do you aggregate the data from the small farmers then the feedback to the scientific community? So it's kind of a symbiotic relationship. They get wide data set to do their studies and then the data then goes back to the source. Yes and no. Okay. A lot of the data we have is open, it's academic, it's published research papers. That we make available to everybody in various different ways. Okay. The data we get back from the plant clinic, so the clinic's the plant office run, that's sometimes politically sensitive data. So we signed data agreements with those countries about what we can use at data fall. Okay. Because it could be if we release data saying there is this pest in this country that could have trade implications which is bad if they've got that pest but what if that pest wasn't actually there? What if it was a misdiagnosis? There's lots of politics going there. So from a technology point of view, yes we can do that but from a political point of view and our sort of our responsibility to the countries we work in it's a little less clear than that. What we're hoping to work on is as Kaby sort of proves the value of having open data, we're hoping the other countries will see that and then if people like producers and traders see which countries are more open about their data, they'll be more willing to trade with those people. Right. Because if you have a country that says oh we have no plant pest in our country, then no one will believe that. It's like okay that's the official line but no one's going to believe that. Right. Whereas another country says you can see all the problems you've got, that's where we are. Right, now apply your brain power to help us solve those problems, right? Exactly. So in theory it should be a nice positive spiral. Yeah, so there's a lot of complication there and most of it is nothing to do with the IT side. It's politics, it's trade, outside of my area most definitely but it's an interesting thing to be on the edge of. And how many countries do you guys work in? We're working in about 70 countries I'd say. Okay. We have member countries who sort of help steer our sort of direction. We have 48 of those but we're actually active in about 70 countries, give or take. So looking forward, I mean we talked to a number of customers today that kind of started with BarkLogic around one use case and then expanded to multiple use cases. To the extent that you can share, what are you looking forward to in terms of leveraging some of the other capabilities in the platform and really moving forward to kind of your data strategy? But our key focus is just getting a central data store. So it won't necessarily be the master for all of our data but somewhere where we can easily do a combined search across our different sets of data. So that could be stuff we currently hold in relational, could be in documents, could be in access or Excel or whatever because it can be going back like a decade or some of our data. And it's getting that into a form we can use it. When it's there, that's when we can start collecting the different ways we can use the technology. But it's building that core data. And then you can kind of build the top of that. We build that foundation. That's where the main value for us is. So we're short on time but I want to get your impressions of this event here. I mean, there's a lot of folks here. We heard from our last guest that one of the biggest values they get is from hearing stories from other customers, other users. What's kind of excited you that you've heard here today and what's your impressions of the event? I do like the event here because the thing I like about this one, so there is an equivalent in the UK but this one is more technically focused. There's a lot more detail but you also get to talk with the people who actually build the product. And that's what I like about being here. I don't know how many of the my logic engineers are here but it seems to be wherever you turn this one, not too far away. So rather than asking people who maybe they're in sales or support, you're talking to the people who are actually building the product. Hopefully being able to influence that as well. And that's why I find this one very valuable. So the sessions are great. I like the sessions but it's the interaction between the sessions that I actually find more valuable to us. Well, Derek from Cabby, thanks for joining us on theCUBE. We appreciate it. Hopefully we'll see you back here next year at MarkLogic World with theCUBE. So thanks again. Thanks for watching everybody. Stick around, we'll be back with our next guest here live at MarkLogic World right after this.


it's harder for us to notice and categorize anxiety rising up and sort of attaching it to external things that are happening to us in the moment. And so being aware of these physical behavioral cognitive signs that we're going towards a state of anxiety or burnout, notice them, write them down, characterize them. You don't need to write them down if you don't want to, just help. And next time that they happen, take note of them and say, all right, okay, I don't get any bit anxious. Am I being productive? Am I in a rut? And there's nothing that I can do. And I should probably take a break and approach it next day. Might be really, really difficult the first time you do it. But after a while, you will get the hang of it. And it will have absolutely massive effects for your overall productivity in general, you will have these nasty burnouts, you will have this, these awful transition times that just eat into your sleep.


In the upcoming half hour, you will learn a lot about in-app purchases. These are payment products that are mandatory if you are trying to sell virtual products of services on the App Store. If you are looking to sell your own products on the App Store, or maybe your employer is looking for a trusted business model, then this will be most relevant to you. I'll provide an overview of the types of in-app purchases, how the API workflow can be, how an app can manage its in-app purchases inventory, some case studies on in-app purchases, and of course the obligatory code demo near the end of the talk. A short intro about me. With ACP from 2013 to 2016, I worked on their smart cities efforts and their indoor defecation projects. With NCS, I delivered SingTel's short-lived price call app, if you heard of it, that was made up with a good team from China. I also have been developing my own products for Mac, iOS, and TBRS as a product side job. For more of my profile, you can connect to me on LinkedIn or send a QR code to get my full profile. Yes, shortly, what in-app purchases are? These are non-contruable, non-renewing, outdoor renewing, and subscription. Non-contruable, just like book. Probably the most common in-app purchase that we have right now. It boggles on, meaning that it goes from disabled to enabled, and that's it. Similar to like buying a book. One interesting use of in-app purchases is that you can use it for trial periods. We'll talk more about this later on. You can also provide it for upgrade discount, meaning that if an older version of your app is tall, you provide this in-app purchase for upgrade. Or if a competitor's app is tall, you can also provide a discount upgrade using in-app purchase. Non-contruable data are present in the receipt. The receipt file is a file that is scripted definitely signed by Apple, so you can verify its validity. It contains all transactions including non-contruable books. So, non-renewing subscription, just like a tourist pass. You can buy multiple times, and it expires after a certain period of time. It is meant for purchases that are maintained manually, meaning that users keep on buying and buying and buying themselves. The duration is typically hard-to-get in the app, so the app leaves them though. This purchase is for one week.  The feature can foggle on or foggle off, meaning that once we reach sufficient last, you need to foggle off. And you also can get a history of purchases in the receipt file. Take note that Apple review will probably need you to have this sentence in your UI when you are offering in-app purchase for subscription. Okay, auto-renewing subscription. This is the holy grail of SaaS software as a service. Apple would keep on rebuilding the user until the user cancels. And guess what? The user needs to cancel from iTunes and not from your app. So you have an excuse to not provide the UI on your app. The receipt will contain the in-app purchase as well, the receipt file. And take note of these legalities that you need to put in your app when you do a subscription purchase. Auto-renewing subscription supports subscription groups, meaning that a group that can only one be active at the time. And these are typically different durations of the same product. You can configure the group in... okay, consumer world. Consumer world in-app purchases are often found in games. The purchase can be consumed by the user. Take note that the receipt data is not present. The receipt will not have consumer purchases. But your app will need to have pre-stop purchase as well. If the user installs on another device, users will expect to have pre-stop purchase as well. And therefore, you need to have a back-end to keep track of your purchases. Here's a summary of in-app purchase types. Non-consumer world can be bought only once. Consumer world would need to reside on your back-end. These in-app purchases have their own validity period. Non-consumer world should always be valid, but consumer world will degrade as the app is being used. Now, this section will discuss a sequence of calls that you need to do when you are implementing in-app purchases. This will be listing products, making a purchase, restoring purchases, and validating purchases. In a typical in-app purchase store, you probably need to have a list of products that the user can buy. So, how do you list? First, the app is an SKProductRequest object and gives it the collection of product IDs to receive. These product IDs are typically hard-coded in the app. Since you need to maintain a relationship between the product ID and the features that product ID enables. Then, set the delegate. After that, call start. This will invoke a confirmation dialog, system-confined dialog, and along with some network calls to Apple. After a while, you get a callback, product request is received. You will get the localized title, localized description, and localized price. Take note, do not keep this data inside your app. Why? Exchange phase. Exchange phase, change every time. The price needs to be displayed in the user's currency, not in your app store developer ID currency. That's why, along with the price, the title, and the description as well, these are maintained inside the app store. Now, making a purchase. This is the workflow that your app does when your user selects when they are purchased. They click buy button. To ask the app store for the user's money, and to enable the features that the user has bought. First, you call add transaction observer, as usual, to the object, giving it the callback object that will receive the transaction program. Then, you create an escape payment object. Just give it the product ID that the user has purchased. After that, how? They work. You wait. You wait. But after you add payment, you wait. And as the user buys the product, maybe enter their password, and confirm it with conversion. This is the time where to send this various subscription dialog. Afterwards, you get a callback. Payment queue, updated transactions. And you respond by enabling your features. By enabling your features and updating in your inventory. Meaning that users are in so have bought this product and have, I have to enable these features. Now, if the user installs the app, or install it on another device, here she would expect the pure purchases to be enabled as well, right? This is the restore purchase flow. Apple's support for restoring purchases works for almost all in a purchase except consumable. You need to provide these services for consumers from your own back end. How do you do it? Now, this flow is for the other in a purchase. The non-consumable one. First, as before, you add a transaction observer to the escape payment queue. Then call restore completed transactions. Which will directly make a network request to the app store. Then, you will get a callback payment queue after the transaction. You should respond by updating the in-memory state of the app. Enabling or disabling features as required. Remember that your callback may be called multiple times in a loop. So just be aware of it. At the end of the loop, you will get payment queue restored for completed transactions. This is a good time to stop the spinners and enable the UI. Finally, you shouldn't call restore protected data unless the user asks for it. Because that will invoke a login. That will get a complete list of Apple IDs. If done without a user consent, that is suspicious as best and inconvenient for the user. Therefore, at startup, you will need to verify your receipt yourself. How do you do this? First, call NS bundle. Where's the receipt? App store receipt URL. You will get a receipt to a local file containing the cryptographically signed receipt file containing a history of purchases. Everything but consumable. Then what do you do?  Take note that there's no API for this. Each developer will need to come up with their own unique implementation of reading receipts, validating receipts, and updating inventory. Why? This is to prevent widespread tracking. Meaning, as a cracker, you need to crack all app store apps. By having developers having their own unique implementation, it reduces the possibility of having one app to crack everything else. See. Now, data model classes. The main goal of these classes is to be the single point of contact to answer things like, have the user purchase X. How much balance for product type Y? And each feature X enable if you have more fit when you want fit for product. This is not the type of in-app purchases, but the inventory of the purchases that user has for an app. This is the crack class hierarchy. I came up with this class hierarchy when I was developing my own product. It became complete because I had to switch through a number of in-app purchase types. I started at subscription, the auto renewing subscription, and that was rejected. Then I moved to non-renewing subscription. That was rejected as well. Then I go to be non-consumable, which finally got accepted. It was a long journey and fortunately at now I have this class hierarchy which you can reuse in your app as well. Let's start with the product inventory item. This holds the common attribute for all product inventories. Remember, this is the inventory of purchases that the user has made. The most important one will be product identifier, which contains the product ID that matches with the one in the app store. And the last transaction date may be just for display purposes usually. Next, we have a toggle product inventory item. This is the base class for 75% of all in-app purchase types. It goes from disable to enable or disable. Again, back and forth. That's why it's called toggle. The non-consumable product inventory item is the most common toggle product inventory item. It goes from disable to enable and that's it. It doesn't add much method beyond the one that's provided by the toggle product inventory item, but you need to have these things perfect and light. Also, if you implement a $0 in-app purchase trial, you probably need to subclass this class as well for in-app purchases that expires after a few weeks being purchased. More on that later on. Subcussion product inventory item can go from disable to enable and back again. That's why there's a base class for it. The most important thing is the expiry date. And it has a group which groups the product that a product inventory item may have. This usually if you offer multiple tiers of the same product, you put it in a group. Next, the non-renewing product inventory item. It has a subscription duration. Why? Non-renewing product subscriptions need to have the duration inside the app, hard coded inside the app, because the app store does not maintain it for you. However, an outdoor renewing subscription has the product duration maintained in the app store. Different things, remember? Non-renewing, you maintain the duration yourself. Outdoor renewing, you maintain the duration in the app store. And finally, consumable product inventory items. This is like gasoline or coins or credits that you put. Remember that you need to keep the number of purchase items inside your server and then communicate regularly to reduce the deduct that amount that the user have bought. Now, some business cases that are maybe common, maybe not common in nepoticious. We'll discuss about pre-trial for the sale and watch OS add-on. So, according to section 312a of the app review guide, you can provide a trial period now. This was a many people's struggle and negotiation effort. So, you really need to be thankful to them. It's implemented by a pair of non-curchable in-app purchases, whereas one is the $0 trial in-app purchase. There's a non-curchable for $0. And the other one is the one with pricing on it. The happy flow will go like this. User will download the app. Then user will buy the $0 in-app purchase. This will unlock features of the app. And when the user likes it, the user will buy the feature which buys the other in-app purchase and unlocks the feature indefinitely. But if the user doesn't buy it, then the $0 in-app purchase will expire by itself. How do you do this? You look at the purchase date plus some weeks, and that will be the trial period. Remember that all in-app purchase site has a purchase date. But the app needs to be useful without a purchase. Say as a viewer for its old file types, which is a document app. One example that does this is Omnigrafo, which is a technical search business drawing application. You can use it as a viewer for its own file types without purchasing. Or you can buy the $14 in-app purchase. They also implement an upgrade mechanism this way. If you have an older feature of the app, you can upgrade it as a newer feature for a cheaper price. Now, there is a question on Apple's developer forum asking, how can you have an in-app purchase that's available sometime in the future, only for a limited time. But if the user bought it during that time, it will be available forever. And you can take a guess. If an in-app purchase can be made available in the future, if the user bought it in the future, then it will be available for the rest of the lifetime. But it's only available at that time. So, can you take a guess how to do this? Take one. 25. Holiday special. Holiday special? Yeah, this is a holiday special in-app purchase. How do you do this? Is that a holiday special toggle? Okay. How do you do that? Let's say today is September. You want it available for holiday special in December. The answer is to have a non-futurable product that's only visible during that time only. And if the user bought it, that is there. If it passed that period and it's gone. But you also need to make it available during app review as well. Remember that if the app reviewer doesn't see your in-app purchase, they won't like to prove it. So, you control the visibility by having the in-app purchase available during app review and during the holiday period. You can control that using timing or using a switch on your back end. Now, watch OS app on. How do you sell a watch OS app? Of course, watch OS 6, you can sell it without an iOS app. But what if the watch OS app is an add-on to the iOS app and really needs the iOS app? You need to make sure that the user, when the user buys the in-app purchase, the user only have a watch OS attached to the iPhone itself. Remember that not all watches can be attached to iOS, such as the iPad or the Apple Touch. They cannot be attached to our watch OS. How do you make sure? You need to make sure that the app is installed before you display the in-app purchase. This is how you do it. First, the iOS install to watch OS app. And then the watch OS app notify the iOS app that it is installed. Right? When it is installed, you only when you get a signal from the watch OS app, you say, yes, show this in-app purchase. And after that, when the in-app purchase is purchased, you signal back to the watch OS app, enable yourself. Okay, now for the code demo. We're going to explore the contents of a receipt file. This receipt file contains a history of in-app purchases and is technically signed by Apple, so that you can verify that it's genuine. For this demo, we are going to look at the copy of a receipt and genuine one obtained from one of my apps during development. I'll be using a TP in-app receipt library that you can get from CocoaPod for really receipt files. Now, let's go to Playground. Okay, this is the receipt file that I already have installed. That's why I will read the receipt. When this is genuine, you will get the receipt URL from NSBando, not from the in-app receipt URL bundle. And you see the file. When you read the file, just run the code. Okay, now we can get the data. You can get the data file. So this is the in-app app that was used to generate the receipt. And these are the purchases. I was testing the subscription in-app purchase here. So you can see this is product ID. Product ID. And you have the transaction ID here. The original transaction ID here. Remember that for subscription, every transaction has an ID. But the first one will have the original transaction ID. You have the purchase date. You have the original purchase date, which is the starting one. And the expiration date of the subscription. Only in line of ID. Okay. Now, summary of in-app purchase type. Remember, non-consumerable can be bought only once. But it's available for eternity. For consumable, you don't have it on receipt. You need to keep it on your own back end. Just take note of these purchase types. And that's it what I have for now. This is my contact info if you'd like to connect with me. Any questions? Yes, non-consumerable. How to test for lose? How to test for lose? Because they don't inspire. So what I did that time was to buy it from the test line. It saved that while I was. And I found it there with that account. So I need to account every time I test. Yeah, that's one way. You could create a new product ID. Another way. Yeah, new product ID as well. But it would be a good practice if you keep your receipt file. And use that as unit test. That was very useful for testing for my point of view. But keep in mind that you need to remember that a receipt file will have a device ID in there. You'll have an ID of your device in there. So you need to disable the validation of the device ID when you run it in unit test. Otherwise it works on your laptop and someone else's laptop, it breaks. Or it moves to your CI and it breaks. So when you do that, when you do your validation, right, disable the device ID validation. Okay, good luck with your business. And thank you. So when customer buys, the money goes to Apple first and then to you? Money always goes to Apple first. Yeah, yeah, yeah. Then it goes to Apple first. And then usually until something around 2010, they buffer it until it reaches $200, then they give it to you. But now they tend to flush everything every month. So you get it at the end of the next month. Any updates? They still the same. Still the same, no updates. And still no family sharing for in-app purchase. There is a loophole for in-app purchase, right? What's the loophole? It's called viewer apps. It's called viewer app in the app for in-app purchase. Meaning that if the user sign up to your website, right, and bought the stuff there, you don't need to use in-app purchase. So your solution will be start on the web first. Start the business there first. Then you have apps for iOS and Android. Therefore you can go over, not provide the in-app purchase. In-app purchase? Yes. How is Apple is reviewing this kind of app when you sell international? How is reviewing? The hard part, I don't know, there's a guy there. They got no idea. Yeah, hard part, I don't know. All I see is that you need to provide the product ID to press the description and price. And you need to provide the screenshot. How the screenshot is being used to review in-app purchase just beyond me. But we take note that the Apple reviewers would need to see the in-app purchase and they have their own opinion of in-app purchases. Another thing is that if you want to do a subscription, you need to have a cloud service attached to your app. Even though if it's a SaaS, you can say if there's a service, if you don't have cloud service, it will be very likely that your app will get rejected for subscription. Would you also provide the Sandbox and One? What? Sandbox and One. Yes, you can. You can provide, they will provide, during testing, during testing, and I believe test flight, you can use a Sandbox account, which you go to iTunes, right? You get to iTunes Connect, and then you go to a screen to create this thing called Sandbox Accounts. Again, you can use it for purchasing. What's nice with their Sandbox accounts is that the subscription duration is much shorter. I think one day becomes five minutes, one day becomes one second, and then one week and five minutes also. So you can test it really, really quickly. Any experience you want to share with regard to the, when it comes to writing in-app test cases for this? Yeah, one experience. When you do the first time without unit test, keep your receipt file as a backup. That will be very useful for unit test later on. And second, you need to separate your inventory logic, which means the logic that says, yeah, user has product A, B, and C, and A is labeled, B is disabled, C is labeled, which is the one that varies when there's a receipt. The one that varies the receipt, you will likely want to have a thingy that auto-generates the code for every build or every version so that it doesn't get tracked easily. But those things cannot be unit tested easily. So you can use it like the TP in-app receipt where you can disable some of these validation, like the hardware ID validation and all the stuff to run it in a CI environment. Good, no good. How do you handle RIPan? How do you handle RIPan? Have a special page in your website, in your app site. If you have an app, it's a good idea to have an app-specific site, like an adult app to me. And then just say that, refund request, how do you do RIPan? If you want to know, this is how I show RIPan. So it's quite simple, have a pair of page and say, how do you do RIPan? And then support, refund request KC. Just have this somewhere in your website and then point it to this page. When the user bought something from iTunes, they will always get a receipt. And you can always, they can always click a receipt and then ask for refund. That's why you should just do this for virtual products only, not realize products that's hard to define. Does it affect you? Yeah, it did several times. You will see a negative one in your iTunes Connect report. But you don't know who is this customer. Apple don't tell you. Apple don't tell you. Yeah, that's one of the hard things when you sell things in the app store. It's not, you don't know the identity of the customers. But, hey. So when you refund, when you do the superman, they send you the receipt. Ah, when you do a restore purchase or whatnot, right? You should see a cancellation date. Although I don't know how to do a refund on a test account to be frank. So just be, just be aware, there's a field called cancellation date in the receipt. That is usually empty. But if that's there, and that date is in the past, you should disable the functionality. Yeah. Okay. Thank you so much. Thank you. New product. Oh, okay.


Hi all, sorry for the delay to start. I have some problems in the laptop with the web application file we should use to the demonstration. Well, who am I? My name is Lando. I work in information technology more than 10 years. I work for a company in Brazil called Security Labs. We work for credit car company, government, and the older big companies in Brazil. I'm affiliated to Hakiaholic, which is an international group of security research. I had been working with penetration testing the last three years. I had discovered some vulnerabilities in web mails, like open web mail, web meow, and the others. Access points like the link, Citrix Metafame, and the other softwares. I wrote some tools that was used in some magazines famous around the world like Hacking9, and the PC world. I also spoke to some conference very famous in Brazil, like Huckers2Huckers conference called Breakers, and was invited to speak to others, like IT Underground in Italy, and the It1Take1 in Mexico. Well, I am from Brazil. Slide, speak by itself. Yes, when I arrived here, everybody spoke to me about Brazil. Are you from Brazil? Soccer, what do you think about the next games? How are they stuff? So, yes, I'm from Brazil, the city of Pelé, and the slides again speak by itself. Well, what do you see in this presentation? What is the Waf, who we are speaking about? Type of operations, common topology, passive and reactive mode, tricks to detect the Waf systems, tricks to fingerprint to Waf systems, genetic evasion techniques, specific techniques to evade the Waf systems, and what they fail to protect. Well, we have a small description he tried from the web application security consortium glossary, that basically say that web application files are a kind of files specific to layer seven applications in the case to the web application. That's tried to protect the application from hostility attacks in this specific layer. Well, they are also called deep impact inspection files because they look every request and they're able to look response. Looking for problems in HTTP, HTTPS, SOAP, XML, services, web services, and the others. Some web application files look to signatory attacks like you see in many files and the also abnormal content, behavior problems. That basically are the same we see in EDS and EPS technology. The web application files can be either softer, harder, or appliance. And basically they try to protect a web server. Some notes about the definition. Some new web application files can work both with attack signatures and the abnormal behavior. Some systems not necessarily need to be in front of the web server as most, where in most places we're accustomed to see. They can be directly put in the machine where the web server is running. There's not the only, as described in some older documentations that are really used as reference, able to detect only attacks in the incoming packets. It's able to detect attacks in bound and the outbound packets. Basically they are three types of operation modes. They are negative modes that are based in a kind of blacklist. They are positive modes, which are a kind of white list based, and mixed mode. That's what the most famous web application in nowadays is using. Kind of negative mode, and a kind of positive mode. Well, negative security model recognize attack by relaying a database of expected attack signatures like most of the EDS and EPS that we have today on the market. Example, for example, look for any page or any argument with the user input match any potential cross-site scripting strings. This can be, for example, strings, string dot from char code, and a combination of all these strings. Well, the good point of this kind of operation mode is that it's very fast to install and use it. Basically you have a pre-built group of rules that try to detect the most common attacks. So you basically get to apply, install, install, and this is running. The bad point is it's so generic, so in general you have a high number of false positives, more time to process, and also a small number protection. Positive security model also. Basically it's a kind of enforcement of the application logic. For example, if you have a script called the news.jsp and it have a field called the ID of the identification, that you only accept numbers from number zero to number 10 and they always starting from number O and going to six, five, five, three, five. Basically they construct a kind of rule that only allow this kind of data, this kind of the digit start from the number to the number and no other data will be exceptions and not be processed. Well, by the nature of the rule that's much more small and it's much more fast and have better performance and they also provide less false positives and you're having good rules. The bads are it take much more time to implement. Some web application fighters in the market provide some tools to automatically generate this kind of rules, what they call in general automatic learning mode or similar name. This kind of stuff is good and they are helping a lot in generating these rules. The big problem is that this rule is never perfect and it should work automatically. They need the basic to have human reviews. Else they will also be so leak in a fashion to doesn't generate a lot of false positives and it make the attackers life much more easy. Well, the mixed mode is basically the use of positive and negative models together which is happen most of the today's web application file. However, in general, I'll have one that is predominant, negative or positive model. Well, in general, what systems can be used in three different network topolors between our web server and the web client? This is the most common that we see in the internet. Also, which people call like reverse proxy. We have a web application follow in a reverse proxy appliance or a machine that is seated between the web client and the web server. The second is integrated in the all web server which is more common is mall companies and the third option that's not so common that's directly connected in the switch in part me or span or have that are older asynchronous names to the same that basically get a copy of all the data that a network that's a port to get. It's replied to another one when it's running the WAF system. Passive or reactive. Most WAF systems work both passive and reactive mode. In general, in the first days they are running in passive mode to identify false positives and in special when you are using some kind of automatic learning mode technology to detect this kind of problems and fix before put it in a production environment. In general, when it's running very well with the auto generate false positives it starts to run in reactive mode. A little back from Brazil. Here we have some famous guys from Brazil that probably all should be. The second thing that everybody asked me when I arrived here. Hey, and the Brazilian Jiu-Jitsu. What happened? It really works, blah blah blah. That's the answer. Brazilian Jiu-Jitsu really works. We can see Anderson Silva, Royce Grace, Rodrigo Minotaur and many others. Well, here we have some tricks to the fat WAF systems. Well, WAF systems leave several points that can be used that permit us to detect them. Example, cooks. We have example of Citrix Net Scholar. Citrix Net Scholar, in some case, that on cook in the application that allow us to detect it's running. As we can see in the cook field we have NES that is a Synchron of Net Scholar underlining AF and the other names. This kind of cook allows us to identify this kind of system. We can see in a real life site here. This site is a real site on the internet using Net Scholar from Citrix. We can see the cooks it added to the application and consequently allow us to detect the presence of the web application firewall. It's a common example of seeing or pointing that the web application firewalls add and should never happen. Science it should be stealth to the attackers. Header rewrite. Some WAF products allow rewriting of HTTP readers. The most common is the server field. What happened? Sometimes some web application firewalls when you request a normal and validate WRE or WRL they respond with a valid 200 code and the server field is set for the name of the web server or Apache internet formation service whatever. However, some products when we request invalid or hostile WRL or WRE it is simple remove the server field. So it's a thing that we can identify that there are some system that can be a WAF that is protecting the site and also some older WAF applications. Only override or allow the reader rewrite when we send some bad content. For example, requesting a hostility content below we have example. A valid and known hostility request should return a 200 okay status code and with the server that is the real server content Apache 2.2.9 running UNIX. Within the same server we request a hostility request for example a well-known web exploit that this web application trigger we can get the following response. The server was changed to Netscape Enterprise 4.0. So it's another problem some web application products they only apply the HTTP the reader rewrite in some circumstance that allow us to identify their presence too. Some WAF vendors return different HTTP response error codes in the same URL valid one or not. If you insert a hostility parameter for example just to be more clear since my English is far away to be good suppose we had a not in this case for example a page no is index.php.p ID equal one, two, three for example it's a kind of suppose valid request and this process it. If you put in the same request that you always return the code 200 for example it should be processed okay. If you just change the one, two, three to something like pipeline ID or ID that is hostility content in general people trying to execute the operating system command this can return for us a different HTTP response code for example 404 as if the page index.php doesn't exist which is not true so we can be sure that it's not the content of the web server and the application in the middle like a web application firewall also the same happened with your request parameters that doesn't exist for example no is is to and the same parameters it will return in general errors four, two, four, four, two, three, five, oh, oh and the other so this kind of returns can help us to identify the web application files. Here's a valid example that was used in one of the application files we tested. A valid request we felt still the parameter that doesn't is to return it 501 status code method not implemented for sure it's not valid and also we could note that the options sorry they allow was changed because they allow option of HTTP header in the normal request never returned the trace option so there are a lot of small points that can be used to detect it's kind of systems. Well, back in some of the web vendors provide a filter to close connection you can use it to drop the connection block users and they also have the possibility to interact with external files. Well, ModSecure is a common example. They have a future called the drop the action that immediately initiate a connection close the action using a thin packet. Attackers can launch well now attacks that mismatch the built-in rules from ModSecure and analyze the back to the thin packets and try to identify the presence of a web application file possibility, ModSecure. This kind of filter is not available in older versions of ModSecure. Some of these techniques we are using to identify web application files can be used in the house to identify intrusion preventing systems that are available on the market. What more about Brazil? Well, back in tricks to fingerprint off systems. All off systems have a built-in group of rules in negative mode that are that rule is based in blacklist that you spoke on the start of the presentation. These rules are different in each product. These rules can be specific to a well now vulnerability. For example, the old EES unique code attack can be a generic rule for a well now class of a vulnerability like SQL injection, cross-site scripting, and the others. And in general, these rules are associated with action like drop the request, redirect to another page, et cetera. Attackers can create a set of attacks that test for a range of vulnerabilities that most off system protects against or not. In this way, we are able to identify building rules of a product and consequently what product it is. What it means? We can generate a database of a big variant of a web application attacks that some web application files detect and the others not. Based in the detection or not detection, we can identify the software. For example, suppose we send a request with a HTTP method that is different from 1.0 or 1.1, for example, using 0.9. Some web application files detect this and block the contents or return a different HTTP code or can redirect you to another page, et cetera. Request content language in a method which is different from post. It's the same. It's not a valid request that all web application files detect, but some detect. Where are you with a recursive path? Even invalid. We can request, for example, how are you accessing recursive paths even that doesn't exist. Some web application files will detect the recursive path using UNIX notation or Windows notation. Also it can be used to detect the difference between them. Request where cookie matches the same name that is also detected by the full by some web application files by other not. Request is where the way he matches the string or the way he itself. USR, X11, R6 being X term. In general, we can create a really big database of this kind of checks and run against most web application files and then we can detect based in what can detect and provide a different response for us or not and in this way identify what web application file it is. More or less like we do probes in operation system, basically the idea is the same. The attacker can go deeper and create several mutations of the same attack using, for example, evasion methods. These evasion methods can make a well-known rule be bypassed or not, and these also can be really useful to identify more precisely systems. Some time identify different versions, a new version from our old version. Also some attack is presented on the start on the tricks to detect off systems in the first slide can be used and they help to identify system like the example of C-tricks net scholar. These techniques can be used to generate a big database and as we spoken to create tools to detect this kind of web application files automatically. Well, generic evasion techniques. Today we have a wide range of techniques to evade the EPS and some off systems. Most of these attacks works because we have bad normalization, for example, encoding, et cetera, and the canalization that the implementation in the off does not work very well. Weak rules, the full week rules built in the off systems. We can see a big number of products in the marketing that have pre-built rules that are really weak and they allow a big number of possibilities of evasion. Evasion at network transport layer in some EPS and some off systems depending when they are running. For example, if they are running in port mirroring. Here we have some examples of common generic evasion techniques. A very common generic evasion techniques to bypass some EPS systems is add SQL comments to parameters instead of space to try bypass off systems and some EPS systems. Also is put words in case instance TV and the case instance TV to try the effect some of these rules too. This tool is very common. SQL query encoding is common. We use resources that the on database provide for us, like stored procedures, et cetera, that allows the attacker to encode and decode data. For example, 2x decimal directly in the database. In this way, we can bypass some EPS and the off systems in some case. Double encoding that basically is the same idea of the original encoding, but the sigging itself, for example, the percent is encoded too. So you add a double encoding. Where he encoding? Like, for example, only code forward slash that was possible some time ago. Don't remember exactly how much. For example, in the tipping point EPS, using this kind of technique also applies to some EPS and some off systems. HTTP request is mumbling. That basically is a technique that some attackers use to create a HTTP packet that will be parsed differently in the back end web server and in the web application file, reverse prox or whatever that is in the middle of the connection. If a packet fragmentation in some case, depending off the topology is possible, that basically use the question of fragmentation and the reassembly to bypass the same. Brazilian carnival, that everyone always ask, probability of you know and who don't know should go to Brazil, to the carnival and you appreciate boys, girls, everybody. Specific techniques to evade off systems. Similar as attackers can fingerprint off system as presented, they can use a technique to precisely identify which restriction of a rule applies to a specific class of vulnerability. What I mean, attacker can insert a hostility SQL injection, for example, to a parameter and this expect to be detect and the action take it. For example, a different HTTP error code be returned, a page redirection, et cetera, or simply generate a error code or the normal page be displayed depending off the web application file. The big point is, we can try a high number of different hostility queries and discover exactly what the SQL query is blocked and wishes not, what word, for example, select from where exec, she's XP underline, exec and the other query are used. If we are able to detect these strings that are used by the web application file to built in Hulus, we can try to bypass them. Using try and the ever, it's possible to identify the combinations of these strings, wishes allowed, wishes not. Also, it's common, we have a Hulus that is based on combinations. For example, only select is not blacklisted but if you use together select, from, and where together, it starts to be blacklisted. So, based in this information, we can try to construct SQL injections in the case to bypass the web application file. Also, this can be used to deny, also this can be used to detect how the rule was built. Let's get a real example. It's not can be used the only way if SQL injection, it can be used with a lot of other attacks like cross-site scripting and the others. Real life, in a penetration test, we realized maybe three months ago, we had a client in Brazil, wishes a bank, that had, we had made a penetration test before and we had, we got the access to the system. So, they got the solution from Citrix NetScholar and now we made the penetration test again. All we have to do to bypass the Citrix with the same problem was to identify what strings was blacklisted and what not, what combination was and what not, like in the previous slide, and then we identified the same query we used in the last time to retrieve a lot of information from the bank, we could use it, just removing, removing the query, we fed out new words in the query, using some database query encoding and removing all the single quotes and the Citrix NetScholar was bypassed. So, a lot of the systems that people sell today that are with WAF system are really good, but we cannot trust in the built-in rules, they are only, they help, but they are really far away to be perfect. In general, what are big pitfalls that web application firewall have to have problems to detect? Cross-site scripting. Cross-site scripting is really hard to effectively detect and block. It's extremely mutable and consequently very hard to detect, in special when we are dealing with the outbound traffic. The outbound traffic, heavy rules that in general can't be so, so enforced like inbound. For example, in general, we get a lot of news from third-sites and the older content. These older sites or older content can be hacked and used as a source to infect your site in the outbound connection, even if using a web application firewall. File uploads, many web application firewalls do a great job in file uploads. However, we have some tricks that's really hard to bypass. For example, when you insert web shells inside the images and call it, for example, via LFA, local file inclusions, via comments in JPEG and the others. Also, remote command execution based in server response is very hard to detect. Some web application firewalls in the last year, maybe, tried to detect some remote code execution. Like, for example, the return that when we type in a UNIX machine, the command ID, the command UNAME, PS, LS, they have a kind of standard format. And there are rules that try to identify this standard to identify it in the outbound traffic. It's a good idea, at the last thing they start, science it could detect a lot of people that had broke into the web server. But if the attacker had the access, he's able to, he's a huge comment on the web server. It's not possible to really detect science if he can use a lot of tools that they all will encode providing the Linux or even they all shell, like a bash, you can turn it in X code, you can do shift right, shift left, and a lot of encoding in the output that will not trigger the rule. So this kind of detection is not always useful and can be easily bypassed. Other problems that is probably one of the most hard to detect is logical and design flaws, which is extremely hard by the nature of the problem. Basically, suppose you have a bank and this bank have a place where you can check how much money you have in your account. It's common that this data like a chance, account number is passed as parameter. If there are nor, there is not a stronger check in the application to grant that your sessions, your cooks is related with your, with this data that's passed this parameter, attackers can manipulate this kind of data and the access information from older users. This kind of problem is very hard to web application files detect and block and the very, and the holders in the same idea. Well, our time is finishing. We had a 10 minutes, it's over. I would like to thank you all of you. Also, the site Hakiaholic is looking for new members. We have a private forum where we share a lot of stuff. If you someone is interested, we are looking for new members. Thank you all and sorry for the problems in the laptop and the delay to start.


You're doing well to be here. I know I have the conference head going on. I'm an English professor at the University of South Florida. And that's in, it's actually not in South Florida. That's another story. It's in Tampa. And I have a project that's about 11 years in the making here. So I'm excited to show it to those of you who are here. I have a few kind of overview slides just to give you a sense of it. And then I'm going to show you the resource. If you want, a lot of you have your computer. It's at writingcomments.org. So this is a very comprehensive rhetoric. So I direct the undergraduate writing program. It's a kind of standard program for a lot of universities. Probably some of you took it.  It's a large program. And what I've tried to do now working with colleagues is develop a resource for that audience at this point. So the writing site we're kind of hoping will become a community. Right now in a sense it's, you know, there are resources there. But ideally it'll be a place where people come because I think the idea that a textbook is in a PDF is to me a little outdated at this point. So we wanted to be an online space and we wanted to be a creative place. The way it is right now, I'm going to show it to you. It's really a big rhetoric. But it's a core, the content that we have there right now is appropriate for composition, which in itself is a good start. But my hope is that it will be good for courses in creative addiction and poetry and technical writing. I mean, my hope is that we're going to be able to grow it. It's somewhat interactive right now in the sense that, you know, there's the ability to talk back and have dialogue and chat and sort of some features that are a little bit interactive there. But we hope to grow that even a little bit more. We wanted to be very comprehensive and right now it's basically the content that I wrote a while ago. But the idea is that it's going to be updated with video. We have some video in the site that's been developed. Podcasts, video cats, interactive kinds of quizzes, that kind of thing. But also it's kind of nice to have a resource. Every institution has its own resource. At my school it's fyc.usf.edu. So that's kind of common, but it's kind of nice to... I think one of the things that these new tools are doing is to kind of break down the barriers that typically have defined equity. General Grapp talked about the problem in the humanities being professors on talk with one another. There's kind of an isolation, close to classroom door. Well, we're feeling like writing commons in a sense there's an opportunity for people at different universities to talk with one another. So I'm going to grow that a little bit. One of the things I kind of added at this particular conference is this idea of enhancing retention. Something I haven't really thought about. We definitely have a retention problem at my university, but that's one of the takeaways I'm taking from here. I never really thought that, you know, I never made that association that... I mean, I never made this sense that, oh, well, one of the things holding people back, congratulations, I can't make a textbook. Who is that, Brianna? Well, you know, sir, certainly learning and teaching are part of it. So I took these notes this morning from Cable Green. I don't really know that those are... I'm kind of curious where he gets that, but he said this morning that there are 55,000 students in the composition. At my school, we essentially have, like, a unique seat count of 5,000, but we have about between 10,000 run through our program. We have 88 teachers. We're a big program. We're probably in the top 10 in terms of size. So probably 55,000 is reasonable. So if the whole country uses writing comments, we save people, well, $5.5 million. I like that, I guess.   All right. So in terms of our goals, I like the idea of peer production. I'm very interested in that. If you look at FYC.usf.edu, we peer-produced our curriculum. You know, there's an argument that you can have a standardized curriculum, but you can't. If, you know, you have the holomassian idea of a public square, where the best argument wins. So I think that, you know, it's very possible to create a space where we really provide a resource that's up to date and very good, you know, very, very high quality. I think that's definitely one of our goals.  Do they want to work on writing comments? Yeah, they're pretty supportive. They're good people. But they would rather write their own stuff. And this idea of constructing this pedagogy, I think it's very important. The idea that, you know, we engage our students as learners interactively, positively. So kind of hoping that writing comments will fit into that. You know that whole thing about the wisdom of crowds and, you know, I think that you go back to teacher researcher movement and the notion that, you know, there are technicians that can be involved and they're very teaching and engaged. So, you know, I mean, hopefully we'll tap into that energy. I feel like we have done that in our writing program. If you look at FYC, it's really, really good. I don't want to jump ahead in my comments, but you develop communal agency, which is something that is very important. So, okay, I wanted to be more than just a textbook. I wanted to be interactive. I wanted to be entertaining, you know. So, we're getting into the peer review part here. Aren't you guys probably already looked at it now? All right.  So, in a way, you're kind of modeling the behavior of a writer when you're engaging in a construction of a textbook with your class and your students. Which is, that's a good guy, right? How many of you are English people? Do I have any? Okay, okay. That's good. So, we're trying to make it interactive and creative and something that's going to be... I mean, I think that idea about OER shouldn't just be free. It should be better. I'm not saying it's better, okay? But I think it has a potential to get better, do you know what I'm saying? I like that idea. So, okay. So, what's the history? I mean, there's a set of work on this for 2001. It's kind of crazy. I was in that earlier sessions on textbooks and I mean, I don't think it's necessarily correct to make the argument that professors are paid to write textbooks because we're paid to do research and theory and scholarship. And that's, you know, we're not going to get... at a research university, you're not going to get anything for writing textbooks. So, you're probably doing that on your own time. So, you want some kind of benefit from it. And I, you know, I did this book in 2001. I published in 2003. Got the Distinguished Book Award. I was very happy, 2004. But they didn't sell it. And so, I'm trying to get my own motivation. I would go to the conference and I'd say, they have all their books out. And I was like, well, where's Randy Gammons? I got one that's called Gammons Writing.  But, and they'd come and they'd try to sell books to me. And they'd go, hey, Joe, you know, look at this great book. I ended up on a search. Got a new version. It's based on World's For Writers, da, da, da, da, da. It's like, where's Gammons Writing? Because what I ran into, then, they said I was ahead of my time. That was, you know, that was their comment. But anyway, it didn't work. And so, like, it didn't do anything. You know, it was behind a lock. So, like, I came out, somebody told me at lunch that I had Stockholm Syndrome, which I thought was hysterical, because I was defending the publishers in the earlier thing. Because I felt that the, you know, I felt, I think it's a misrepresentation to say faculty should be writing textbooks and then we're funding, then all these textbooks are funded by, and I don't think that publishers are bad people. I think they're lovely people. And they're very engaged in general. In trying to make a university that applies to students. I mean, they're not, like, just about the block. They're the publisher that I work with. But in any case, so, like, okay, okay. So, I got this product. It didn't do that great. But I got copyright back. And slowly I put it back up on the web. And now, now the goal is to sort of peer-review it a little bit. And to grow it a little bit. So, we're working on that. Oh, I should know what it looks like. So, this is actually kind of the third version of it. And we're still kind of playing around with it. Before when people came into the site, they would just come right to the textbook and we thought that they were overwhelmed. So, what we've done, I don't know, you know, a lot of this is what's nice about it being digital is you can change and try to make it better. But now you're going to click open text to get to it. And then about us is one of the things, in our writing program, we do these custom textbooks. And we do that for a good reason. You know, we pick up the special curriculum. And I've been a little upset though because they end up being 45 to the bookstore even though we've exported the models. We have paid 15 to 20,000 per book. And we've got the tech fee funds to pay for that. And students can download them from SharePoint or Blackboard or wherever even if we have that behind a net ID. But it's still the book's too expensive, in my opinion. So, I don't know. But anyway, one of the ways we're going to grow the book in the future is we're going to use this book and now we're going to bring in the past three years of custom textbooks, some of the best stuff after we kind of take it through a peer review. You know, I'm still trying to figure out the best model for this. And I think, I mean, I've tried to set up a very distinguished editorial board. I mean, I was really pleased to see what Howard Rungold said. Yes, it's kind of fun to do that. And I think I need the inputter of good people to make it viable, right? I mean, I'm not saying an assistant professor can get too much out of publication. But on the other hand, if we demonstrate usage and it's, at least it's peer-reviewed. You know, we're not going to have anything there. So, we're going to rub things through a staff, assistant volunteers, and some of my doctoral students and some of my colleagues that are kind of left. So, we're going to have a staff and then have an editorial board. And we're going to try to do it. Oh, in terms of what's there, I should show you that. It's a lot of stuff.    And it's kind of arbitrary, but I was trying to chunk it and make it look good. But, you know, there's basically everything you would expect in a, you know, everything that's in everyday writer, St. Martin's Guide, all that kind of stuff. But I kind of embedded some videos. One of the things I'm really struggled with is the coffee, really.  But, on the other hand, I had that experience with Pearson. So, this was my, this was where I came down on it. I could go with Bedford. I like Bedford. They're good people. But I was so afraid that if I did that, we'd get locked behind a password. It was with Pearson forever and it just never got out there. Do you know what I mean? So, now I'm just putting out there. And, you know, I figure I'm a tenured full-progressor. You know, I need to worry too much about things like that. Do you know what I'm saying? But that's still kind of a struggle, you know, but that's kind of where I'm at on the project.  So, that's probably the big ol' timeline. It's a big project. It's been taken forever to get to this point. But, you know, next year we'll use it in our writing program. And I suspect other places will eventually use it. Do you know what I mean? And if they do, that's great. There's a lot to talk about distribution. Frankly, you know, if my school being so big, I'm busy enough with that. But if other people want to use it, I'm happy about that. Do you know what I mean?  I mean, I took a lot. If you drill down in there, it's a big book.   I'm in learning. And, you know, maybe I should try to work out some kind of print-on-demand kind of feature. There you go. Yeah. And thank you for coming, because I thought there'd be nobody here. No one's coming. It's only been, like, 11 years to get here. Yeah. So it's going to use it on Southern Florida University, right? Yeah, Southern Florida University. Yeah, Southern Florida University. So what is it like? How do you think it's going to spread, just, like, for now? What are your ideas on that? Well, I have a really good, you know, I have Mike Conquest, who's president of NCTB, an editorial board. I have, you know, I'm president of the Florida Life Program Administrator. So I meet with, you know, Florida, BCF. We're all really big schools, you know. I don't need it to be very big.   But we're going to push it out there. We're going to, like, do a press release. I'm going to try to, I'm going to put it on the WPA Journal of the Serve, you know, who's writing program, be sure the service is huge. It's my kind of people. So, and then, you know, I look at Google Analytics, and, you know, I've had, like, in the past year, about any kind of marketing like that, I've had 18,000 views and a lot of that spam bots. But even if, like, half of it is, and you get 9,000 viewers, it's been three minutes on average on this slide. And I'm great, but that's not bad. And I think once I push it out there, I think I might get a little bit more. Yeah, but maybe not. I mean, you know, like, coming here, you know, coming, coming, you know, just by looking at the contents, I'm not real clear what level course is this thing, which is a question. Right now, this is targeted toward composition, which is 1101, 1102, or, I don't know why, but this thing is on. The genres that we have are the ones that are in the standard, like everyday writer and that kind of thing.  So it's a freshman comp. Right now it's a freshman comp. But like, you know, as I started working on this, I was like, Oh yeah, yeah. Well, you've done the freshman comp and you want to peer-produce it. Why can't it be for poetry classes or for journalism writing courses? Why can't it become, and I, you know, I was lucky to get writing commons URL because I didn't, I had Joe Montza. I had a college writing.org and I had the .com and I felt like I needed the .com to be able to work. But I mean, if people contribute and we see, I think what I have to do, and we have to work out the details, but the peer review has to be good. That has to be serious. Or it, to me, it's not going to work like this. Now, there's another project, Charlie Love, Great Guy, who's working with Parler Press on writing spaces, which is really cool and it's like the download PDF sort of thing. And they've had good success within composition, but I would like it to be something more than that. Will it become something like that? I don't know, you know what I mean? It's like an experiment, right? It's an experience. We're really not there yet.  It's been kind of quiet. It's taken a long time. I haven't told anybody like this. We have added a discussion for every article. Okay.  And we're, in our writing program, we do, you know, a mid-semester program evaluation where you get their feedback on the textbook. So we'll be getting it then, too. But right now, I don't really get it. I sometimes I get some nice emails, you know. So like, it's really interesting, like, but I kind of, you're talking the same tension that I, I kind of get the sense, like, Wikipedia, for instance, having the early days, like, you know, all these layers of peer review, whatever. Right. And eventually, the Wikipedia model, for them, being what they ended up doing. The wisdom of the chaperones. Exactly. We're just like, anyone can contribute, and there's like, kind of, it's not like that. It's not like that at all. Well, they do have editors who can take control of things and guide things. But I mean, any one of us with a laptop could log in and change, essentially, any Wikipedia agent.  Exactly. But like, we can still change it even though, like, you're reverted. So, I'm just kind of curious, so especially when this is supposed to be, like, academic output, you know. Well, that's why I didn't get the Wikipedia model. Like, that's why I'm doing the peer review model. Okay.  But it's a different kind of project. It is a different kind of project. It's a different kind of project. And I think if I did that, you know, because this is written in Jumla, and you could, you could edit on every page and people could go in and they could, you know. But, I think it, I think if I'm really going to be able to be successful to grow it, you know, I, yeah. So, I was just like, thanks for that question.  Like, who are the people who you think would care the most? Like, all this is a peer review, this isn't like legitimate. Oh, well, I'm hoping that assistant professors and doctoral students will be my chief authors and probably be engaged in the peer review. So, you know, doctoral students are trying to work on a digital footprint. There's really no mystery about whether or not social media is important to a doctoral student. And, so, you know, I need to work with, with other writing program administrators and get their doctoral students involved and assistant professors and that kind of thing.  It's kind of like I'll put it out there and we'll see. Well, it's going to be University for sure. Well, yeah, and that's going to be big for us next year. I think that's going to be really big. And we have partnered with Bedford and they've done a good job customizing the books. We have so much content, but now I have to make sure it goes to a review or I can't, you know, I lose credibility. And I really, I really, really want to do that. So, if any of you are interested and you can shoot me an email, joemoxlate.gov.  But, you know, I was excited when Casey Green was like, I was like, I was like, I was like, I was like, I was excited by that. You know, did you notice that way? You didn't notice it, but I did. So, you spoke about peer review of the actual student writings, but, and maybe I just missed it, but I wanted to know what are your plans on evaluating the actual tool. So, the actual, is it really for, does it really foster creativity? So, I think it's really for, does it really foster creativity? Does it really help? Oh, well, there's a, there's a whole community, there's a whole community tab. And it's, I don't know how that it's loaded here. We're, we're just making a shift from, we had a different version where, but we have, we have a blog engine and then we have this job social, which is like, it looks exactly like Facebook and they can create their own blog sites, teachers can create their blog sites and their class sites and discussion forums. I, I don't have a great server, in case that happens, that could be a problem, but it's there. And, I don't know, you know, we've been arguing about whether or not that should be there or not. You know, though, you know, we get different advice about whether or not it should be there. But we have this community tab and you click it and you can log on, create your own blog, your own webpage and, I guess, I didn't ask. And that's part of making something different. Like, I don't think students really want a PDF like they used to. I guess what I was asking more is, when the new instructional design tools, especially technological tools are introduced, there is usually some kind of new or old adapted way to evaluate them, so that was my question, because how are you evaluating your tools of the, and, I don't have any more time, by the way, so what are your plans on evaluating the actual platform, the actual website, how, if it is helpful. Are we just going to be, oh, the students gave positive comments. It's wonderful. Are we going to see if their writing improved compared to somebody else? Oh, well, I actually, I'm actually building a rubric portfolio tool in another project that does that. This doesn't do that. And I would like, you know, ultimately, that's proprietary and I don't have USF tech fee funds, so I can't really use it there, but we aren't developing such tools, but I would sort of like to develop that in a more open way, but right now, we are building a tool like that, but that's something different. This is just, at this point, we're just trying to make a really incredible resource for students. You know, I hadn't seen it since I came here. I had no idea about it. I know, but I think they make you give away, I think they make you give away that I was commercial. And still, it would be very difficult because then somebody goes and I, you know, and I don't think it's right. I think it's mistaken, you know what I mean? I think both. It's like ETB, so I try to piece these literations. You know, students shouldn't lock it behind the door. You should be free. You should be out there and open. Yes. Thank you. All is done.