Hi, folks. Welcome to the session on Trove. I wanted to start this session off by telling you guys a short story. So a couple of years ago, I was working on developing a new application. I was working for a big company. And as most applications, this one too had some data requirements, and so we needed a database as a place to store the data. And as every big company, we had a dedicated IT team, and so I sort of shot off an email to the IT saying, this was the sort of data we needed to store in the database, and got back another email from them with a Word doc with, that was three pages long, asking me to explain what sort of data requirements we had, what sort of backups we would need, things like that. And it literally took me a day and a half to fill up the Word doc and send it back to them. So, where I'm getting at, let me tell you about Trove. My name is Nikhil Manjanda, and I'm the PTL for Trove, and I'll be co-presenting this with Amrith, and I'll let Amrith introduce himself. Hi, my name is Amrith. I'm one of the founders of a company called Tesoro, and we focus on Trove, and we work only on Trove for OpenStack. So, we're going to give you an overview of Trove, its architecture, follow it up with a short demo, showing you some of what Trove can do, and also sort of finish up with what we have in Juno and what we're looking to implement in Kilo. So, let's get started. Amrith will sort of carry on the first part of the presentation, and we'll talk about Trove and its mission statement and architecture, so I'll hand off to him. Thanks, Nikhil. Okay. I hope you guys are able to hear me okay in the back. Like Nikhil, I've spoken with many companies which have tried to deploy a database-driven application, and one of the major pain points which people talk about is that it takes sometimes months to get an IT organization to provision a database. So, we'll show you how Trove can change that. Just to set our expectations, how many of you here are using OpenStack already in some form? How many of you are using a database in OpenStack? How many of you have considered Trove? How many of you have tried it? Okay. You notice that number keeps coming down. Hopefully, by the end of this presentation, more of you will want to try Trove. Okay. My contact information is here. It's also on the last slide. We will be making these slides available. Copious notes pages on this slide, so you should be able to use them as reference material later as well. And, by the way, if you guys are tweeting about this, some hashtags for you. Oh, okay. Now it works. Oh, forgive the aspect ratio here, but... So, what is Trove? Trove, by its own mission statement, aims to make it easy for you to provision, operate, manage databases in the OpenStack environment. Connect. Oops. Okay. And the goal is to make this easy for people who are using OpenStack to provision and operate a scalable database framework. That's all that Trove does. A lot of times, people get confused when understanding about Trove. So I want to spend a couple of minutes on this picture here. Shown in the middle here is a database. It could be relational. It could be non-relational. It could be any database. And on top is your application. Trove sits below the database, and it's only interested in things which we call the management and provisioning plane. We provision databases. We do some amount of administration, some amount of management. Trove, by default, does not touch your data, does not execute queries, does not get into the data plane. That's what your application does. So your application creates tables or collections, creates indexes, things like that. It queries the data. It inserts data. Trove doesn't do any of that with one small exception. When we do backup and restore, we touch your data. With the exception of that, Trove only does provisioning, management, and things of that nature. With that very broad distinction, it's very important to understand Trove operates in the provisioning plane, apps operate in the data plane. And stop me at any point if you have questions, all right? Okay. So I like pictures. Everybody has gone to a Coke machine at one point or another. Nikhil had to fill a three-page document. Nobody will fill a three-page document to get a can of Coke.  Trove is very similar to that, with a small exception, little plus-plus at the end. So think about a machine, which if you walk up to and you press a button, there's a button there which says MySQL, somewhere in the middle.  And the plus-plus says it'll also manage it for you. It'll take backups for you. It will do replication for you. It'll maintain a replication slave and make sure that you have a highly available master slave network. This is what Trove does. And it does it for a bunch of different databases. You could do a clustered Mongo deployment with Trove. So you have OpenStack installed, you have Trove, and you can click a button and say, Trove, give me a clustered Mongo instance and it'll go off and do that for you. That's basically what Trove does. You do not have to fill the form which Nikhil had to fill, and instead when you need it, you get yourself a database instance. That's basically what Trove does. And Trove is a OpenStack project up there. I'm sorry about how the pictures came out, but now you notice I hope that's not going to screw up your demo. Trove is a project which relies on the other core OpenStack projects. Nova, Cinder, Swift, Neutron, Glance, and Keystone depicted here. All of the core OpenStack services or OpenStack projects are used by Trove in order to give you the databases of service. Trove exposes an API. You can program against the Trove API. And Trove uses only the standard documented APIs of these services. So if you already have a deployment of OpenStack, you deploy Trove on top of OpenStack, and that's it. Any questions? I'm going to pause and see if anybody has any questions at this stage. Let's talk a little bit more in depth about how Trove works. I hope folks at the back of the room are able to make out the lettering on the slide, but the code is approximately as follows. The stuff on the left in green is things which relate to Trove. The stuff on the right are the other OpenStack projects we talked about. Keystone and networking all the way at the very end. Glance, where we store images, Swift, Cinder, and Nova. And on the left-hand side we have Trove. And you notice there's this little overlap in the middle. Those are places where Trove has interactions with the specific underlying project. So let's talk about a life cycle of a simple instance. The user comes along and says, give me a Trove instance. Comes into the Trove API. The Trove API is going to do something, Trove is going to do something, but at the end of the day an image gets picked up from Glance. That image is given to Nova, and Nova spins up one of those compute instances and we'll talk a little bit more about what that instance is. But that is the database instance you requested. So you hit the button and you say you want a MySQL instance. An image for MySQL is loaded into Nova, executed. The connection endpoint is handed to you. That IP address, port 3306, for example, that's your MySQL database. Okay. You could also go hit a button on Horizon Console or with the CLI and say do a backup of a database for me. You created a database like this. Do a backup. That backup gets sent off to Swift and it's stored on Swift. You could at a later point and time come along and say I took a backup the other day, give me a new MySQL instance based on that backup. In that case, we're going to take the backup off of Swift, load it onto your instance, hand you the connection endpoint, that's your queryable database. And then you're going to create a new MySQL application built on top of these services. I'm not going to go into a lot of detail about each of these because Nikhil is going to be showing you a demo. But at a high level, you understand those services, the standard OpenStack services and how Trove builds on top of them. The one interesting thing here is the compute instance up top. Nova uses guest images for compute instances. A Trove guest image is slightly different from a Nova standard guest image. Okay. Next one. And this picture kind of shows you that distinction again. On the left-hand side, if you will, the database agnostic components. These components work for all data stores, whether it's MySQL or Mongo or Postgres etc., these are common. On the other hand, if you came along and requested a Postgres instance, which is now supported, that guest instance is going to be Postgres specific. So there's a guest image which has some operating system. Let's say it has Ubuntu on it. And it has Postgres installed. But it also has that Trove guest agent, which is a Trove specific component. Realize that the Trove API is a common API no matter what data store you have. There's an API which says create an instance. There's an API which says take a backup. But the way in which you spawn Postgres is different from the way in which you spawn MySQL. The guest agent is a thing which understands the distinction. And the guest agent is specific to the database. So you have a Postgres guest agent and a MySQL guest agent. And you install that on your what appears to be like a Nova image. But now it's a Trove image. The distinction is a Nova image just has an operating system. You may have a database on it. A Trove image also has a Trove guest agent. And the Trove guest agent talks to the rest of Trove over a message bus. Very high level architecture-ish kind of thing. Does that make sense? I see a couple of people nodding but everybody else seems to be like it's lunchtime already. No? Okay. Go ahead. Yeah. All of the stuff in there is in the image. Correct. So that thing there, the green box is the image which is going to be spun up based on or is the instance that is going to be spun up based on an image which is sitting in glance. Okay. This, if you will, is the end result. Are we necessarily distributing all of that stuff? Potentially not. There could be on that guest image sufficient code which says the first time I boot go get me the database from somewhere. And that might be some local PPA which you have in your organization. It might be some other licensed database which we're not allowed to distribute but we distribute images, HP distributes images, we don't necessarily have to bake all the software onto the image. Does that answer your question? Okay. Standard Keystone authentication. And again, good point. Database authentication different from trove authentication. The database user, the one which your application uses, that's something which you set up in MySQL, you'd say create user. Very different from the admin user who potentially is setting up your trove image or spawning your instance. I saw another hand up there. Yes, they do. You could probably talk at some point about your deployment in practice where you do that. Do you want to do that now? No. We can talk after the session too. Only anybody with a specific set of privileges can delete the instance. But you really don't want to go and delete the NOVA instance from under the covers. That's really a NOVA instance. You could it's exposed if you have credentials to go shoot the instance from NOVA. Absolutely you could do that. Not recommended. Your mileage will vary if you do that but it's possible. You should hold that question till futures because very shortly you will be able to do that. Sorry. The question is can you provision a Galera cluster using trove? Nikil is going to be talking about what is currently supported and what the futures are. So, okay. So, now that I've convinced all of you that you should absolutely try trove, I have to tell you where to get it. Okay. Trove has a couple of pieces. There's the CLI which you have to get separately. There's trove itself which you have to get. The first two tell you where to get it. Get clone. You're off to the races. If you're looking for blueprints and other things, specs, what we're planning to work on. The third one. And I told you that you need to have guest images. There's a project called trove integration which has all the piece parts for you to build a guest image. There's standard guest images which are available and if you want production ready guest images, contact me after the presentation. I'll tell you where to get it. Oh, great question. Trove was released in Icehouse and now available in Juno. There's a significant bunch of improvements in Juno, so I would recommend you use Juno. But if you want to use Icehouse, sure possible. If you happen to use Havana and you want to use trove, we're not covering it in this presentation, but talk to any of us after the presentation. It's perfectly doable. But Icehouse or Juno are the official answers. Okay. So I just want to end and pass it over to Nikhil by talking a little bit about modularity and trove. So trove works with any data store. Works with MySQL Postgres Mongo, Cassandra, Coach relational, non-relational. I'm playing with a graph database. So how do you do all this? And the important thing to understand is we do it using strategies. So now that the previous slide showed you how to get the source data, once you get it, you'll notice that there's a directory called guest agent, which is all the data store specific stuff. And you'll notice there's a folder called strategies. And I'm going to talk about how we do backup. Because MySQL backup is very different from a Mongo backup. So under strategies, you notice there's a folder called backup. Okay. Well, you go into backup and you notice there's actually an implementation for each of the backups. So MySQL has an implementation there as does Postgres as does Coach. So let's look at what MySQL does. Okay. So there's three actual implementations of backup. The first two effectively do full backup and the third one does incremental backup. Now, when you deploy trove, there's a config file which says, when I do full backup, what do I do? You probably said it to InnoBackupEX. Which means when you go hit the button and you say give me a full backup, that's the actual method which gets called. What this means is if tomorrow I want to implement a new database and support it in trove, there's a small set of things which I have to do and one of them is implement backup. So I'd have a if I had a new database, let me look around for a person here. Okay. Oracle oracleimplementation.py is going to be a file in there and that's going to have a mechanism to do oracle backup. That way there's no change to the to the trove oracle. There's an implementation of a guest agent and you added support for a new database. We do the same kind of thing, of course you do it for backup you must do it for restore, which you do the same thing for various capabilities and that's how you make trove easy to extend to other databases. I think that's about the point where I hand it over to you. Okay. So Nicole's going to show you how all this stuff works in practice and I'll hit space far. So I just wanted to let you guys know a couple of points on how you can get started with trove. So if you're a trove user, there's a couple of distributions that Tesora and HP ship and you can go grab ahold of those the HP Helion dev platform or the Tesora DBS platform. If you're a trove developer we talked about this a bit earlier but there's the trove integration scripts repo that you can get clone and then there's a sort of overall utility called red stack that wraps dev stack and we'll take care of installing trove on top of dev stack. But if you're interested in what's going on under the covers you could install trove just using dev stack without actually using red stack. You just have to add the enabled services to local RC and just one caveat there is make sure that you enable Swift as well because trove uses Swift for backup and restore. So here's where I wanted to sort of dig in and give you guys a little bit hands-on demo of actually going through and provisioning an instance with trove. So just a minute while I bring that up. Okay, so here I have an installation of trove on top of dev stack. Can folks see that or does it need to be a bit bigger at the back? Good. Okay. So just talking a bit about the create workflow for a create trove has the concept just like nova flavors and so to create a trove instance what you need to specify is you need to specify the flavor idea of the instance you're creating and the size of the instance you're creating and the size basically corresponds to the size of the cinder volume that we use to store the data store specific files on. So trove if we actually do a flavor list you can actually see the list of flavors that trove supports and over here let's go ahead and create an instance called test or let's call it something more original trove in kilo and flavor two and size one let's run that and as you can see we get back the instance takes a while to build a few minutes so you can see the status it's in build and you can actually see your list of trove instances over here you'll see I have two instances the one trove in kilo that I just created that's in build status and then I also have a pre-created trove instance that's active so yes very good question so wanted to talk their optional parameters as you can see here there's parameters that you can pass in which tell it the data store type and version so right now without anything you specify a default data store type and I have the default data store type set to my sequel and 5.5 but you can also pass in what other data store images type of images that you have created so if you wanted to do that you can do the same on go and the version couple of other optional parameters we support az if you have them set up in nova using the availability zone option and we also support neutron and you can pass in the dash-dash nick option along with the network id or port id or the actual ip so you can look at that by just a particular thing and it will tell you what the optional parameters that you can pass in are some of the other optional parameters that you can pass in are if you want your database to be pre-created with the certain databases on there or certain users on there you can pass that in as well and then there's these other couple of parameters called configuration or replica off and I'll come to them so so that was the basic create workflow any questions on that go ahead so you get back an ip which is part of the connect string through which you can access the database you don't have ssh or you don't have access so let me take that back depending on the deployment the deployers could choose to give you access or not like in a dev stack where you're setting everything up of course you could configure it so that you have access you can ssh under the box and what not the deployments of Trove that I know of just give you back the connect string and don't really give you ssh access to the database that so and I'll talk a little bit more about that in a couple of slides later so just wanted to talk a bit about once you have this database now some of the Trove operations that you can do on the database and so what Trove how Trove helps you manage this database some things that you can do is you can resize your flavor this under the covers uses nova resize so you created a database that was too small now you need more memory more compute power you can issue a trove resize that will then do a nova resize and you have a bigger database you can resize the volume so if your cinder volume is not enough for your data store size go ahead resize your volume and now you have a bigger volume there's also some data store specific extensions that we've implemented and you got to see some of these optional parameters to create before so you can pass in or calls to create specific databases on your trove instance and create users on your trove instances or grant permissions and then if you want to manage your database offline using whatever data store methodology the data store supports you can enable a root user and then use that root user to create other users and other databases so quickly show that so if we do a trove list you can see the trove and kilo instances now active we could do a trove root enable and that gives you back a root with a root password so if everything went well actually I'm not sure what the connect string is so if I get the password we had gotten earlier there we have we have access to our database and this is a root user and we can use it to do whatever you want I want to talk a bit about what trove is doing under the covers here when you're actually setting up your database instance so when the guest agent comes up and when we get the mysql instance in this case is being provisioned trove is doing a few things it sets up same defaults for mysql we set up in odb only and we disable load data in file and select into our file trove also basically goes ahead and tunes your database for its configuration based on whatever flavor size that you specified so you have the ability to specify configurations based on different flavors so for example your max connections might be much larger if you're using a larger flavor or your buffer pool size and things like that so the guest agent goes ahead and tunes all of that depending on the flavors and what you've set up trove also has an api called the config groups api and that was one of the optional parameters that you saw earlier where you can target specific configurations to specific instances and groups of configurations to specific instances where this is useful is for example if you wanted to change the default character set on the particular database or anything that you could change in my.conf you can do that programmatically via the api as well so apart from tuning via flavor you could do that separately via the api any questions before we move on to the next slide other things that are going on so the guest agent is also sort of securing the the data store instance underneath we remove the anonymous user we remove non-local host users other than the ones that you programmatically create via the api we remove local file access mangle root password and also so we set up security groups in nova and depending on the the data store type the actual security group report is configured correctly for the data store type you can set up default rules for your data store instance if you want it to be accessed only to a certain IP range and then you can also create change that IP range programmatically via the api so trove does expose a security groups api that allows you to do that user ssh access I think somebody mentioned this earlier talked about this earlier is not required so all of your management happens through the trove api so you're free to turn off ssh access if you're a trove developer for debugging reasons you might not want to do this on your dev stack box so you can go ssh on to the trove guest agent and check the guest agent logs and things like that but in most trove production deployments that I've seen ssh access is turned off and everyone sort of either uses the database through the connection string or uses the trove api to manage the data store yes wanted to give you guys a quick demo of backup and restore as well so if we go back to the screen here so we have two instances right now trove and kilo and we can create a backup for that particular instance and the command there is just backup create so if you look at trove help backup create it takes the instance id and the name is parameters and optionally it can take a description and a parent so the description is just any description that makes sense to you to tag that backup with and a parent is in case Amrit showed the incremental backup strategy earlier if there's an incremental backup strategy defined for that data store type and you're trying to create an incremental backup based on a previous backup that's where you'd specified through the parent parameter so I'm going to go ahead and create a backup here trove create backup backup create sorry there you go it says status new so it kicks off the backup so this is actually talking to the guest agent and kicking off the backup so it happened quite quickly here because our database doesn't have very much info in there but what you'd notice happening is you'd notice the instance going into a different state instead of active it goes into the state called backup and if you list your backups by doing a trove backup list you'll see that a new backup shows up in the state called new the backup is already done so it's already in the completed state and so I already had a backup that was pre-created so as we had mentioned earlier what this is doing is talking to the guest agent kicking off the backup strategy and then the files that are created as part of that backup gets streamed to Swift where it then gets stored in Swift so since this is DevStack Swift is on the same box and I can actually look at Swift to see what's there and you'll see there's a container called database backups where we store the backups if you look there you'll notice that there's a bunch of files corresponding to the backups that are streamed and gzipped and optionally encrypted as well so depending on your config file options in Trove you can specify different types of encryption and different keys and things like that so if you'll notice that the actual backup file corresponds to the backup ID that was created so if you look at the two different backup files that we have that one is the pre-created backup and that one is the actual backup in kilo that we created I already talked about the optional parameters that we have so you can create the description using a description you can use incremental backups using parent I wanted to talk a bit about what's going on under the covers so backups in Trove are fully managed they're triggered and tracked through the API as I mentioned they're streamed to Swift which is the object storage and then we do support multiple formats per data store so for MySQL we support extra backup and MySQL dump and as we're going further and further the different types of data stores including NoSQL ones we're adding support for backups in each one of those as well so this is coming to the exciting part stuff that we've been working on in Juno in Juno we added support for replication and so what you can do in Juno Trove is you could say create an instance but set it up as a replica of a different Trove instance so basically under the covers what Trove does there and this is for MySQL so it sets up a MySQL slave instance using the instance that you specify as the master later on if your master goes down you can manually detach that slave and promote it using the Trove update detach replica source command but this is all programmatic in a way and in Kilo we're looking at trying to figure out a way to actually make this so that you could do some sort of achieve auto failover scenarios and things like that but replicas are important not just from failover or HA standpoint but also for read scale out and things like that so they already do exist in Juno so how are we doing on time I can demo replication as well let me take questions though if folks have any questions before that okay good point somebody here speaks from experience on that okay go ahead any questions how about we start up there yep does Trove handle upgrades of minor versions of the database no currently Trove doesn't do any upgrades by itself there's work that we've had discussions of trying to figure out whether Trove should get into upgrading minor versions or upgrading databases by themselves today what folks do usually is publish a different image and then tell folks to do a backup and restore to the other image of a different data store type or different version but there have been some discussions about Trove getting into database upgrades as well but as of today does not do any I had a hand somewhere up here in the middle yep go ahead does Trove have its own what sorry say it again okay does it have its own resource typing so Trove today supports two different modes of operation it supports either creating the instances by calling the native open stack services themselves so if you configure have that option switched on in the config file it will talk to Nova separately it will talk to cinder separately and to provision the instance so there's a Trove task manager piece that actually does this and orchestrates that or you could turn on heat support in which case it just goes through heat we've published templates for the different data store types heat templates and that's configurable as well so Trove uses that to talk to heat to actually get your instance up and running as part of a heat stack so let me kick this off you had a question up here in the front so the backup and restore that we do is based on the volume right so we're actually using backup and restore tools to do the backup and restore so if you look at the backup strategies that we have we could either do a logical backup using MySQL dump or an actual backup using InnoBackupex which is a tool that Percona built that actually at the MySQL level gets the file streams them to Swift and so your actual data on the volume is not actually being touched there's nothing that prevents anyone from building a strategy that's a volume backup that sort of works across data stores but you'd have to think about quiescing the database so that when you take a snapshot of your volume it's consistent right so at the end of the day the backup is only going to be of your data it's not of the metadata on the instance that is an excellent question so if I understand the question correctly it was what's the current list of data stores and versions that Trove supports there's a wiki page that I can provide you guys a link for so it depends on what you actually configure for that particular deployment this deployment has MySQL 55 as the only data store configured because it's on one single VM that's running right so depending on so in order to get another data support another data store version what you'd need to do is create the images for those guest agents and having that particular data store having a way to go fetch that data store in that image and then telling Trove using a register API that this data store corresponds to this image and this version and so Trove is extensible that way so maybe one other thing to add just the names not the versions MySQL including MySQL MariaDB Postgres MongoDB Cassandra, Coach, Redis Postgres that's the let me send Postgres twice that's the complete list of all the data stores which are currently supported each one is supported to a different level for example we know that replication is now supported with MySQL and variants clustering is supported sharding is supported with Mongo but we're working on rationalizing that thing and coming up with the full support you could run Trove in a container for example but if you run Trove in a container some of the capabilities are not going to be available in the container like resize things like that so it's really a combination of where are you going to be running the data store what data store version do you have and what do you actually want to do that's the full matrix that's why there's a wiki page for it so yes there is an API that Trove supports that's the resize volume API so in case your volume is insufficient for whatever reason you can you can issue the resize volume and then it will change the volume size underneath it using cinder so it will call cinder to do that it might be a little long to do it as a demo but we can definitely do it after okay sure great question so today we do not actually integrate with Solometer for monitoring it's something that we've been talking about and there's a design session on it we do have notifications for metering so there's config values for rabbit queues that you can turn on by specifying the name of the rabbit queue so when instances do get created and deleted it will drop messages into that queue specifying that that particular instance got created and deleted for monitoring the guest agent themselves Trove uses heartbeats so the guest agent code actually has depending on which manager for which data store you deploy has logic which checks on the data store, makes sure it's up and running and then sends a heartbeat back to the component called the Trove conductor which says hey this data store on this particular guest instance is up and it's running so when you do a Trove list look at, you see the word active there which means that the data store is up and running if the data store goes down for whatever reason the status will go to shutdown and you could use that for monitoring to build out say things like failover but there's nothing automatically that happens today based on that and we're talking about doing some automation there as well yes, so if you're using heat separately you can use heat to auto scale I specifically haven't looked into that much so I don't have much experience with that go ahead so great question the question was are there any tools to help building the guest images so we use disk image builder which is a triple O tool to actually build the guest images if you go to the trove integration repository which the link was posted earlier and clone that you'll see elements for each of the data store types that we support all you have to do is trove kickstart data store type that will actually build that particular image and register the image in your trove local deployment so that you can then go ahead and test it out so and that will also leave the image around for you so if you want to pick it up and move it somewhere else you can use it if you want to get some other custom images happy to help you with that also to dovetail to the other gentleman's question about monitoring we do have images which have other monitoring things baked on to them so if you want to use those quickly to fast forward since we're running out of time something we added support for in Juno is clusters we only support MongoDB clusters and the call is pretty similar it's cluster create data store type data store version and the actual creation of the clusters creates shards and you can actually we support scaling that out horizontally through an add shards API call action but before we're done I also wanted to just mention what we've completed apart from that in Juno and what we're planning for Kilo so apart from replication clusters we added support for Neutron and Postgres we added support for some enhancements to configuration groups like configuration groups for MongoDB so these are basically the parameters for the data store that you can tune via the API backups for Cassandra and Couchbase and Tempest tests but the interesting pieces that we're looking for help with and where more developers can jump on or design stuff is planned for Kilo we're planning to build out clusters and cluster support including support for Galera clusters, Synchronous MySQL clusters some of that monitoring and failover support for replication associating flavors with data store types so if you're deploying Mongo and you want to make sure your Mongo instances only get attached to these particular flavors, support for that accessing the data store logs via the API so for example you have a MySQL instance and you want to check the slow query log to diagnose why certain queries are taking long time how to do that programmatically via the API and then some housekeeping work with removing deprecated OSLO code and upgrade testing using grenade that's it, that's not a finalized list if you have an idea how to make Trove better please come talk to us work with us on it and we're always looking for feedback and folks to come along and help us out that's pretty much we're growing community and find us on it IRC hashtag on free node thank you all for sitting through this and any questions, more than happy to take them now or even if you want to talk to us after the session is over, come find us and more than happy to talk about Trove if you have specific questions for Nikhil I think he's going to be at the Genius Bar in the HP booth later today sorry, they call it Genius Bar but that said I'm willing to talk about anything other than Trove as well at any point of time buy him a beer and he'll do that buy me two beers be clear about stuff any other questions, go ahead does it work with SQL Server, Oracle and DB2, the answer to two of those is I have them up and running and to the third, not yet the community version currently doesn't have it we're talking about how to do that right now so currently as of Juno Trove, we don't have image elements for anything to build those so we don't support it by far, there's work in progress to support some of that and Amrit's been working on that so he's probably the right person to talk to in that regard yes, so if you do a Trove show and maybe I lost over this a bit earlier it gives you some very basic information about how much your volume is being used and things like that but apart from that Trove doesn't really care about the data plane as much so let me just show you that if I take that instance and do a Trove show 0.13 is the volume 0.13 is the volume used and the volume size is 1 but other than that we don't really care about the data plane so in order to talk to the database you can use MySQL directly and Trove really is more of a provisioning and management tool and doesn't want to get into the data layer at all so if you were using MySQL in this case you can always run MySQL admin 1003 status and you'll get whatever you need and the same kind of command for any other database but not much more through Trove I guess so there have been it's a great question and there has been some discussion around getting some of the MySQL stats through Trove as well sort of like hey what's my replication lag now that we have replicas what's the health of my MySQL database that's not there in Trove today it's sort of minimal but we're still talking about trying to build that out so great question well funny thing you should mention that yes we do we do and if you're installed Trove and you get this pane called database where I created a bunch of database instances in a different tenant that's what the reason why it doesn't show up but there you go those are the databases that we just created up and running you can create backups from here you can do your resizes, restarts and then if you go to backups you can actually see the backups that you have so we do have support we added support recently for replication to Horizon as well and as we go along we're adding support for a lot of the new features that are coming up so we're fully integrated with Horizon go ahead so this is a great question the question is the actual guest agents do they get so it's not really request response because they're getting messages through MQP but is that over a separate management network or is it over the same customer network so in Trove deployments that I've seen on Neutron it's always been on a separate management network where the guest agents are talking to the Trove MQP server just to separate out and the network access and the other question is the other aspect of that is that when you create a Trove instance the customer can also specify a NIC to what other network his instance is attached to and so we really don't control that very much so we do create so I've seen Trove instances actually talk to the rabbit or AMQP servers on a separate management network and that's usually how you deploy it but that said it's really up to the deployer who's deploying the Trove deployment to come up with that architecture and figure out how those pieces are working together exactly it's on a separate management network precisely for that reason it's configurable on a separate management network yes so that's time so folks I know it's lunchtime want to leave please feel free I'm going to be here taking more questions so maybe I will come down and get demyced and we can talk about it a bit more so thank you all for coming hope you had an informative session about Trove


My talk is GRPC, Protobufs, and Go, oh my. So how many of you have already used GRPC and Protobufs?  This might be a review for you. This is kind of like an intro, sort of getting started. And so we're going to go through. I'm John Griffith. There's my Twitter. There's my GitHub. Reach out to me after this if you guys want to chat, if anybody has any questions, or wants to tell me I did something wrong or whatever, or just wants to meet me and talk. I work at Red Hat, and I'm mostly around the Kubernetes space these days. So RPC, basically we're just talking about machines, talking to machines. The whole GRPC thing and everything, it's not necessarily a groundbreaking new idea. This technique and these kind of ideas have been around for a long time. I'm old enough to have worked with most of these, unfortunately. So GRPC is kind of the newest take on this idea. Simple idea is basically you have some clients that you want to be able to actually affect actions on a remote server. Usually that transport that you're going to use is TCP. One of the nice things about GRPC is that it's going to use HTTP2 specifically, which gives us a lot of flexibility and a lot of capabilities and much better performance and efficiency. So my talk really is actually just to tell you that GRPC is really the only option out of all the different things that are out there, and it's the only thing that you should use. So that's really why I'm here. OK, not really. Actually, on the opposite, there's a lot of really good options out there. There's a lot of really good technologies, a lot of different things you can do. But GRPC is actually really cool. It's got a ton of super, super good features, things like that. And so what I always tell people is just please always investigate new things and look at other options out there when you're doing your research. Don't just listen to what somebody said at FOSDOM or anywhere else or what they wrote on their blog. Check it out for yourself and see what you think. So some of the things that I think are really nice about GRPC is you get pretty much a complete solution out of the box with GRPC. So I lump Protobufs into GRPC because they're hidden when you're actually trying to do an implementation or build this. So these Protobufs, it's got some really cool advanced features that we might get into today, hopefully, like streaming. So you can even do bi-directional streaming. So you can have a client in server streaming back and forth open a channel, blocking and unblocking options, flow control, all the sorts of things you would expect. And one of the really cool things is the ability to build in with Protocy a reverse proxy for REST. So if you're hung up on REST and you need a REST compliant API, you can actually still do that. You can actually use things like Protocy to build that. So that's really cool, in my opinion. But the main thing, the reason I love GRPC and I started using it a while back is because it's super easy and I'm lazy. So basically, building a service with GRPC, it distills down to just three steps. So the first thing we're going to do is we're going to create a interface definition. So that's our protofile. And that's going to just be saying what it is that we're going to serialize back and forth across the wire. The second step is to actually implement the server. And then the third step is to implement the client. Now, I did leave one important thing out of here. You have to have an API of something that you're building to do something useful. But that's something you've probably already done. So my argument is that it's so easy that we could actually just build a GRPC service right here and now. And not just the usual ping, pong, anything like that. So what we're going to do today is we're going to take the simple LVM package that I have, the simple LVM API. And we're going to build a GRPC service for it right here on the fly in this session. So it's going to be really interesting because this is not my laptop. And it has a UK keyboard. So you guys should be entertained, if nothing else. All right. So what we're doing is you should always kind of have an idea of what you're trying to do. We're going to basically build our own sand device. So I'm doing a lot of stuff with Kubernetes and containers and all these different things in the storage space. And I don't want to go out and buy a storage box from somebody or something like that because I left those startups and I don't do that anymore. So what I want to do is I want to take LVM.  I actually started to do this as a project for Kubernetes for iSCSI-D driver that's in there and found that all of the code that's there is actually deprecated and obsolete. So I'm rewriting it and this is what I'm doing. So OK. So I've got this package. It's called Go LVM. And we're going to start off really simple. It just creates logical volumes, lists logical volumes, and deletes them. So everybody knows what LVM is, I'm assuming. So all right. So step one, I'm going to walk through the steps again real quick and just give you a preview of what we're going to do. So we're going to define our interface. So that's going to be our proto file. And then we're going to implement a server real quick. And then we're going to implement an actual client. And then we're going to deploy this and run it and check it out. So we'll start off and we'll just make this really simple. We'll just do create volumes. So I'll try not to talk too much while I'm. Well, that's not going to work. Who am I kidding? Yeah, OK. I can talk really loud. So if I can find my mouse or my cursor. All right. So this is going to be good. Let's minimize this and get this over here. All right. So I've got a couple of Fedora boxes running on VMs that I'm running on this machine. So what I'm going to do is I'm going to go on to those. And let me actually mirror this because, well, we'll just try it. OK. So the first thing I need to do is I am in my GoPath. I'm in my directory. So if you look PWD, you can see there's JGriffith. Can you guys see that or should I make it bigger? All right. So we'll do that. So I'm going to set up my directory structure real quick. And the way I'm going to do that is I'm just going to say I'm going to create a directory for my protofiles, a directory for my server, and a directory for my client. OK. Required. Oh, see. And there's the UK keyboard right there. Oh, no. All right. Relax. Don't panic. There we go. We can recover. OK. So let's go into the protofile. The protofile is actually, this is really kind of the nuts and bolts, right? This is the main thing. And what we're going to do inside of here is we're going to create this file, and we're going to call it lvm.proto. And what this is going to do is this is going to define what we want our service to be, right? So let me see here. So we start off, and we just say, OK. So syntax, we're going to use proto3 as our syntax. So this just tells the protocompiler what protocol we're using in this definition, right? So this is what we're looking for. The next thing we need to do is we need to give this a name, right? So we use kind of Go-ish syntax here, and we say package. And we're going to call it lvm. And if I'm typing things or spelling them wrong, holler at me because I can't see. So the first thing we want to define is this thing called a message, right? So a message is the actual thing that we're going to serialize and send over the wire, right? So anything that you want to serialize and send over the wire, it's basically called a message. So we're going to create a message, and we're going to call it a volume, and hit caps lock instead of uppercase. So we're going to call it volume, and we're going to give it a string called name. The next thing you'll see is this one. One is the parameter, so what you do is you order the parameters that you have. So it identifies that parameter inside of the message, and that's what we have there. The next thing we want is our actual create volume. That's not even close. Hey, I could get used to this keyboard. OK, create volume request. So string, we're going to give it to do a create. We're going to need a name, and we're going to need, if you know anything about LVM, you're going to want a volume group. And in our case, we're going to use thin provisioning, so we're going to want a pool. We're going to want to specify a pool. And then we're going to specify a size. Num, num, num, num. All right, OK, so we've got that piece. The next thing we need to do is we need to have a request. So we do a message, create volume request. I'm sorry. Yeah. Holy cow. Off the rails. All right, so we're going to give this another minute, and then if things don't get better and people keep leaving. Not cool, by the way. Then what I'll do is I'll just go ahead and bring up my code, and we can just kind of walk through it instead. OK, so there we have a response we're going to call it is just message. So the last thing we need to do is to find the actual service. So we'll do like that, and then we want to do the same thing. I almost got it. That's not terrible. All right, so here's what we're going to do because we're going to run out of time. What I'm going to do instead is I'm going to pull the code because I already wrote it. And we're going to look at it.  OK, so I'll just use magic. Voila, right? OK, so if you look back, remember our proto file that we were building? Oh, stop that. And he fell apart. So that's the proto file that we were trying to build. Those are the last two lines I was trying to enter there. So the next step that we needed was we needed a server, right? So if you look here, I have a pretty simple server that I put together. It's just standard go. We have a main.go here. What we did is we went ahead and we defined our function for create volume. So you can see there we take a context. All of our gRPC stuff is going to pass a context back and forth. And then there's that create volume request that we had. So what we did with that proto file is we actually ran the proto compiler, the proto c compiler, and we compiled a pb.go file. And that's what I'm referencing there, that pb.create volume request. So that's where that's coming from. So you can see we have the create volume request. We have the create volume response. And then I just go ahead and the server is actually where LVM is installed and everything. So I'm just executing the LVM commands on that server when it gets this command. So that's how it's working. One of the things that I say that's super nice about gRPC and one of the reasons why I like it so much is in terms of setting up the server and get everything wired together, this is basically it, unless you wanted to do some more advanced things, which you can. But this will get you a basic running server and everything will work. So it's pretty simple. All right, so we've got that. And then we would need a client. And the client is even easier. We just put basically the calls into our client. We're gonna just do this directly right now. We're gonna go ahead and wire in our IP address for the server. And we're gonna go ahead and just use this hard-coded create volume request, okay? So if we look here, so if we do a pseudo LVS, we look, so right now you can see we've got the system partitions and we've got that thin pool for LVM thin provisioning and that's it. So what I'm gonna do right now is I'm gonna kick off this server, okay? So now the server, the gRPC service is up and running and listening on this server, right? So we can go over here. I have this other box called FETI 2 and what we're gonna do over there is we're gonna run the client. Oops, and I'm already there, I think. Okay, so this is the same client. It's all the same code and everything else. I just copied the whole thing over here. So what we do is we do a client. We run that. Did I not pseudo that when I started it? So very important. Okay, so we get our message as it was created successfully. So we'll go over here and we'll do, and you can see we have a new LV called test and that's the demo that I was trying to code on the fly. I have a much greater appreciation for just for fun now. I'm sorry, what? Oh, yes. How's that? Is that better? So, about our time, but real quick, anybody have any questions? All right, thank you.


Hey Virgo, this is Kelly from House of Virgo doing your next 24 hours love reading for Virgo's son Moon Rising or Venus. Some of the messages may or may not resonate, just take what fits, leave the rest. If you'd like to get a personal reading for me there's a link in the description box below, just go ahead and click that link to book your reading. You guys can also follow me on Instagram, House of Virgo. All the links and everything you need is below. Okay, so please do hit the like, share, and subscribe. We just hit 5,000 subscribers so that's awesome and I would greatly appreciate it. If you're new, welcome. Okay, we're going to take a look at your person and love. This is the past position. Wow, the Empress is here yesterday. That was in the same exact spot in the present. Okay, the hanged man, justice, the future. We have the eight of wands. You have, okay, judgment, page of pentacles. Okay, very nice. So Scorpio, Leo, I'm sorry, I said Leo. I was about to say Libra, but that's for some reason. I always mix them two up. So we have Scorpio, Libra, twice, Pisces, Scorpio twice. Okay, all right, let's take a look here and see what the love messages are. The overall love messages for Virgo, next 24 hours. What does Virgo need to know in regards to their love lives? Okay, Virgo's messages. Okay, romantic feelings. Your feelings are real and worth exploring. Wow. Okay, you've got somebody here who is definitely going within and exploring how they feel, their romantic feelings for you. Okay, codependency, addictions are affecting romantic life. Okay, so there might be like some healing taking a look to see, is it really love or is it codependency? Okay, is it truly like unconditional love or is this transactional love? Okay, this person might be looking at that. You might be looking at that. You know, is there a need to be with each other or is it just free flowing and it's nice and enjoyable? Okay, trust. The situation is calling for you to have faith. Okay, you also have separation time apart from your partners on the horizon. Maybe there's a trust issue and you guys are in separation at this time and reevaluating the relationship. Let your friends help you ask for and accept support from others. Somebody here might be getting some assistance, counseling, help from friends, family, tower readers. Okay, getting to know each other as you reveal your innermost cells to each other, your bond deepens. So something here going on like Virgo, you might be asking your friends to help you sort your feelings out, your person might be doing that as well or sorting things out with a spiritual mentor or a counselor. There might be a trust issue and there's separation because of that because it's kind of like trying to get the faith back or finding out if you both can trust one another. Okay, so the past position, false starts, stop and start, stop and start, and then it finally came to an end. Okay, there's a lot of defensive energy, a desire to move forward, overcome any issues or like this is the past. So there might have been travel, blockages involved, the death card, like I just feel like something either transformed your person making some changes in their life, or they blocked the connection and they left and moved away. And for some of you, you lost someone due to a passing, okay, in the physical world. And that person is their soul has moved on. Okay, might have been a difficult transition for you to have to deal with. But whatever the situation, you know, if that's the case, that's not your reading here that is just saying, okay, for you, maybe some counseling or some grief counseling or something, and then go from that point forward. But for the rest of you, I just feel like something here has passed or ended or there was a major shift or a change. Okay, there might have been a big change or an ending or somebody this person decided to move on. All right, but let's take a look at this death card. Okay. So somebody moved on from a Libra. Okay. If not, they felt like it was, I don't want to say a knowing situation, but it seemed like what was best for them was to move on and just say, you know, it's, there's, there are a lot of blockages in the past and defensive energy. And then, you know, it feels like every time we get a few steps ahead, then it stalls or we take a few steps back. It's like this constant struggle. Okay, like, if there's an energy in the past of like, we knew we needed to make this transition, your person was trying to make a transition, but then they would stop it, or they would block it, okay, or become defensive. Right? And I feel like your person in the past just said, you know what, what's best for me is that I just move on. Now, maybe they said that to you, or maybe they just left. Okay, because death can be indicative of like, it's just over, it's just final, it's just done. Okay. Or maybe they got sent to jail. Okay, there might have been that, for some of you, the seven ones, why were they blocking blocking because they wanted their freedom? Okay, but then why was the Six of Swords here because that Six of Swords energy is always like relocating, moving, making a necessary transition. Why was the Six of Swords here? Okay, because they truly wanted to give to the connection. They truly wanted to give and to try to make something work, but it felt to them more like a sense of duty, not I have no cup energy here. So this person, I feel like they were unsure about how they felt, or they felt something, but they weren't sure if it was love or not. And that's why I feel like this person is currently exploring that to see, do they truly love you? Or do they just need you? Right? Okay, that may have caused a trust breakdown and the separation.  Now the current energy here is the Empress, the Hangman, and Justice. I feel like your person is like coming to an awakening here about you. Yeah, it feels like they're seeing that actually this relationship was bringing good luck. This relationship was a golden opportunity for them. They're coming to terms with that. It kind of feels like that they're coming to terms with how much you invested yourself, they're seeing how much you invested in the relationship and that Hangman energy. Every single time for me is like enlightenment, they're getting this different perspective on your relationship. And that's what they're sorting out here. Okay, yeah, seeing how positive, what a positive influence and light you were, they're seeing this now in the present, what a light, how, how your light warmed them in a lot of ways. So what's justice here? Now, they want to do right by you. First, it was about them. Okay, because they weren't sure about how they felt. Now, they want to make things right with you in the present. Okay, let's see the two of pentacles is here, bring balance into the situation. Both of these are balance things, they've been weighing things, weighing it out. Do I love Virgo or do I just need Virgo? It seems like it's romantic. It seems like they're coming to terms with they do have deep feelings for you. They do love you very much that to a pentacle energy is like, it's kind of like, making you a priority, but trying to find a way to balance things between you and their life, right, but to bring you into balance, right, to a pentacles like they've been weighing their options here. And I don't mean options as like, Oh, between you and 10 different others, it feels more like two is about partnership, you and them, how can we make this work? How can I bring this into balance with Virgo? How can we like get steady, right? Because like I said, it was off and on off and on in the in the background, and they may have been sticking around because they felt like that's six of pentacles, like, you know, I just I want to keep giving and I want to receive and I want to help and I want to be as generous as I can. But I don't know if I really love Virgo. And I feel like they were confused about that. And then they just decided to end it. Now, that Empress energy, Empress is Venus, that's love, and they're your love is what empowers them. Okay, and I don't really feel like this person understood what that meant. Okay, so going forward, the eight of ones, keep its arrows reconciliation with the judgment, everything coming out a revelation and communication going forward about your self wars and about how much they value you and your connection. Okay, so there will be travel. Look, here's the star. This could be somebody who it's healing energy. This could be somebody in the spotlight or putting a spotlight on this relationship. Now the light starts shining down on both of you. The star energy is renewal hope this person having inner clarity. It's like a miracle coming in. I mean, this person can't even believe that they this is it truly is love. Okay, this is a card about having faith.  Okay, queen of cups energy. Did I say the signs? I did, but I forgot to mention that you also have Leo and Aquarius. And now we have cancer and yeah, cancer here. So I feel like this person is going to open up their feelings. Usually the queen of cups. You know, she's very nurturing and very healing and supportive. But and I feel like your person, that is who they are. But I feel like going forward, they're going to try to bring healing and reconciliation, but also to open up and reveal their deepest feelings to you. I feel like that's just going to be a continuation that keeps going on here. Why speech pentacles here. Okay, page of cups. Yeah, love and instability here. That's their communication going forward. Okay. Is their marriage is their engagement? No, not yet. But there is, there could be a legal thing between you that binds you together like a binding contract that could be marriage. It could just be communication about that. But I feel like right now, it's all about growing closer. It's all about exploring the emotions and to see if this can move forward. Now, I know that may sound different than yesterday's reading, it may not be connected. This might be something completely different. But let's take a look and see how your person's feelings for you. What they would say to you when opening up with this judgment card, their expressions. I know that I crossed the line with you. Okay. I remember every detail of our day together. Let the last time you were together. I'm starting to understand our connection. Okay, I don't feel like this person was fear based. I don't unless the death card could represent that fear. I don't feel like they truly wanted to end something, but they couldn't. It feels like in the past, they just couldn't be in that love energy. You know, the timing just wasn't right for us. That's really what the issue was. I wish I could take back my words or the things that they might have said to you that maybe you didn't like, or even the words of, well, I can't, it's not the right time for me. Or even the words of like, I don't really want to be in a relationship or something like that. Or I don't love you as much as I thought or I'm not sure they wish you could take that back because it feels to me like this person made a mistake in their mind, not in your mind, but like, they're like, wow, I came to realize that I really do love her ago. See, I love you. So this person definitely dealt with some kind of an addiction here. Okay, some urges that got in the way of the relationship. Okay, I wish I could share my good news with you. Okay, well, they will going forward right now, they feel that way, but going forward, that's what the Eight of Wands is that that's always about good news coming in positive news. And I do feel like they're going to share it with you. You speak to me through music. This card always shows up. There might be some kind of music connected to the two of you. This person says you are the best thing in my life. You are Virgo. I don't feel like that's ever changed.  And I feel like that's why they may feel like it's over. You feel like it's over. And you're both in separation. And I feel like you're both trying to discover how you truly feel. Was it was an addiction? Was it codependency? For them, they are seeing that that might have played a role in them not being able to move the relationship forward. They needed you, Virgo, more than you needed them to satisfy this. You know, codependency is a form of addiction. And it's a form of control. I want to feel that way again. New love or a recommitment to love. They want that how it was in the beginning when things were really good, Virgo. There's new development of love happening here. And I feel like this person really wants to try to get to know you again, have like a restart, it feels like this person admits that they took the easy way out. Yeah, when things got tough, they ran. I understand you better. And I do feel like they're looking at, you know, the role that you played in their life. Whereas before maybe they just didn't see absence. It's the absence which makes the heart grow fonder. I can still see you standing there. Yeah, there's like a shadow.   And I know some of you're like, well, that's not good enough, they need to connect and you know, like, but they're saying I am different now this person's making huge changes and what they come forward with might be a little shocking to you. But the message here is you didn't imagine this, you really did feel their love and they really did love you and they still do. We are more alike than I admit. Now, some of you Virgos, believe it or not, you might be condemning this person for doing the same things that you did, right? Because you might have been reacting to them in the same way they were treating you. Okay. So if this person was hot and cold, you were acting hot and cold. If this person was giving you mixed messages, you were giving mixed messages. If this person was being avoidant, then you were being avoidant, right? By going into no contact or vice versa. If they were giving you the silent treatment, you might have been giving them the silent treatment or just being silent, both of you, like it doesn't mean it was intentional. For some of you, it could have been. But I feel like you both were mirroring each other instead of just one of you taking the reins and saying, you know what, you connect that way all you like, but I love you and I'm going to keep pressing forward with this relationship. Now, you might have had your good reasons for not doing that. There may have been fear at the root of it. Okay, fear of rejection, fear of being vulnerable and being rejected. Okay, this person has really been struggling without you. And there you are struggling trying to sort out how they truly feel. I feel like there was a breakdown of trust. I do because of this flip flopping and this connection.   I feel like they want you to be happy, which is very nice. It's good. But they do hope they will see you again. And that is what this person wants is to connect with you again. Even though I feel like Virgo, because there's a breakdown of trust, you're separating from them at this time. And that is causing them or influencing them to really explore how they feel, right? And I feel like that might have been a necessary thing to come.  But they don't know how to let you go. They don't want to let you go, Virgo. Even if you're like, Nope, you can't do it anymore. I don't feel like they just don't know what to say right now. They don't know how to approach. They may have fear. Because they left, they may have fear that you will reject them or you won't take them back. Okay. And so there's an element. Sometimes I hear your voice in my head. Okay, I don't know how that is. But they hear your voice. Maybe they listen to your voice mails. Still, are they just hear you somehow, maybe through music?  But there was a fear, total fear of you rejecting them. So Virgo, this person is coming in. I do feel like there's healing here. And it's just going to take them a little while they are waiting for, you know, they are having an awakening here. And I do feel like they do want to try to make things right with you going forward. But once, once I feel like once they get clear, and it's out in the open for them, then I feel like going forward, there will be messages of healing and communication. And it'll be like about change and positive news coming forward. And, you know, that they want to heal this connection between you. So you just keep doing what you need to do Virgo to heal and make your life better. And, you know, keep it moving. Alright, Virgo, that's your message. I hope you enjoy that. I will see you guys very soon. Take care.