So this talk is really like, it's more, I would like to share or use an issue that I have currently with some tools integration. And I would like to have feedback on that. If it's just my issue or is it something, or do we have a way to fix that? Because currently it's like a lot of work we are doing. And yeah, I would like really to have feedback there. So the thing started from, so some of you probably know, most of you know that I'm working on microbiome a lot lately and especially on metagenomic assembly. So the idea is when we do metagenomic assembly, we take a sample of, I don't know, soil, gut, whatever, where there is some microorganisms inside. So we have the microorganisms in there. There is a sequencing step. So where we sequence the DNA from the organisms. So you can think about the microorganisms. You can see the different colors in this image. I'm sorry if it's not really easy to read. I drew it quite quick. So I hope it's still readable. And so you sequence and you get these small pieces of DNA from different organisms, but we still don't have no idea to which organism that belongs. So one steps we can do is either we try to classify these sequences to identify the paxon. Another things that is happening more and more is to do assembly. So we do use specific tool that are called metagenomic assemblies tool that try to come up with longer sequences by combining these short sequences into longer into context. And when we do a metagenomic assembly, the next step that happens is usually binding these longer sequences because of this context still doesn't represent the full genome of an organisms. So we need to somehow make groups of these context to group them in a group that corresponds somehow to a species, for example. So we bind them and we try to cluster them in groups or bins. And this is called metagenomic ensemble genomes. And for doing this metagenomic assembly, there is two possible approaches that we can do. It's usually you can, for example, if you want to learn more about the microorganisms and the sequences of microorganisms in a soil sample, you just don't take one sample. You take several samples, maybe at different timing to really have a representation of a good representation of the microbiome at this time and the different microorganisms that are in this soil. And then you can compare to another soil, for example, in another location. And so you do a lot of sampling for that to get the data. And when you do the assembly and especially in metagenomic assembly, you can use two approaches. You can use what is called co-assembly where you take all the samples, all the reads that you got from all the samples, you assemble them in run-run. So you run that you merge somehow all the reads together before running the assembly to get the context from all the samples in one box or in one outputs. But there is another approach that is becoming more and more used also. It's called individual assembly where you assemble in parallel the different samples. And then you process them in parallel. You do the binding in parallel. You do different things in parallel. And at the end, when you do the clustering and you clean the bins that you got, then you combine the different samples together to identify really the metagenomic assembly genomes. So that is the two approaches. The only thing that we realized by trying to do that on Galaxy is that we have several issues then. For example, when you have multiple inputs, so these multiple samples as inputs, we need to explicitly add these two modes. So when you have a collections, by default, only the co-assembly approaches is implemented. So for example, when you look at metaspates that is used, by default, whatever the type of data that you have here, you see that you have always the four loops on all the input that you have. So if you have a collections, for example, of data, by default, all the wrappers for metaspates, in the wrappers for metaspates for Galaxy, then by default, there is a four loop that go through all the samples and add them to the same command line. So by default, metaspates is one, only ones. So we need to add another, we have to explicitly add, okay, to which type of modes do you want to use? Do you want to use a co-assembly mode or individual modes to define, to put that directly, to put that in the common sections of the wrappers, to say, if you use the co-assembly, yes, you can do a four loops on that and making only one command for that, for all the samples together. If you do an individual assembly, then we need, you don't do a four loops, and by default, several jobs will be run for each of the samples in the inputs there. So it's a bit, yeah, the issue that we add, so we have to do that, and we have even tricker cases, for example. So in this case, we add quality control. So we have an assembly where we do individual assembly, then we have the context on the second rows, and then we do a quality control where we compare the context to the inputs samples. So how it should work when we do individual assembly, we should be able to run the quality control on each of the samples individually, and that there is a match between, even if we give two collections, where the inputs, the samples inputs are, the raw data are first collection, the second collections, we should there, and then the QC is run for several times, so the number of samples we have, and that match every sample and each context to run the quality control. But with one truth, with Quast, I was investigating the implementation, and I found out something that was a bit more trickier, freaky, so it's like, if we have a paired collection as inputs for the samples, the collection that is given by the assembly is not a paired collection, it's a simple collection because we have only one output from the assembly, one Contic file, even if we have a paired collection as inputs, so when we are forward and reverse, then the output of the assembly is one Contic file, so it's good. But then when we do the quality control and we give the samples paired collection as input, and the Contics collection as the other inputs for the quality control, there are a mix of for loops and non-for loops on some aspects, so that makes the thing that there are, yes, several jobs, so the number of samples of jobs that will run, but the commands that, and how the commands looks like, it's all the Contics were put every time as inputs for the collections, for the quality control, and so the sample one, we're compared to the Contics from all Contics outputs from all the samples, so the raw reads from sample ones were compared to all Contics from all the samples, same for sample two, for sample, blah, blah, blah, et cetera, and when we do with this quality control, what we do is we do a mapping, so there is a mapping of the reads, the raw reads on the Contics, and so it's mean that in this case is the mapping was there is mapped mapping of each of the reads on each of the Contics every time, so it's a lot of computational resources that were lost when we do that because of the way we had to, the way it was defined in the command sections of the wrappers, and another case that I figure, found out also in this also tool is when we have a forward and a reverse collections, there were a mix of everything we're concatenated in ones to really give only one output for, so not one report for each of the samples, but just everything aggregating in one report where everything we're compared to, so it was a bit of mix. So what I wanted to share here is just currently, it can be quite tricky to identify the case when it's happened, so these cases of reality is different than what we expected, and we need then to modify all the XML of a lot of tools to add this assembly mode to say, okay. So we need to, sorry, I'm not sure if it was a question or not, but yeah, okay. So we need to modify the XMLs of these tools to add this assembly mode, except if there is another solution that I would be really love to hear, and then we need to remove the for loops in the individual assembly mode in V6ML, and there is still a lot of tools, so I already started, we already started to do that on several tools, but there is still a lot of tool that doesn't provide that, and I know that we have that some collaborators that try to run that and got some issues, and for us, we also would like to implement this Metadromic Assemble Genome Building Workflows, and we need to cover this aspect of individual assembly. So, and then- Can I ask you a question? Yes, sure. So just to understand the thing, so you basically have a list, and your list gets merged into a single dataset because it iterates over all the items. Is that what's happening? Yes, yes. Okay, so there is the applied rules tool that takes your collection and places each dataset into its own collection with a single item. Would that work? But it's meaning you add another steps, do you mean in the XML wrappers, or do you need to add another- Or in the workflow? Yeah, but the idea is why not, I'm not sure if it will solve, so it's not, but then the users need to know that they need to do that before, so which is not- So your users are running tools? Are we needing what? Your users are running tools, not workflows? So this case is a lot, so it's tools that we already have, so mega-eat, metaspate, squast, check-in, cover-m. So there is a lot of other tools in these cases, more and more also, where, yeah, I'm still not completely sure how to express that, to be honest. So it's just a way, I just wanted to get feedback on that also there. So we have the way that we have collections in Galaxy and every time you have a collection here, in this case, you do a for loops over the item, so like in metaspate, the way it's implementing the command sections of metaspate is this way. So by default, in these cases, here you see that all the items of the collections are put in the same command. Do you get that? And you just want to have it operate on a single file of the collection? Yes. So that's the solution then, and like, you know, if people are not comfortable with the apply-rules tool, which is super generic, but maybe a bit hard to understand, I mean, it's pretty simple to do another tool that just reshapes your collection. I mean, it doesn't consume extra data, it doesn't consume extra compute. It's just, yeah. But does that work? I'm a matter of being familiar with Galaxy. I'm just not sure if it will fit to old cases. So when you have these cases like here, these things, will that work? Well, I don't understand the graph, so you have to walk me through it. Sorry. So on the top, so you have the raw data on the top. You do the assembly, you get an input which is the context. So you have another collection. So you have a collection on the top, a collection on the outputs of assembly. Then you want to compare the output of the assembly to the inputs of that you add to the assembly inputs. It's the quality control step here. And you want that to be run one by one. So you want to have the S1 compared to S1. S1 context compared to S1 raw data. And currently in some of the tools, that is not the case. It's not what is happening because of these small loops. So that's exactly what my suggestion is about. So when you have the for loop and you iterate over a simple list, you have to have a list. So what do you do if your list shouldn't be merged? You make a list of lists where the inner part is just one item. I mean, we do this a lot. So I mean, I'm a little sad that you say you spend a lot of time adapting the XML, which is good. I mean, it's certainly something a lot of tools have. They have a pooled mode and a single mode where pooled means, you know, you iterate over the entire input. But is it some, yeah, okay. So then I didn't know about that. We already, we discussed a bit with Bjorn. And I think the only solution we had at the time was putting these two modes. So I didn't know about that. But Bjorn is not the Galaxy community. So I think that's a great question to ask in one of the channels. This, I mean, I've been giving this answer a few times. And I mean, of course, it's interesting to know that this is still a problem. The question is then how to communicate about this. Yeah. Yeah, that would be good to know, because maybe I need to try that first, also to see if it's fitting really to the use cases here or not. But yeah, that was mostly what I wanted mostly to discuss with the community there. It's also because I didn't figure out, I didn't find a solution there. And I had to rework on the tool. So it's good to have it to know if there is a solution for that. And that would be really helpful there, definitely. So. Marius, is there like a tutorial you would recommend? Or I mean, I guess it's kind of hard to look at the tool XML and know if Galaxy should sort of say, hey, do you want to build a list of lists here? Is there a good example tool XML that has a help section that would explain that, that we could sort of say for tools that have this pooled versus single mode, you know, look at this example tool when you're sort of writing the wrappers.  I don't think I would adjust the tool XML because if it's much more generic and simpler to write a light tool that does the for loop and then just have Galaxy pick out the right structure. I mean, the good thing about the other mode is that you would do it, like if you only have a single file, it's kind of easier. Galaxy doesn't offer the ability to sort of auto promote a file to a singleton list. But then you do need to have those two independent command and sections independent. Like, I mean, they're in the if statement, but like kind of disconnected. Yeah, which I mean, I guess I'm fine with. I guess more, my question is, I mean, that's fair. I mean, I think that's a perspective and it's, I trust you use tools and I don't. I guess maybe the question is about like, examples that we could like point people at for how to write tools with like a help section like, I don't know. I mean, you guys know how to write the tools, right? Cause I think like the planimo docs go into depth on this, right? No, but the tools don't have, yeah, there's that.  So I understand that. But I mean, like the help section should have maybe a description of like, okay, if I have a single file, how do I use this? If I want to break it into batches, how do I do that? Like maybe just pointing that up at the applied rules tutorial or something. I guess we do have that tutorial, but it's not structured in this way. So if I was going to improve tutorials, I would put a couple of like commonly used rules that you may want to use in the applied rules tool. Okay. Which isn't a direct link to, you know, the tool form. Yeah, I mean, it feels like there's some disconnect there between, I mean, we do say in the tool form that we are, I mean, we say something about this. I mean, it is clear that you're going to reduce the data set on the tool form. It's just you got to know what that means. And like, if you don't want to reduce it, how to get around that, right? Like I don't think that's there. Like, this is going to reduce it all down. If you would want to consume it in batches, here's, you know, maybe that's a little link we could put in the tool form. It feels very specific, but anytime you're reducing a collection completely, sort of add the option to have a little help hint that like, if you don't want to reduce the collection completely, here's how to restructure your data potentially. Yeah. I mean, I wonder how we can do this. So it is not confusing. Because I mean, it's not that common, I guess. You know, when you build a workflow with 50 tools, then not common becomes common. But like for the average user, they probably don't have to worry about this. I guess that's part of the problem here, that those tools have this as a very common problem. So the tool on its own, even if you don't use it in workflows. I mean, I'm not sure it's a problem really. I mean, like, I mean, you know it's a problem because there's an entire meta genomics and communities that struggle with that. I mean, I would say the question is, why did that? I mean, I don't know. I mean, why did nobody ask the question? Or like, I do answer that question like three or four times a year. Well, Bjorn, maybe to flip it back on you then, like we've, I think Marius and I have put a lot of effort into the apply rules tool and the training around that. I mean, you know, I never write documentation but I understood this was complex. So I actually wrote documentation for a change. Like, is there, what do you think the missing piece of documentation is? I think maybe Marius's point is that this is not a problem with the tool. It's a problem with how information is flowing through the ecosystem. Is there, do you have an idea for how to improve that? Not really because it's a, I mean, we have two kind of different forums, I guess. We have the developer, I mean the tool developer problem, right? I mean, I guess this we can improve with documentation maybe part one of these tools and reference that then as a best practice, whatever implementation of this kind of use case. But then we have also the end user problem that consuming those tools and needs to use it. So these are two different kind of problems, I think. And for the last one, I don't know. I mean, we need to link the rules tools more prominently in the tool form then, I guess, I don't know. And probably also have more training where this is used really because currently I didn't, I think I didn't realize that could be used. So it's been that there is probably no tutorials or almost not that many tutorials that use that but it's not the one I checked or I worked on. So we need to add more to that, show application of that maybe. Yeah, I mean, I think that's a fair point. It's on the eternal to do list. It should get done, I agree. I don't know. We could certainly also offer the option for that thing. So if you have a tool that consumes multiples of a data side, we could maybe add a hook where we just reshape the connection in a way that things do not get reduced. But I worry about the, I mean, it's easier than opening up the apply rules tool but I worry that it's not, I mean, I worry about the extra button and you still got to understand what's happening there. So I have this open PR that I started that's like, it's all, I was trying to make a little modal or something to have all the different collection operations with like pretty pictures and maybe just like a higher level view. And one could easily imagine like, you know, one button use the apply rules tool to, you know, been single-tonized the list into a list of lists like, but then, you know, we probably wouldn't want to put that modal on every tool form, but maybe the tool XML could have a hook. Like if this is the tool like an assembly tool where we expect this is a common operation, we could have a little flag that would do that. I mean, so let me restate that. I mean, this isn't an assembly problem in any shape, way or form. Like, I don't know how many projects are doing this. And I think we don't know that in advance. We don't know how our users are going to compose what they're doing. And so I don't think that it makes a lot of sense to include this in the tool XML. I mean, I think if something consumes something else than a simple data input, it should be an option on the row with buttons that switch between collection and multiple inputs. And you would be okay with that button? I mean, I would find it handy, but... Okay. For that, I think we probably need to explore how people understand this, if they understand this.  We can say this tool consumes a list of data set and produces a list of data sets. You know, that sort of simple thing. I mean, we don't even showing what outputs tool has, right? But in order to make that right choice, I think the first step would be actually to show what outputs the tool is producing. That's currently not visible at all in the tool form. And then I guess the next step would be to say, okay, we're taking your list and we're creating a single output. At which point the user will say, oh, but that's not what I want. I want the list and I want to list out. And there is the logical link where you would put such an option in, right? Well, yeah, that's a great point. So maybe before the tool success page at the bottom of the tool form, we describe the outputs and even maybe just having that there would sort of catch people a lot quicker and then we could sort of go from there. I think so. And I think it also ties in with the common be requested feature of being able to name your outputs in the tool form. Yeah, that seems like a pretty natural, good place to go. Or could there be like a tab in the form where there's one tab where you can kind of put in all the inputs deal with all the parameters of the tool and then maybe have a tab where you can deal with the outputs, see what they will be and then possibly name them if that's of interest. We have a great UX working group that I think could explore those different options. But yeah, I think there's a bunch of different ways you could do it. Ernest, does that sound like it all kind of helped you get towards the names for your tools? Yeah. So yes, I will try, I will have a look definitely at the app libraries before going further. And if I, if I still doesn't solve my issue, maybe I mean, I couldn't, I check with you, Marius, we can discuss again, I can try to explain again and see different use cases where it really doesn't solve the issue. That sounds good. I mean, I guess my other question is like, naturally, where would you ask a question like this? And sort of what should be the place where people ask this question? Because I think like the most knowledge in this regard is in the IOC. But it's a difficult for me, I had difficulty to formulate the problems, to be honest. So already identify how to formulate that. It's also making this presentation is a way for me to be sure that also I understood the problems. But then how do you, for users or for someone that cannot, doesn't want to give a talk or something like this, how do you ask them to formulate the chart is difficult, I figure, I found. So already we were, I was trying to explain to Christo that chart about that issue and it was really not easy. Yeah, so I can't say I understood the chart, but I mean, it's a problem I've heard multiple times. I think if you're talking to an audience of Galaxy people, maybe a link to a history and saying, here I ran this tool on that import data set, I got that output and that's not what I want, what do I need to do? Would be something we can easily answer. I mean, I guess ideally in an ideal world, this question would go to help.galaxyproject.org because there it's discoverable. It's just, yeah, a lot of questions there and it's a bit intimidating. I think that part of the issue is that we don't have a visual language for collections. And so no, like if people saw a chart, like this is how this tool is gonna consume it and this is how they're gonna produce it and they could see like boxes and lines connecting them. I think that people would be a lot easier. I think maybe Bernice's whole presentation would be like, the boxes look like this and they need to look like this. And we want, people wouldn't have to understand like, there's like 10 terms that we use like reduction and consuming and list of like singletons and like we've used a bunch of very jargony computer science terminology here and we want our users to know all of that but I don't think it's necessarily realistic and it's a very visual application already. So just like, I don't know, it's something. I mean, at one point Anton had some great little charts of the, what the collection tools were doing but if we could put more of that in the UI, I think it would really help. I think even like the workflow editor, people can talk, I think it's a little bit more communicable because you can see ribbons or not and sure even that could be vastly improved by like, a lot of information is lost in those ribbons but just ribbon or not, it's already a start. I mean, in a sense, the best way we had to go about this, we've actually removed with a multi-history view because we had like these boxes where you could see your structure off the connection but I guess there's of course more involved in sort of showing the transition between your input and your output. And presumably we could build a diagram for each tool run, right? Like we've got, there are gonna be a couple of tools where we can't, but for the most part we can and yeah, I think that might be like, step one is just how do we display the outputs? And then step two is, well, we could add the naming and that should be a pretty short project. Maybe step three though is in like, how can we communicate that visually?  What do I do?  I guess my question was mostly what avenue like, I think maybe for users it's not entirely clear where they can even ask that question. It's like, I mean, if they want to get an answer from me, I mean, I obviously should be looking at help but realistically I only do if somebody points me at it. I am however present on the IOC channel. Is that a channel we would want to point more people at? I, as a newer member, I would just be curious or, you know, if Bernice wants to put together or work on a training video together with me for this case in the short-term, that might be a nice short-term answer while the long-term response on the functionality and the feature set gets flushed out a little more if there's anyone else who's interested or thinks the training video using the pride rules just to see if that will help solve any of the issue that Bernice is experiencing, you know, let me know. I mean, definitely working on trainings would be good anyway. So having more training for that and more use cases that would be really useful and to show that how to solve that issues. But anyway, in the max, so metatomic assemble, workflow, genome, yeah, whatever, workflow, I would put that in the tutorials when we write tutorials, but so a short, yeah, would be good, yeah. If you want to work on that, I'd be happy to help you. Great. Thank you so much, Bernice, for sharing the issue that you run into and hopefully we can kind of get you towards an answer. I think maybe we can put together like a blog post or something like that that can break down for folks where they can plug into the community if they kind of get stuck. And maybe this kind of training video will already help to address it, but I think also maybe like a blog post of this kind of story of how you got stuck here and the solution and maybe like pathways for other users so that this can be a little bit more discoverable. Might be also a good idea, but the tutorial may take care of that, so that might not be good. I just want to point out also for the new people that we've had videos in the beginning and there's a reason we don't do them anymore because they can't be updated easily. And that's why we went with tutorials and it's so much easier to find tutorials than videos. That's a good point. It's hard to scale some of this like transfer of knowledge where videos take a lot of time to produce and can be extremely helpful, but as soon as something changes it might not work for you anymore.  If there's nothing written I don't know, I will use something else. Personal opinion, but I mean, I think it's probably quite common that business scientists will not take half an hour even five minutes to watch a video. Yeah, that's fair. We could add more FAQs for example in the GTN for that to solve that so that where explain, okay, your ear is your cases, ear are in different solutions for the different cases. And then you can point to the FAQs because they have FAQs without each one page. So then it's easy to, but then we probably need to identify the different cases and different solution for that. Yeah, I mean, I think these FAQs are fantastic. We need more of these because they are to the point. With Andrew, we use like an automatic rendering of Google Slides to video. So that might be something we could explore here where the slide notes are kind of red, kind of with the robot voice, but that could be still helpful and then a little bit more updatable in a manageable way. It's already implemented in the GTN. So when we have slide, we have automatic videos generation. Yes. Okay, that's great then. I'm seeing some comments from Dan in the chat and mentioning that it's okay for people to end up in the lobby anytime and that way they can ask their question, get an answer there or get pointed to the right channel that they need to be in. And it's okay to ask any kind of question even one that's not fully formed is a little bit hard to communicate just as kind of a place to start the communication. And then... I wonder if we can... I'm sorry to interrupt, but I wonder if we could use one of those texts generating AIs to steer people in the right direction? I don't know if that's too spammy. It sounds like it'll be fun. I mean, they generally even generate good-ish or workable templates for tools. So I wonder how usage questions would come out and if we have the funds to pay it. I think that'd be a cool idea to explore.  Oh, I was gonna say that Hopkins is having like a meeting about chat GPT or like a town hall or something. So I don't know how... I don't know what we'll be able to use, but it's an interesting question for sure. But from a policy standpoint, like I don't know what they're gonna do or say about how academics can or can't choose that. It was like a whole new world. And then I'll also just read out the last comment that Dan made is that we kind of have two problems. One is answering questions and that people will have by providing documentation and materials. So I guess being a little bit proactive and then also encouraging folks to kind of just ask if it doesn't exist or they can't find it. And I think sometimes as someone's learning, you don't know if it's something that you're missing or and just can't find or if it doesn't exist, which is totally a possibility. So I think definitely this is a good... I think this meeting has been helpful for us to kind of bring some of those up and we can kind of continue to brainstorm how we can encourage folks to ask those questions. Awesome. I hope this has been a helpful discussion for you, very nice and for you all joining today. Any other questions or comments? I had a quick comment. I apologize if I came off a little defensive in this meeting, but I really, and that's on me, but I do really love this format of like, we've got a specific problem, here's a presentation. I think this is really helpful. So yeah, so I just, I'd really like to encourage us more and I apologize if my tone or comments discouraged it because I think it's really great. So I just wanted to throw that in. Thanks so much, Bernice. Completely agree. Thanks for, I mean, I wanted, I feel a bit like, stupid, I didn't know the apply, I mean, I didn't check the apply rules on that and I didn't try to see if it works or not. But on the other hand, I mean, it's helpful to have the discussion to also see if it's, I mean, if I didn't figure out about the apply rules, I mean, there is probably a lot of users that didn't figure out about the apply rules. So how can we solve these big issues of finding that and making that more obvious? So at least my feeling there is good to help also building better Galaxy for that. So and better interface or better materials for that. So all good. Thanks. Thanks so much. Thanks everybody. With that, we'll pose our community call today. The next meeting that we'll have is on April 13th and we'll get an update about Anvil. See you all in about a month and many of you and Baltimore hopefully. Bye. Bye. Thanks everyone.


Hey guys, what is up? Welcome to my channel. For today's video, I am going to be reviewing the Tom Ford Solanage Naked pink palette that just launched for the holidays. I'm also going to be doing some comparisons of some other similar Tom Ford palettes. So if you want to see that, then just keep watching. This is quite an impromptu review. I wasn't planning on actually really ever filming this. I bought it with intentions of filming a review, but then time got past me. And at this point in YouTube Bakeup World, this is no longer a new launch. However, I was watching my friend Tara Lynn's review on this palette. And if you saw that review, it was, um, it was less than positive. And so I was like, oh my gosh, I need to play with this palette. I ordered it. I spent a lot of money on it. So I figured since Tara Lynn inspired me to play with this, I might as well do it on camera. So here we are with the review. It's going to be a little bit more relaxed than normal since just kind of putting on the camera and sharing my thoughts with you. So this is Tom Ford's holiday palette for 2021. And I have to admit it's a little underwhelming. Normally the Solanage launches are more exciting. I've loved the last two years launches of the Solanage palette. Two years ago, I believe he launched three or four quads. I loved all of them. And then last year he launched First Frost, which was just one, very minimal. But it was a beautiful quad. It ended up being one of my favorite Tom Ford quads. So I figured might as well keep this tradition going because I'm picky about what I pick up from Tom Ford at full price. I'm a cosmetic company at store Stangan. And if you don't know what that is, it's basically, I believe it's the Estee Lauder brands. They're overstock and discontinued items go to that store and you can get them at a great discount. And a lot of times, I don't know what Tom Ford is doing, but a lot of times you can find some newer Tom Ford stuff at that store. That's where I get a lot of my Tom Ford stuff. I like to buy it on sale. But I don't know. I just saw it launch at Beautylish. I got a notification and I was like, I'm going to order that. I like the Solanage palette. So I got it. This is $89. And like I said, the Solanage is his holiday collections every year. And this also launched with one of the lip balms, which is like almost $60.  I'm not willing to buy a lip balm for $60. I'm still quite weak. Don't congratulate me for that. I still bought a $90 palette, but I'm not at the lip balm phase yet. Nonetheless, let's get into the review.  As always, if you aren't shopping at Beautylish, you should start. They have really great fast shipping and their stuff comes packaged wonderfully. Amazing customer service. Definitely one of my favorite spots to pick up makeup. I get very excited when they are the first to get a product because I love purchasing from them. So here is what the box is going to look like. It is all white. And then here is the back of the box. Turn the light down a couple of notches if you want to pause and take a look here at the ingredients or any of the extra info. Lighting is changing. I'm filming at a weird time of day. So the eyeshadow quad itself is going to come in the white slip cover, which I'm keeping these now. I used to like want to throw these away, but since I'm going to be moving, I just dropped the palette. Okay. Since I'm going to be moving in a few months, I think this is going to help a little bit in transit, one will hope. But the packaging is the same, but it's a little different this year. I mean, I don't have a problem with the Tom Ford eyeshadow quad packaging. It's very elegant and I just love the way that my collection of quads look. So I'm happy he launches his quads of all the same size packaging because they just look so uniform and beautiful in my drawer. And I love how the Tom Ford label here at the bottom is always up. So what's different this year is it is silver in previous years. I believe it's been gold.  It's just been flame. So a little nice extra touch, something different. So this palette is a made in Italy. And like I said, this is the Soleil Nage collection. It's a very small collection of two pieces. Were there more? Were there supposed to be more? Anyways, so let's take a look on the inside. You do have a mirror and here are what our four shades are going to look like. Please note that if you order this, this is not the wet to dry formula that previously isn't Soleil Nage quads have been, which is why I'm surprised. So if you aren't familiar with the wet dry formula, if you look at any of the quads, it's going to look like this. All four shades are going to be shiny. They're going to look kind of like a baked gelet formula versus the regular eyeshadow formula, which just doesn't look the same. And the wet dry formula also will always have the Tom Ford embossments on the inside. It's by far his most popular formula amongst at least the people on YouTube that I watch and based on the comments that you guys give me, you all love the wet dry formula. And I will say you guys, I love the wet dry formula, but I also really like the regular formula. So I wasn't mad that this was the regular formula, if you will. We'll just see how it works. I don't know. Let's see. And by the way, this concealer, I'm wearing the Hourglass Vanish concealer. It's not doing me favors. So we need the eyeshadow to fix that. So let me get these annoying things out here, but I can never actually throw these away. I don't have the willpower to throw them away because you never know when you're going to need them. Here are the four shades. So let's get, this shade feels very smooth. Okay. So we have that shimmery shade, which is less than exciting. And then we have kind of like a bone colored skin tone matte shade, maybe slightly satin. The Charlotte Tilbury kind of pop style formula, that looks great. Then this shade looks satin. Yeah. Beautiful. It's a little underwhelming. Now I will say with Tom Ford Quads, I usually, I'm not too excited by the looks of them or the swatches of them, but when I apply them is when I really like them. So I'm not disappointed yet, but his eyeshadows do normally swatch better than this. I will say that. So I pressed just as hard as I normally do with the eyeshadows. I mean, I don't swatch too hard to swatch. Remember folks, I paid $90 for this. Okay. Let's see how they look on the eyes though. I'm not going to judge too soon, but the swatches depth seemed quite soft. So I'm using the hourglass concealer that I was just saying negative things about because it doesn't look good on my under eyes. We're going to use that as our eyeshadow base because it's the closest thing to me. So we'll start off with this shade right here. This powder, I don't know if you can see that, but I am getting some kick up. So for me and my skin tone, this is about a shade darker. So for me, this is going to work as a subtle crease color. I know on Tara who has medium skim and she has a very olive undertone. This was kind of like nothing on her. I personally like shades that are my exact skin tone. I love the canvas that they create. Anyways, if you have like a light medium skin tone like myself, this gives a nice shadow. I like that. I think that looks good. Let me put this on my lower lash line so I can blend out this crusty dusty concealer. You know, I've tried to make this hourglass concealer work and I just can't get it to work. And so many people love it. I don't know what I'm missing. So I don't necessarily know that this is meant to be a deepening shade, but I'm going to use the same brush and I'm really packing it on here. I mean, this is more of like a highlighty shade. I guess it wouldn't really be for depth. So I'm just going to put it in the outer half of my lid. I will say normally with Tom Ford palettes, I'm impressed with how they apply and I'm impressed with the overall look. Tara was right. This palette is kind of useless. It's like not even given me something. And I wanted to do this review because I like his normal eye shadow formula, but this would be much more valuable in the wet, dry formula because they have more intensity and they have more glow. So there's a bigger purpose on the eyelid. I'm not even talking about the lack of depth, which there is a huge lack of depth. But with the wet, dry formula, I feel like I don't even need depth with their shadows because the glow is so pretty. This really has given me nothing. So that's with a lot of packing on of this shade. Let me use this shade and I'm using my fingers. Hear me out. Like you could get a pretty soft look with this palette. Very, very soft with this in the crease and this all over the lid. But I could not exist as a beauty reviewer and live with myself if I told you that this was worth $90. Okay, let's use the pop shade. This is going to make it a little bit better because I do like his glimmery Celestial style formula. See, this is going to aid the look for sure. Look how pretty that is. So this shade is right on par with his regular eyeshadows and with his other palettes that he has this formula in. This is the only shade to be honest that I really like in this palette and it's doing 99% of the work in this palette. So essentially when you buy this palette, you have some background shades, but this is going to be the only shade that really works well. Okay, we're going to go into this shade again. Really dig in there. I just want to blend it along my lower lash line. And I know there is a market for very soft eyeshadows. Don't get me wrong. I've used many beautiful soft eyeshadows. This is not a very beautiful soft eyeshadow formula in my opinion. It's given me nothing. This may arguably go down as one of the worst Tom Ford quads I've ever used. And this is the one that I paid full price on. I was watching Tara's review and it was quite a negative review and I thought I feel like I could make it work. I feel like I could make a pretty look. And I normally like his quads. I really, really do. Even if what meets the eye doesn't attract me at first, I normally like it on the eye. Tara was right.  There's no way you can look at this palette and tell me this is worth $90. There's no way. I mean the glow and the glimmer is going to be pretty. Don't get me wrong. The look's not ugly. But there's no way for the look to be ugly because you can barely see it. Okay. I'm going to put on some lashes and I'm going to do some comparisons to some other palettes for you guys. I just put on mascara. I didn't do anything fancy because this palette is just terrible. If this glitter shade wasn't in this palette, you would quite honestly be throwing $89 in the trash. I'm just looking at myself and literally the only thing I can see in the monitor of my eyes is the glitter shadow. All of the other three shades that you saw me pack on with my finger and dig in later have disappeared. Okay. So let me show you some quads that I thought potentially could be comparable. But all of these quads that I'm showing you are 20 times better. So the first one that I pulled was Sula Sable. And this one isn't one of my favorites, but I think it's really wearable and work friendly. So I still enjoy this palette and reach for it from time to time because it's just super wearable, soft, pretty rose tones. No dupes really. Maybe this shade and the pop shade are similar, but Sula Sable, I mean you can actually see the shadows showing up. So they're not similar, but this one is a lot better. I got a request to do this one. So this is from the Soleil Neige collection two years ago. This is Soleil Diva. And yeah, they're kind of in that similar tone in that it's very monochromatic and they don't have too much depth. But the swatches really are different. And this is what I mean. Like the lighter Tom Ford palettes still look beautiful on the eyelid and this shade swatches with so much depth. And this is the Wet Dry formula.  And by the way, something that I didn't mention, it's not a very pink palette, to be honest. The colors lean a little bit more peach. It's naked, but it's not very pink in my opinion, anyways. Let's compare it to last year's Soleil Neige palette, which is First Frost. And this palette is phenomenal. It might not look like too much in the pan, but on the eyes, it's breathtakingly beautiful. Obviously the tones are very different. Do you, did you see, by the way, how, I'm sorry, this review is all over the place. But did you see how much more like warm and peachy this look? And this is called a pink palette. But here's the swatch differences. Incomparable, truly. I thought maybe Delacreme would be worth swatching. Now knowing what naked pink swatches like, it really wasn't worth it, but here is what they would look like. I was feeling this formula and really testing it out because it is the normal eyeshadow formula, just like the naked pink. And this shade didn't give me too much. This actually has a very similar press to the naked rose, but the mattes, like, they show up. They have pigment. It's still a beautiful palette. Okay, and then last one I picked out. This is the shade and illuminate in rose cashmere. And I just want to swatch the quad. But you can see these are kind of similar. It's the wet dry formula. See, to me, this kind of looks like what naked pink was trying to do and trying to be, but a little bit softer and playing with a little bit more different textures.  I mean, that's all I really have for this review. If you couldn't guess, I don't recommend this one and I'm disappointed because I paid for price for it and I don't do that very often. So that's why I'm here, though. I spend money so you don't have to make sure you guys like this video and comment down below, engage with this video so I can get my money back on this, okay? This review is gonna pay for this. If you are interested in Tom Ford quads, I do love so many of his quads. So don't go just off of this review. I think he has some beautiful products in this line. I just don't know how this got approved. I'm very, very confused. Ooh, the box is silver on the inside. I don't like that. Not the only thing that I like. All right, you guys, I hope you enjoyed this video and I hope it stopped you from buying this palette if you were thinking about buying it. And for those of you who did and bought through my original link when I first saw that this launch, I'm very sorry. I really hope you like it better than I did and get some loose out of it. So thank you guys so much for being subscribed to my channel and watching this video and liking it and all of that good stuff. And I will see you all in the next one. Bye, guys. Have a good one.


From around the globe, it's theCUBE with digital coverage of AWS re-invent 2020. Special coverage sponsored by AWS Worldwide Public Sector. Welcome to theCUBE virtual. This is our coverage of AWS re-invent 2020, specialized programming for Worldwide Public Sector. I'm Lisa Martin. Got a couple of guests here from Noveta. Please welcome Steven Adelman, Principal Computer Scientist and Kevin Heald, Vice President of Information Exploitation. Gentlemen, welcome to theCUBE. Thank you, thank you for having us. All right guys, so Kevin, we're going to start with you. Give our audience an introduction to Noveta. What do you, what do you guys do? Who are you? How do you play in the public sector government space? Great, yeah, thank you Lisa. So Noveta, Noveta is a technology services company focused on government solutions. So primarily national security solutions. So think customers such as DOD, the intelligence community, FBI, law enforcement and things like that. About 1,300 employees worldwide, primarily in our field, clear resources that really focus on cloud-first solutions for our customers. So solving the tough mission challenges our customers have. So that can be in technology solutions such as data analytics, AIML, IoT, secure workloads, full spectrum cyber, COP, video processing, really anything that's a high-end technology solution is something we do for the government. We have been a privilege, it's a privilege to be a partner with AWS for some time now. In fact, I think the first reinvent we may have been through, Stephen, was six years ago, five years ago. 2012 or 13. Yeah, so we've been around for a while, really kind of enjoying it. And certainly sad that we're remissing an in-person reinvent this year, but looking forward to doing it virtually. So we are actually an advanced year partner with AWS with a machine learning and government competency and really kind of to pump the ML side of that. That was one of our first competencies with AWS and led by a center of excellence that I have in my division that really focuses on machine learning and how we apply it for the mission. And so really we focus on protecting the nation and protecting our equities as a country. And on behalf of the country, we thank you. Stephen, give me a little bit of information from a double-click perspective as computer scientists. What are some of the key challenges that know that it helps its customers solve and how do you do that with AWS? Yeah, thank you. So really as a company that is data-first, so our initial love and still our kind of strongest competency is in applying solutions to large data sets. And as you can imagine, the bigger the data set the more compute you need, the more resources you need and the flexibility from those resources is truly important which led us very early as an especially in the government space and public sector space to be an early adopter of cloud resources because of the fact that rather than standing up a 200 node cluster at many millions of dollars we could spin up AWS resources, process a big data set and then get the answers an analyst or an operator needed and then spin down those resources when that kind of compute wasn't needed. And that is really kind of informed how we do our work as novatans that cloud infrastructure and now pushing into the edge compute space is still kind of keeping those cloud best practices in play to get to access to more data that the two biggest I think revolutions that we've seen with regards to using data to inform business processes and missions has been that cloud resource that allows us to do so much with so less and so much more flexibly. And then the idea of cheap compute making it to the edge and the ability to apply sensors to places where it would have been operationally or cost prohibitive to do that. And then ironically, those are two things that aren't necessarily data analytics or machine learning focused, but man did they make it easier to collect that data and process that data and then get the answers back out. So that really has kind of shaped a lot of the way Noveta has grown as a company and how we serve our customers. So Kevin back over to you, let's one of the things that we've been talking about almost all year is just the acceleration in digital transformation and how much faster organizations, private sector, public sector need to innovate to stay relevant to stay competitive. How are you working with government customers to help them innovate so quickly? You know, we are very fortunate that a set of customers that's focused is actually innovation. It's focused as I read. And you know, we can't do the cool things we do without those customer relationships that really encourage us to try new things out and quite frankly fail quickly when we need to. And so by establishing that relationship, what we've been able to do is to blend agile development, agile acquisition with government requirements process, right? If you know, the typical stereotype of government work is it's this very stove piped hard core acquisition process, right? And so we've been fortunate to instead try quick win kind of projects. And so one of the biggest things we do is partner with our government customers and try to find a difficult challenge to solve over a six to 12 month time, right? So instead of making this long four or five year acquisition cycle, it's like show me, right? How can we solve this problem? And then we partner with the mission partner, show success in six months, show that we can do it with a smaller part of money. And then as we're able to actually make that happen, it expands in something bigger, broader and then we kind of bring it together a coalition of the willing, if you will, in the government and saying, okay, are there other stakeholders that care about this problem, bring them on, bring their problems and bring them together? And we can't do that with some of the passionate people we have, like Steven's a perfect example. When we talk about Picard and the project we're doing here, Steven's passion for this technology partnered with our customers having these challenges and try to enhance what they're doing is a powerful combination. And then the last thing that we're able to do as a company is we actually spend a decent amount of our own dollars on IRED. So R&D that we fund ourselves. And so while finding those problems and spending government dollars to do that, we also have spent our own dollars on machine learning, IoT, sensor, next gen 5G and things like that and how those can partner together, partner together to go back to the government. Yeah, Kevin. Go ahead, Steven. Oh yeah. So I would even say, there's a conventional wisdom that government is slow in plotting and a little bit behind commercial best practices. But there are pockets in growing pockets across the government where they're really kind of jumping ahead of a lot of processes and getting in front of this curve and actually are quite innovative. And because they kind of started off from behind they can jump over a lot of kind of middle ground legacy technologies and they're really innovating. As Kevin said, with the Picard platform, we're partnering with PEO Digital in the Air Force and SAFCDM and Air Force Security Forces as that kind of trifecta of stakeholders who all want to kind of saw a mission problem and wanted to move forward quickly and leave the legacy behind and really take a quantum leap forward. And if anything, they're driving us to innovate more, to introduce more of those kind of modern back practices. And Noveta as a company loves to find those spots in the government sector where we've got those great partners who love what we're doing and it's this great feedback loop where we can solve hard technical problems but then see them deployed to some really important and really cool and impactful missions. And we tend to recruit that kind of nexus of people who want to both solve a really difficult problem but want to see it executed in a really impactful way as well. And then that really creates a great bond for us. And I'm really excited to say that a lot of the government is really taking a move forward in this realm. And I think it's just good for our country and good for the missions that they support. Absolutely, and it's also surprising because as you both said, there is this expectation that government processes are lengthy, laborious, not able to be turned around quickly. But as Kevin, you just said, helping customers, government agencies get impact within six to 12 months versus four to five years. So you talked about Picard, interesting name. Kevin, tell me a little bit more about that technology and what it is that you guys deliver that's unique. Well, honestly, it's probably best to start with Steven. I can give you the high level. This is Steven's vision. I have to give him credit for that. And I will say we have lots of fun acronyms. So it is an acronym, right, Steven? Doesn't it actually stand for something? It stands for platform for integrated C3 and responsive for defense. And I know that the Star Trek theme is a leg up from the last set of programs I had which were my little ponies, so. Oh, wow. That's a definite step in a different direction. I like it. Part of the great thing about working in the government is you get to name things cool things. But to get to your question, so Picard really sprung out of this idea that I had a few years ago that the world, but for our spaces, the Department of Defense and the federal government was going to see a massive influx of the desire to consume sensors from areas of responsibility, from installations, and frankly from battlefields. But they were gonna have to do it in a way that presented some real challenges that you couldn't just kind of throw compute at it or throw traditional IT processes at it. We have legacy sensors that are 40 years old sitting on installations. Old programmological controllers or facilities control systems that were written in cobalt in the 70s, right? And are not even IP based, most of them. And then on the other end of the spectrum, you have seven figure sensors that are throwing out megabits of second of data that are mounted to the back of jeeps, right? That are bouncing through the desert today, but will be bouncing through the jungle tomorrow. And you have to find all of those kind of and combine all of those together and kind of create a cohesive data center for data set set for the mission for what we call a user defined common operating picture for a person to kind of combine all of those different resources and make it work for them. And so we found a great partner with security forces. They realized that they wanted to make a quantum leap forward. They had this idea that the next defender so they're like a military police outfit that the next defender was going to be a data-driven defender. And they were gonna have to win the information war as much as they had to kind of dominate physical space. And they immediately got what we were trying to achieve. And it was just this great synergy. And then we've piled on some other elements and we're really moving that platform forward to kind of take every little bit of information we can get from the areas of responsibility and get it into your modern data lake where they can extract information from all that data. Kevin, as the VP of information exploitation that's a very interesting title. How are you helping government organizations to win the war on information, leverage that information to make big impact fast? Yeah, I mean, I think a lot of it is is that we try to break down the barriers between systems and data so that we can actually enable that data to fuse together to find and get insights into it. As ML and AI have become trendy topics, they're very data hungry operations. And I think what Stephen has done with the card and his team is really we wanna be able to make those sensors seamless from a plug and play perspective that I can plug in a new sensor. It's a standards-based interface that sends that data back so that we can then take it back to the user to find operation picture and make some decisions based off of that data. What's more is that data could even be fused with more than the data that Stephen's collecting off those sensors. It can be commercial data, other government data. And I think as Stephen said earlier, you have to get it back. And as long as you've gotten it back and you ever share it with some of our mission partners then you can do amazing things with it. And Stephen, I know you have some pretty cool ideas on what we're gonna do on the edge, right? How do we do some of this work of the edge where a sensor doesn't allow us to pull all that data back? Yeah, and to follow on to what you were kind of referring to with regards to handling heterogeneous data from different sensors. One of the main things that our government customers and we have seen is that there are a lot of, historically there are a lot of vertical solutions where the sensor, the platform and then the data lake are kind of all part of this proprietary stack. And we quickly realized that that just doesn't work. And so one of the major thrusts of that card platform was to make sure that we had a platform by which we could consume data through adapters from essentially any sensor speaking any protocol with any style data object, whether that was an industry standard or a proprietary protocol, we can quickly ingest it and bring it into our data lake. And then to pile on to what Kevin was talking about with compute, right? So you have like almost like a Maslach's hierarchy of needs when it comes to cyber data or to this IoT data or kind of unified data, you wanna turn it into basic information, alerts alarms, then you wanna do reporting on it or analytics or some higher level workflow function. And then finally, you probably wanna perform some analytics or some trending or sort of anomaly detection on it. And that gets more computationally intensive each step of the way. And so you gotta build a platform that allows you to both take some of that high level compute down to the edge, but also then bring some of that data up into the clouds where you can do that processing and you have to have kind of fungibility between that. And so that card platform allows you to kind of bring GPUs and high processing units down to the edge and make that work. But then also, and then as maybe even a first pass sieve to rule out some of the most, some of the boring data in the video analytics platform we call it blue sky and blue ocean, right? So you're recording lots of video that's not that interesting. How do you filter that out? So you're only sending the information of the interesting video up. So you're not wasting bandwidth on stuff that just doesn't matter. And so it's a lot of kind of tuning these knobs and having a flexible enough platform that you can bring compute down when you need it and you can bring data up to compute on big cloud while you need it and just kind of finding a way to tune that. That really does, I mean, that's a lot of words about how you do that but what that comes to is flexible hardware and being able to apply those dev ops and CI CD platform characteristics to that edge hardware and having a unified platform that allows you to kind of orchestrate your applications in your services all the way up and down your stack from microcontrollers to a big cloud instantiation. You make that sound so easy, Stephen. Kevin, let's wrap it up with you in terms of like making impacts and going forward we know the edge has exploded even more during this very interesting year and that's going to be something that's probably going to stay as a permanent impact or effect. What are some of the things that we can expect in 2021 in terms of how you're able to help government organizations capitalize on that, find things faster, make impact faster? Yeah, I mean, I think the cool thing we're seeing is that there's a lot more commoditization of sensors. There's a lot more sensor information so let's use LIDAR as an example. Things are getting cheaper and so we can all of a sudden do more and more things of the edge than we ever would have expected. When Stephen's team is integrating camera data and fence data from 40 years ago, it's just saying on off, it's not doing anything fancy. But now Stephen, I can't remember what the measure you gave me before was but the cost of LIDAR has dropped so significantly that we can now then deploy that. We can actually roll it out there and not be locked in there for proprietary system. So I see that being very powerful. Also, I can see where you start having sensors interact with each other, right? So one sensor finds one thing and then a good example that we've started to experiment with and I think Stephen, you could touch on it is using like triggering a sensor triggers a drone to actually investigate what's going on and then therefore a hybrid video back and then automatically can investigate instead of having to deploy a defender to actually see what happened at that end point, Stephen. I know there's some more detail you can provide there. Yeah, no, so exactly that Kevin. So the power of the sensor is something old that gives you very uninteresting data like a one or a zero or an on or off can detect something very specific and then do something kind of high speed like task a drone to give you a visual assessment and then run object detection or facial recognition on, you know, do object detection to find a person and do facial recognition on that person to find out if that's a patrol walking through a field or a bad guy trying to invade your space. And so it's really the confluence and the gestalt of all of these sensors in the analytics working together that really creates the power from very simple delivery. I think there's this idea that, you know, 100 bytes of data is not that important but when you put a million sensors giving you 100 bytes of data you can truly find something extremely powerful. And then when you make those interactions sing it's amazing to us the productivity that we can produce and the kind of fidelity of response that we can give to actors in this space whether that's a defender trying to defend a base or a maintenance person trying to proactively replace of the fan or clean the fan on an HVAC system. So, you know, there isn't a fire at a base or interesting enough one of the things that we've been able to achieve is we've taken maintenance data for helicopter engines and we've been able to proactively say, hey, you need to take care of this part of the helicopter engine and it saves money, it saves down times, it keeps the birds in the air and it's a relatively simple algorithm that we were able to achieve and we were able to do that with the maintenance people bring them along in this endeavor and create analytics that they understood and could trust. And so I think that's really the power of this space. Tremendous power. I wish we had more time to dig into it guys. Thank you so much for sharing, not just your insights, what Novot is doing but your passion for what you're doing and how you're making such an impact. Your passion is definitely palpable. Stephen, Kevin, thank you for joining me today. Well, thank you. For my guests, I'm Lisa Martin. You're watching theCUBE virtual.