So, the title is here, Algorithmic Resolution, in fact it will be, what I'll talk about is an algorithm of resolution of singularities of logarithmic varieties, this is a sort of algorithmic identification of classical algorithm, so half of my talk will be about classical and half will be about the modification, let me start with introduction, I'll formulate one of many results and also with some motivation introduction, so first of all we always fix characteristic to be zero, the only case where classical is known, so we also only work here, what is over Q, we always work over Q, in addition by Varkey, I'll denote the category of integral or maybe locally, maybe the joint union, it does not really matter, integral varieties, if you care your characteristic zero, so for simplicity I'll only discuss this case and by Varkey log, so varieties of finite type of K, which are local integral, and Varkey log the same guys with FS log structure over K and trivial log structure, so classical resolution deals with the resolution of these things, canonical, factorial and so on, as I'll formulate in algorithmic we'll deal with this, okay now let me define maybe first a theorem classical, theorem one, which is classical, factorial resolution, it says the following, for any Z in Varkey such variety there exists a modification, it is proper vibrational map, Z res to Z, I'll denote this map F of Z, such that Z res is smooth, so this is indeed a resolution in a classical sense, proper vibrational, and the source is smooth, and in addition this is about funturality, F of Z is a funtorial, or compatible funtorial, for all smooth maps Y to Z, which is F of Y is just pullback of F of Z to Y, so funturality maybe just to mention few names, I don't know try it on the board, but this was proved without funtoriality just existence for any Z separately, by Hironak in 64, when it was done canonically, but is at least compatible with all toporphisms, canonical was done by sort of I think in the beginning of 90s by Gerstle-Millman and Bill Major, and later Lodaksek in 2000 actually showed that in fact this is even better, this is funtorial false smooth morphisms, there are many different descriptions of this algorithm, but it's always the same algorithm, so far, yeah and in fact the algorithm I'm talking about it also applies to the case when Lodaksek is trivial, so it also recovers this and it gives up in you, so I think I remember from some references sometimes it goes that the algorithms in the 90s like there was Bill Major first and it was not exactly the same that is there are different steps in some way, they are absolutely minor, but this is set up in neutral parts, you can do them in a little bit in different order or you can be not sure what it is okay and you can do it twice or twice instead of once and this is more or less with only difference, so to large extent it's the same algorithm, it's the same in gene, the same invariance and it was also insane as I think in Bill Major paper, yeah okay let's let's discuss it in questions, yeah okay and now theorem one law, logarithmic modification, this is by joint project with Abramovich and Lodaksek, everything I'm talking about here is in the framework of this joint project and it says we're folding, for any z in logarithmic there exists a modification z res to z, I'll denote it f logarithmic of z, a logarithmic algorithm such that the origin is log smooth and equals f of z product with y over z product in the fs category, so this is not just pullback, it is fs pullback we are working here, for any log smooth y goes to z and this log smooth functionality is much stronger than classical one, for example we can extract roots of exceptional devices with this we can cover branches covers branches of exceptional devices and it's still a kumar cover, it's log smooth, so we get from the divided bzb, so the main strength of this result is that it goes for log smooth, okay and what do you mean by the modification? it's just on the level of modification, it's no restitution on log structure, just modification on level of, but I will, it's just modification on level of under 1, right, but your condition of locally integral is not stable under the general local models of log smooth things because you you can have something which becomes at least for normal thing usually is the same but for just locally integral, if you take tens of zp, zq, it's not necessarily, so for even just extension of the field of scars doesn't preserve locally integral so it's, I don't know if it preserves, it presents a problem in the, it is definitely not a problem, in fact our theorem is a little bit more general, I just, I don't want to go to equidimensional and so on, just let me keep it yeah I allow for myself to cheat a little bit at two stages, yeah and the main cheating in this theorem is not what authors is saying, the main cheating is that in fact 0s is what we call a toroidal orbital, that is it is a dilemma for a stack with finite diagonalizable, so to get this stronger funtoriality we are forced to go to much more wide category and as far as we understand there is no way to get full funtoriality without passing those stacks, at least we have no idea how to do it and we suspect it's probably it's impossible, okay now one more remark is that mismatch z res to z is an isomorphism over the log smooth locus, so if I'm given z res to z and here I have z log smooth, over sub skin where this guy is log smooth, when actually I'll get isomorphism above this thing, this immediately follows from funtoriality because first of all this morphism should induce resolution of z log smooth and z log smooth is log smooth over point, resolution of point can be only a point, so just I have no choices, contrary to what it implies but I'm not touching with whole locus where this question is already resolved, okay good now maybe I'll just say one word about motivation, motivation is to the result morphisms z to b modifying this we would like to get to find modifications of the base entropy source so with this map is log smooth or lodging out for appropriate, for lodging out log structure on b prime and appropriate log structure on z, now it's known that just what's usually called semi-stable is impossible if the base is of dimension 2 and more and the relative dimension is 2 and more but semi-stability is an initial thing to expect with this possible and in fact such a curem is known in non-colonical folk way, it was first proved by Romesh Karo for Varietis and in a work on Travodegaber with Lukluzy we improved it to classic schemes of characteristic zero or it is a class of classic schemes but it's not factorial and because of that it modifies with whole generic fiber, it can modify generic fiber even if generic fiber is smooth so this weak version of resolution of morphisms does not imply semi-stable reduction and over-versionary and one of our motivations was to improve semi-stable reduction of general work with space and the only curem which I see is to prove phantorial resolution of morphisms it must be phantorial for log smooth, it must have stronger funtoriality than classical and actually the algorithm which we discovered now should be the generic fiber what's going on in hypothetical relative algorithm over generic fiber of zero. So what is the non-phantorial thing you are thinking about just not alteration but modification? Modification yes, I okay let me I just give maybe I'll say one word but we'll never see it again yeah it was just for to motivate the point is that we resolved indeed by modifications of both below any modification I don't care much to be a large enough modification of it and after large enough modification it should be just depend on pullback of Z in a log smooth phantorial way okay I'll yeah proceed good so far is introduction now let's go to classical methods so now I'll tell about hirnakas approach and developed by all the other people I mentioned so two classical methods so let's start with principleization hirnaka reduces in question of resolution to be following to be following question so I'll just say okay maybe I'll just say the words the idea is as follows first of all we want to prove something phantorial so it's enough to construct our algorithm locally because of cononicity it will glue automatically locally with Z can be embedded into a smooth ambient space and up to it our morphisms we is embedding only depends on the dimension of the ambient manifold so in principle there is not much choices here so it's more or less safe to assume that we are given economical embedding into something smooth ambient manifold and we would like to work on this smooth manifold such case we can work with Z or we can work with so embed this is a manifold that is just by this I mean smooth something smooth and work with the ideal which defines the Z inside the restructure so and we'll want to achieve something which is called principleization of this I now let me give a few definitions so first definition boundary is a strict normal crossing divisor e inside x so even if we start with empty boundary it will appear after blowing up when we'll start to blow up x this is very crucial for runacosmetic second we say that v inside x a closed sub scheme has simple normal crossing with e if locally there exists coordinates t1 up to tn at the point such that v is given by vanishing of t1 up to some number t r so in particular it is smooth and e is also given by coordinates but maybe some different coordinates maybe few of them from here and few of them not from here so e is given by vanishing of ti1 times ti2 ti s okay so namely I guess it's very simple we can just pick up local coordinates so with all our data is given as coordinate as linear planes with respect to these coordinates another definition i admissible is the following datum x prime e prime this is just formal notation I don't want I don't have time to define category of triples so this for us it will be just formal notation where x prime is blow up along some v v has simple normal crossings with the boundary we started with so we are only allowed yeah admissibility means that they are only allowed to blow up something with normal crossing and in addition and in addition v of is contained in the vanishing local supply and also we only allowed to modify the locals where I is not trivial e prime equals to let me denote this map to x by f e prime equals to preimage of the old boundary we call this old component union with preimage of v which is new component and i prime is just pullback of i so the ideal is just pullback and now exercise x prime is smooth a manifold and e prime is a boundary just because our definition immediately allows to go to local coordinates you just compute everything locally and what's it okay now theorem 2 principalization says that for any such a triple e i where exists a sequence of admissible I'll denote it x n e n i n this guy is sort of result what do I mean result that means we call it i n is invertible and monomial the vanishing locals of i n is contained in e n so after this procedure we manage to make the ideal to be just product of few components of e n with some multiplicity okay and now maybe just remark why this theorem is stronger than theorem 1 so if z is vanishing locals of i in this situation and we start with empty boundary then let z n equal to empty set goes to z to z n minus one and so on to z sequence of strict transform where the total strict transform is empty why because the pullback of i is supported on the exceptional boundary so it necessarily means that and we started with empty boundary so it means that some fish strict transform was killed and then take maximum such that the i is not empty easy to see I will go to detail but it's very easy but in such case the eye is just the eye eye center and hence the eye has simple normal crossing with the eye and this implies that first of all the eye is smooth but even more it implies that the restriction of the eye is bounded is is the simple normal crossing device so in a sense this theorem implicitly gives indication that you are not just resolving z you are resolving a logarithmic scheme you solve strong the problem you resolve logarithmic scheme z i and the restriction of the boundary so even in classical situations there is a flavor of logarithmic things and another maybe at least I shall say the question what the boundary is so usually at least when I started to deal with resolution I thought it's just some nice subscreen and it's not good to think about this as a subscreen because because materiality is wrong there's no map from e n to empty to e which is empty so this is not this morphism they're not morphisms of pairs if you want to view this voluntarily we should think of e as a lock structure we should consider the lock structure associated with the simple normal crossing device and when indeed we have such a map from between logarithmic schemes and on the gravity of the ideal is direct okay good yeah I'll just say it without without writing on the board moreover we can a little bit it was not written anywhere but one can prove a little bit stronger result instead of resolving z with empty lock structure you can consider z with a deline falting lock structure lock structure whose monoids are free any such lock scheme can be embedded by strict closed immersion into some pair x e where x is loose and e is simple normal crossing device in such case actually you'll get a strengthening of pure one to lock schemes with deline falting's lock structure so it was very natural question also if the classical algorithm which in fact results deline falting's lock schemes can be extended to the algorithm which works with general lock schemes at cost of working here with general lock smooth lock operators and this is indeed what's done in our algorithm so this is in other motivation sort of absolute orthogonal to the post I started this to construct logarithmic extension of this algorithm good now next topic is order reduction so main variant of the algorithm is order of i at the point okay I gave a lecture a little bit before I stated this a little bit differently but again I allowed myself to cheat a little bit order of i which is which is just a minimum over all f inside the stock of i at x uh excuse me minimum of order f at x where f belongs to the stock of i at x and order of a function on a manifold is just the order of minimal um non-venetian term in Taylor series for example what did you write main invariant f at f what is after of f is order of i at x what is f f is the algorithm let me say okay of the algorithm okay okay I didn't just let's give a definition I would definitely prefer weighted but people coined you know we weren't marked so now it's always marked ideal it's just id where i is ideal and d is a number which is like one zero and it will be just formal combination of but so so I don't pretend to give any deep meaning to this but I will consider a singular locus of such guy as all points in the vanishing locus of i such that the order of i at x is at least so we would like to reduce d so essentially we look at places where the order is at least some number and the formal will work in such way and in addition we say with a blow up let's say an admissible such guy is called id admissible blow up if is contained in id singular that is everywhere where we blow up the order is at least d and f is i admissible in all sense but this we blow up the multiplicity is d and the the center has simple normal crosses with the boundary okay and moreover we define transform for a such we define transform i prime which is smarter when what we have done before before we just took pull back but now I say let's take pull back and let's divide it by this power of a new exceptional divisor what is I take ideal of v pull it back this is the ideal of the new exceptional divisor and take it to minus d so is here there is an e of boundary or not in the definition of mark day in the definition of mark ideal there is no there is no d there is no d it is okay e is not is not okay but the the invisibility are required to be simple normal crossing with the power yeah when I say that it is i admissible it means that it's simple normal crossing with the power okay so such a thing is defined only because of the condition that my center is inside d multiple locus it's very easy again simple exercise check that this is defined that we can divide by this okay and then if you're in free it's called order reduction of marked ideals and it says this following it says that any marked ideal e i possess a sequence d admissible blowups you know the x prime d prime d but it's a sequence it's not a single one it's a sequence and each time we transform ideal by such a such way this guy is a result it's order it's momenty again as in theorem 1 or theorem 2 this sequence should be factorial and not spelled without but it's natural continuity for arbitrary smooth morphisms between initial that and maybe i'll just mention that theorem 2 is just theorem 3 for d equal 1 so theorem 2 is just part of your case and it was something for inductive reasons it's much much much more convenient to prove okay now in a logarithmic setting if you get to some details it will be the same structure we'll have fewer in one logarithmic, fewer in two logarithmic and fewer in three one so here x is smooth always it's manifold okay it's a manifold is a boundary i is any ideal and d is any number after reduction of theorem 1 to theorem 2 we only work with smooth things and ideals inside well we don't see any non-smooth geometry okay all singularities are encoded in the ideal which is just algebraic work so if the ideal is zero you just blow up everything and get the amplitude yes yes yes and in induction this sometimes this happens so it can't happen with the other if you ask it now when yeah it can happen when the other is infinity for ideal zero and indeed in such case you just should blow up everything and in logarithmic setting we'll have what infinity not only for zero it will be more interesting but okay we'll get good now next so let me explain a little bit how one proves theorem 3 so remind me if d equal yes equal one then i prime is the pre-major of i or but you each time you subtract one copy of exception divisor okay and you won't invent to get to empty guy okay if this one when the resolving it means just with its locus is empty okay so by subtracting each time just one copy of exception divisor you invent manage to completely resolve it and obviously this implies theorem 2 because it means that your public of your initial ideal becomes just union of exception okay so maximal contact and induction on the dimension okay so what i said so far actually is characteristic 3 this is an indication that it's not that deep but maximal contact is the first notion which will be really good okay so my miracle is that the maximal order that is the case when order of i at any point x is less or equal to a reduction to small dimension and it goes as follows the problem of resolving i and d is equivalent to the problem of resolving ideal c of i called coefficient ideal restricted to h called maximal contact hyper surface and beyond this defector so the induction on the mention runs as follows we replace our data by some other data and any sequence which results we knew that there is also real data in vice versa so the problem is sort of equivalent we encode everything which we had about original question into a question which lives in dimension one less not the way we can run induction and this is what you're an hacker did on idealistic this is a there is an exponent it's more about definition of market ideal okay this is okay but anyway idea maximal contact was formalized by zero but ideas going to run out about you know it took a lot of time to refine them and to understand what is real in gene self-contained in gene yeah and we thought out that there is an odd reduction part ideals is the minimal self-contained block which can prove itself by induction though the rest was sort of unnecessary in original but okay okay anyway now main example just to illustrate how this works uh so let's assume that i is just given by t to some power d plus a1 uh td minus one uh sorry a2 td minus two plus a d where t is t1 top coordinate and a i actually depend on t2 up to t on the other coordinates yeah for example locally we can write some Taylor series which gives something like this so let's assume that we are given such a just particular case or for the and then the maximum content is just vanished a lot of softy and coefficient ideal is just the ideal generated by a2 d factorial over two ad de factorial over d so it's just generated by coefficients but in weighted weight we should take each of them with correct weight and it's more or less clear if the weight of this guy should be two and the weight of this guy should be d so we weigh them and uh i'll just uh show one thing i claim with id singular with this we play the points where the multiplicity is d is the same as co5 restricted on two page de factorial singular that is the other obvious guy is d even though only the other obvious guy is at least two the order of next guy is at least pre and the order of this guy is at least d and this precisely means what is the definition so at least uh equality of initial sin singular and local lots of it's clear so uh uh id admissible block is any block who centralized here it's the same as block which is admissible with respect to new data now it's much more difficult but possible to prove it this miracle persists after any admissible block so after first block again we'll have a quality of singular lots of transformations so and you can also ask what where is a1 so the answer is that because of characteristic zero i don't need a1 i can always get rid of a1 and this is the case but i can't take this if uh not reveal a1 completely now let me say a few words about general case yeah now what i wrote now is sort of very coordinate dependent and i would like to construct something canonical something coordinative so this is achieved to large extent by derivations so one considers d just ideal of derivations on x over k the i is generated by blocker is generated by dt1 okay and uh one defines d of i to be equal let me say d of ideal generated by let me let me say i is generated by f1 fm then d of i is generated by f1 fm dt1 f1 dt1 fm dt2 and so on so derivation of ideal is generated by the ideal itself and all derivations of elements inside and then we can iterate and divide uh iterated derivation of good and uh when we can encode almost all basic uh of the algorithm which i described so far by use of derivations so it goes as follows first of all order of i at the point x is just minimal number d such that with this derivation of ix is trivial so derivation just reduces order by 1 it's a very simple exercise in local coordinate okay two uh maximum compact is any h which is v of t where order of t is one that is this is a really smooth guy t is a coordinate and t is contained in d minus first so we know where d minus first derivation has order one yeah because order reduces by one each time and so it contains elements of order one and these are naturally also this is very intuitive but such an h has maximal contact with our initial problem it's as close to initial problem as possible and free uh coefficient ideal is just weighted sum of derivations power d factorial over d minus i yeah so it's just weighted sum of derivations okay now i managed to define order in coordinate independent way i managed to define coefficient ideal in coordinate independent way i have a choice of t here and this choice is really here they can the only real issue to prove independence of the construction is independence of maximal contact this really this was an issue and it solved in few ways but no secret so in a sense this is maybe one of main technical uh problems in constructing our uh okay good uh i think okay maybe just one what are the complications of this method this as i see them yeah uh because of time restrictions i you'll have to believe me i i didn't give enough details that you you'll feel it but uh so uh first of all complication one is that uh p can be non-transversal in such case uh i cannot restrict my problem to h because restriction of e to h is not a boundary uh to solve this one actually first of all takes care of the of the old boundary new boundary will not pose a problem it will be always transferred to h but old boundary is a is a problem and one actually solves uh first of all resolves i the order of i reduces order of i uh it places where there is a maximal multiplicity of the old boundary so there is a secondary invariant already in here like a paper multiplicity of the old boundary and because of this invariant of the algorithm looks as d one s one d two s two and so on where this is because order we start with the order of i or maybe order of its non non non non part and then the number of exceptional divisors for a point after that the order of d uh the order of uh coefficient ideal restricted to uh maximal contact and uh again number of components of exceptional divisor and so on good and second i'll just say by worse uh in principle it would be much better to work with logarithmic derivations because all formulas even what happens to derivations after uh admissible plot they're easier for logarithmic derivations and not for usual derivations there is one point where uh use of usual derivations is completely critical and it is for the definition of the order our definition of order does not separate uh coordinates corresponding to exceptional divisors and other coordinates so we do not separate different uh logarithmic derivations they are different with respect to usual coordinates at logarithmic class but the order is uh completely not sensitive to this uh in a sense all combinatorial complications of the classical algorithm those which i put on the verac are actually because of uh this non-separating of two types of variables exceptional and non-exception okay good now let me uh start uh the logarithmic option so what is this conflict i didn't understand about this conflict about this complication so if the maximal contact hypersurface is not transversal to the boundary when i just i can i cannot i cannot use this h i can restrict i but i cannot restrict the boundary because i cannot restrict the boundary i cannot guarantee that uh the uh i can resolve i with empty boundary on this h yes but it can be no non non-admissible it can be non so no simple crossing to h but if i always blow up also in the maximal multiplicity locals of the boundary when i automatically uh have simple normal crossings with the boundary okay offer let me yeah yeah i guess everybody else is often enjoying but i'll explain i'll explain you later but just private okay good and i'll look where it becomes okay uh so main idea is uh favorite one look just replace look uh for example instead of xe just consider any uh log manifold which i'll sometimes maybe write as x and mx just look smooth right or sometimes we can represent it as x e but this time e is just x minus the reality for the log structure in addition replace d by logarithmic derivations by d log and replace other by log order and so on so just put log everywhere you can so let me start with such a procedure log order log manifold is just minimal d so with logarithmic derivation of this top of i becomes trivial so this guy belongs to n and infinity yeah it can be infinite it may happen but we never get but already in classical situation this was the case for zero ideal and only for zero ideal here it can happen more frequently if you take any uh monomial when any log derivation uh just multiplies it by number yeah all monomials they are eigen uh functions of log derivations so log order of any monomial is actually infinite so there are a lot of uh guys of infinite order but this will not be a problem in a sense this gives us this separation of two types of coordinates we have usual coordinates of what about and we have monomial coordinates which have infinite order you should not deal with them at all by use of derivations or by use of the classical immigrants they are completely in combinatorial side of algorithm so we'll get distinguishing between combinatorial parts coming from log structure and geometric part which is maximal contact okay well again just put here walk and you are done coefficient ideal maybe i'll put here logarithmic coefficient again put here log and you are done uh uh four as i said this time x is any log main point to go to another board what about e the boundary boundary is what i wrote where e is just x minus x three so log manifold is a f s uh smooth or what is log manifold yes f s smooth all all yeah yes log smooth log smooth maybe i'll just give uh six local uh let me see local picture completion uh formal completion uh looks as follows it looks as k you are joining uh some monoid and also you are joining t one up to t n so we call these guys regular coordinates we have other one and we go everything here monomial coordinates and this time i don't have any other for monomial coordinates all monomial coordinates are equally good for me and uh for example my algorithm should be compatible with taking extracting roots of monomial coordinates so i have no chance to have any or any reasonable lot of monomial coordinates we adjust combinatorial part of it okay and uh j uh d equal differential focus of j is admissible if j is over four t one up to t i let's say r comma m one comma m s where these guys are just every every terminology so now we are allowed to blow up any guy for this form so uh r of t one t n is a log sub manifold this guy is uh what's is a monoidal uh separate and uh no restrictions on numbers so uh obviously uh even if i start with something smooth after doing such blow up i immediately get something which is only log smooth but not smooth exercise again exercise check the blow up of x if need is log smooth again a very simple exercise but i just only should play suffer blow up and then suffer okay i should go to conclusion so i one t i something not not here j is t one up to t few t's and then some monomials okay the indices are not what you want no indices this is aren't yeah yeah in this okay yeah yeah i want to i am yeah okay and uh now uh in fact uh so uh the proof theorems uh two log and three log with respect to this notion of order and uh obvious uh generalization of uh marked ideals and uh the by the same by the same procedure it even uh the main invariant is just d one d two n where d i leaves inside n and infinity so the algorithm simplifies we don't have to separate from the old boundary uh we only have usual uh passage to maximum contact and that's it but we must explain how about uh plus one more stage which we call monomial stage which happens if d is in let me just illustrate how this may look like this may look like if i is generated by functions like m i j t to some power j by elements uh which for example informal completion look like this and all these m i j are non invertible in such case the other is yeah uh in this case just blow up just blow up the ideal generated by m i j by the monomial coefficients just blow up this guy and may transform with respect just subtract the exceptional device what you get will be ideal of finite order and after that you can run so m i j are where are in the here are monomial are monomial coefficients of uh unusual coordinates in yeah i i i i i am working in in this yeah so you have six in the monomial okay times uh powers of the t's yes yes and you just form ideal monomials this is where this is in fact this is the minimal ideal monomial ideal gene containing i this is invariant definition yeah if you want invariant you just consider minimal monomial definition containing i blow it up and you get finite order and uh i'll just say by worst yeah unfortunately yeah we are completely out of time uh the real complication yeah which i wanted yeah ten minutes to talk about is that uh sometimes the algorithm insists that you must blow up something like m to one over d for invincibility reasons and because of that in fact uh one such thing kills you must go to kumer uh italic apology and in fact we work not with ideals on what we want is kumer uh ideals and uh blow up of such things actually provides stacks so this is the reason why stacks appear and the i stop here i think i guess you can start with the linear monomorphs logarithmic stack from the beginning yes absolutely yes uh that's that's correct uh but uh i i i think we formulate in this general in generality of the algorithm but to formulate first of all the the theorem it's easier to start this this this right also maybe just as a remark if you start this variety you can also want to finish this right no this is a step so after that you can there is also a step how to pass back to varieties but it is less factorial it's only factorial with respect to saturated uh locksmith this is a statement so you have it you have a better statement for saturated morphisms or i have a statement which says that if you start with variety you can end with variety with law with locksmith writing in the process will be factorial for saturated logmore and it means not for kumer covers but about other questions so in the right in case you need to introduce the doing of the stack which point you need right at which point i need step okay uh 35 minutes okay okay so i'll i'll try to very very briefly yeah so the point is involved i i skipped one more condition condition i know six seven six admissibility or id admissibility and this actually uh in this case it's not so clear what should be this condition of admissibility because my center is complicated i contain cinnamon but it turns out that the condition actually is very simple we just want i to be contained in this power of j this is the correct generalization of kernakas condition which we have blown up uh demultiple centers if this is satisfied when after the transform i can divide transform of a public of i by this power of public of j so this is the the correct condition now in case i apply this monomial stage for example uh i want uh let's take this big k of m monomial coordinate and i have ideal m2 for inductive reasons it can happen with starting from something innocent i i have to resolve something like this in such case obviously i would like to blow up m but j equal m is not admissible after blowing up such ideal i cannot divide by m square what is admissible is j to one half with this m to one half when i can blow blow it up and divide by by by the square so here it looks completely as a trick but if i if this happens only on h and i started from some x or x this can be highly non trivial just just one example which completely illustrates this is a suppose takes m one coordinate is remember and one is monomial yeah so e is vanishing of m and let's take i equal x square minus m so in classical situation and in classical situation uh it's only one but all the other is two because m is of infinity so uh so we'll consider this guy is order two now resolution says that okay you should go to h which is k of spec k of m and then either uh and you restrict i and you still have you restrict i to h and you have multiplicity so uh on h you must blow up m to one half and on x you must blow up something like x comma x to one half this is the center you must now how can we understand such a thing there are two ways first of all we can try say okay let's try a trick let's try to blow up x square comma m problem is that what we get will not be walk smooth uh we insert the bed singularity and this will be very difficult to control so blow up of this guy is not smooth not walk smooth not walk smooth but instead of this you can say okay my algorithm is factorial for loxmas covers so instead of uh walking is x let's go to y just adjoin a root of here we can easily blow up yeah so let's call is n and when my yield is x square minus n square it's very easy to resolve idea x square minus n square just blow up blow up x comma n we get by prime this is zero to cover so we can now divide this by zero two and that's what we get here if i just divide as a course model space i get the bed blow up i describe here so i cannot divide back with skipping lock structure but i can divide this as a stack this guy's and this is the sort of blow up what we call kumar blow up of x and one half so it is a stack for the log kumar log tantopology no it's stack for usual though it's enough to work as usual for policy we don't have to go stacks are just conventional stacks the ideals ideals for kumar tantopology yeah it's a little bit confusing but again we wanted to keep everything as simple as possible so for stacks it was possible to work on it is uh yeah but the y to x original is not an attire it's a tile kumar attire and not usually an attire for the stack quotient of y but okay okay maybe maybe we should discuss questions from paging yeah no questions from him to question from paris maybe yeah question i can discuss this later we see so you can say the speaker again


Hey everybody. So I'm going to start right on time so that we have as much time for questions as possible at the end. So thanks for coming to our presentation on Drupal Media Image Captions using AI. To start off, we'll do a brief introduction. I'm Laura and this is Vauban. We are both senior software developers at MyPlanet, which is a digital studio that specializes in smarter interfaces. We have a wide variety of clients and a lot of different technologies we use, including Drupal, AI implementations, custom JavaScript applications, just to name a few. MyPlanet is located in Toronto, Canada, which is where I'm from, and Vauban lives in Serbia. So one really cool thing about this talk is that this is the first time that Vauban and I have met in person after working together for almost two years. So that's one of the really cool things about DrupalCon is that it brings together developers from all over, including the ones you work with. Okay, so on to the presentation.  So first, the burning question. When you think about Drupal Media Image Captions suggestions using AI, one of the first questions you might ask is why? What is the use case for this? So, oops, wait a minute, sorry, my technical moment function. It's not changing my notes for the next slide. All right, I'll just wing it. Okay, so why use AI captions in a CMS? So a lot of the time, so basically it saves a lot of time. I know that as a developer when I am developing an image feature, for example, and I have to enter in an image over and over again, I'm just going to enter a test in the alt field. And I think that it's true that a lot of us take shortcuts, which is why a lot of the time we end up with sort of bad alt captions or no alternative text at all. Another use case is if you have a large archive of images, you know, I mean, if we think that it takes a lot of time to make one image caption, if you've got a large archive, that's going to take you forever to add image captions. So having a tool that is going to insert image captions that you can go and review later on is great. And we would strongly recommend that in both cases, like whether you're using it for entering one image or for a giant archive, that you go back and review the image captions. Because a lot of the time, well, sometimes, a percentage of the time, you get unexpected results. So you want to go ahead and review them if possible. So another reason to use it is that it is a prompt for content authors to enter a proper caption. So in the same way that, you know, Drupal has the alt field is required by default. And that in itself is a prompt to enter something. But if you have a button that says, you know, generate image caption, it's a prompt to enter a full sentence or, you know, a good caption rather than just, you know, a single word. And another possibility is as an educational tool that describes what types of captions are best for screen reader users. And I'll talk about that a little bit more later when I talk about the Microsoft ability initiative. But so just keep it in mind for now as possible use case. So a bit of background about the model. We decided that we wanted to focus in this demo on the integration of the model with Drupal. So while Boeben is going to talk a little bit later about how, you know, creating a model like this is within reach of any of us, you know, using TensorFlow. The model that we're using is implemented with TensorFlow. We're using an API that is provided by IBM. So the model was already created and it was pre-trained using a data set. It is based on a model called the show and tell model that was developed by Google scientists, research scientists. And yeah, and they developed it at the, in the beginning, they had another model that they were using to translate from one language to another using an encoder, decoder type model. So they decided that they were going to try to take that model and use it for translation of images into sentences. And so they replaced the encoder part which encoded sentences with a CNN that encodes images. So the encoder now encodes images and the decoder decodes them into producing sentences. So this is a little bit more information about the encoder, decoder model. So the encoder is a deep convolutional neural network called inception of U3. It was developed by Google and it's pre-trained using a very large data set. The model, these inception B3 models are currently considered to be state of the art for encoding images. The decoder is an RNN which is also called our current neural network and also called a long term, a long, short term memory network. And they are good for sequencing and in this case the network produces one word at a time. So the sequence is actually the sentence that it produces. So what we're looking at here is a deep convolutional neural network. The role of the DCNN is to reduce images into a form which is easier to process and less expensive to process. So what we see is this is a visualization of convolution. So in convolution of an image there is a filter which is the smaller object there that's also called a kernel. And it scans over images and moves across and transfers them and takes out a lot of the dimensions. Making sure to save the edges and the most important aspects and then passes that along to the next layer. So a deep convolutional network may have many layers which is actually what the term deep refers to is layer upon layer and each excessive one sort of extracting more data from the image and reducing it more. So recurrent neural network. Now this image is actually a representation provided in the show and tell documentation. So what we see on the left here is the deep DCNN. And that image even though you can't see it right now is a representation of the convolutions. So like all of those the little chunks sort of represent the different convolutional layers. It outputs an image vector and then that gets handed over to the decoder which is the RNN. And so basically at each stage in the RNN it's generating one word of the sentence. Then they use a tool called beam search that at each stage grabs the top three sentences that are produced. They did a little bit of experimentation and figured out that the correct beam size. So beam size meaning the number of sentences that it keeps at each iteration is three. And you'll see during the demo that what the model gives back to us at the end is in fact three sentences. The data set. The data set is really important. So different data sets will give you drastically different results. The data set that we're using is sort of a generic one. It's very large. It's meant to cover most use cases. You can also like if you had a use case that was very specific. For example, if you wanted a model that was able to identify specific types of clothing, you would use a data set that was specifically designed for that. They could look at an image of a person and say they're wearing a double breasted suit and a winter tie or whatever. Microsoft ability initiative. It says accessibility because I always get that wrong, but it's the ability initiative. And we've got links to all of these things in this slideshow which we're going to publish if you're interested. The Microsoft ability initiative is they are working to produce a data set that is designed specifically to give good captions for screen reader users. So, you know, sort of looking at what types of information tends to be missing from AI and AI generated captions. And also they're looking into AI. You know, an AI caption can be more than just text. It can be multi-dimensional. It could include multimedia. So there's a lot of different things that AI could potentially offer for visually impaired users, which is pretty interesting. And I think it's going to provide great information for anyone that wants to create their own data set on how to create captions that are good for screen reader users. Why not use AI captions? So this is just a caveat saying when you're using them, be careful, because sometimes you are going to see unexpected results. Microsoft published a tool called Caption Bot.   You know, for example, this one is like a picture of the first lunar landing, and the caption says, I'm not really confident, but I think it's a man standing on top of a dirt field, which is kind of funny. It also illustrates one of the challenges with AI captioning, which is that a caption ideally should communicate the intent of the author. And AI captions, you know, that is something that, you know, while they're improving, they're not quite there yet. You know, one of the improvements that is being looked at for AI captioning is that, you know, in addition to the actual image, the AI will be able to potentially look at the surrounding text in an article to try and get a better idea of the context. But that's not quite there yet. So, yeah. So, in conclusion, user responsibly implemented as an assistive tool rather than as a default. So, you know, review the captions. Make sure they're good. If you are using captions that potentially are not reviewed, make that known to visually impaired users that there is a percentage of error.  And so prompt content authors to review the questions also. If you would like to, so I'm about to hand it over to Boben. I see there is one question. Should we answer that now or? Absolutely. Okay. I was wondering if you have any... Can you step up to that? Sorry. I was wondering if you have any tips on how to communicate it to the visually impaired user. Like, would you include it in the LTCH or do you have a general message on top of the page? You know, I think that it could be communicated in different ways. I, you know, one proposal I had that is that, you know, you could always include that information in the LTCH, and then if they are being reviewed, you could just remove that information from the LTCH, which would be a good way to do it. On the other hand, if they're about to look at an archive of images that are not reviewed, you could put that information at the start. So, yeah. All right. Over to Boben, live demo time. Thank you, Laura. Hi, everyone. My name is Boben. You'll have to, first of all, forgive me. This is my first time doing any presentations at DrupalCon. So I have to set a timer so that I can constrict myself, not talk too much about such a complex theme, such as using AI, especially in Drupal projects. So Laura gave a very good intro in how and why we might have a purpose in using AI models and detecting captions from a range of different images in our websites. And at my planet, a lot of the sites that we had worked on in the past had such a need. So instead of bugging content moderators, entering different alt tags for different images, we had sites with a fairly large amount of images, and it was important that each of them had an LTCH for visually impaired users. So as the AI technology progressed, we explored various different technologies that were offered. As Laura already mentioned, IBM technology is just one out of many technologies that we can use to train a machine to detect something from the image. And for the purpose of this demo, we used IBM because it was, I wouldn't say the simplest one, but it tends to serve as a proof of concept into how you can go ahead and train your own models to detect various different images. I'll go back to this point a little bit at the end once we show the live demo and go through a bit more into how we implemented this. Before we dive really into the code, I just want to say a little bit of history on the demo. Before we started implementing this solution, we needed to ask ourselves two questions. So the first one would be how to do this in Drupal. The other one would be what to use as a server that would give us information on what is showing on the image. So on the Drupal end, we decided that we just simply use the regular module and library approach. The library will be used to initiate API calls towards the whatever endpoint that will give us the results. The module will be there to give you administration over the API usage. One of those usage in this case is definition of the base URL that you can use to ping the model to give you the results. Again, this is an important point that I will stress out at the end and you'll see why it's needed. So we had the model library model implemented on Drupal end and we decided to go with the IBM one to get results on image captions. Let me show you. What we did is this is of course Drupal 8 site and since the field API has been largely upgraded in Drupal 8, for us it was extremely easy to implement enhancements over the existing fields that exist. In our case, we used the image field which is part of the core and we enhanced it so that we can add additional configuration options when you're extending the field in any of your content types and also we're extending the field widget functionality that determines how the field is represented on our coreless non-creation section. So we extended only two base classes, the field type for configuration options, field widget for display. Of course, you can extend the module itself to include the field formator options. So, you know, show the representation of the image in the way you want but since the alt tag is part of the natural render of the image field, there was really no other need for us to do anything additionally. How does it work? I know you're all excited to see this. Oops. At least it's not a white screen of death. But here we are trying not to touch anything else. So, we have the image field. You can select any image, I've numbered them, so let's select the image, it gets uploaded. What you get is additional options besides the alt tag which is by default available. So, we added this field set here with the single button. So, if we hit this, it contacts, it does all the stuff in the background that we'll pretty much discuss about soon enough and as a result it gives you this additional select field. Now, every model, of course it doesn't work 100% cases. So, even in those cases if it gives you a good caption, it still doesn't give you one result. It gives you at least couple. It all depends on the model. In this case we're using the select option because we have a number of different options. So, as you can see, I'm hoping everyone can see that I have an image of a mountain and a cloud. So, all three results here that are showing relate, well, pretty much describe what we are trying to set in the alt tags. It's not very literal. It's not, you know, something that a poet might write, but it's still a very good representation of what the image is all about. Now, you'll also notice, of course, the sentence itself does not have the first letter capital. I mean, it's something that you guys as a group of developers can do additional formatting in order to show, but the important stuff is at the end. So, you see a very large point number at the end which represents, by model opinion, the probability that whatever is offering to you is the correct definition of the image. So, it's a very small point number because, like I said, it's not an exact representation of the image, but it's something that it sees from its own, presumably large, database of images. So, this option, the probability what the IBM Max Image Generator calls can be, you know, when you add it, it automatically populates the alt tag field. This is an optional value. So, when you're setting the field configuration, you can just simply, we added this configuration option to either use it or not. If you disable it, then let's see. And let's add additional image. Again, let's generate the alt tag. And, again, let me see if I can just quickly show. So, it's a train on an AC site, which helps your resource. And the definition that we get from the IBM model is pretty good. It's pretty accurate. And, again, we get the probability showing if we want, we can just delete it. The good thing is that the alt field itself, whatever you select here, it's still editable. So, you can just simply delete and add any whatever text I want to change it into. So, that's one additional step that it's optional for content admins to do in case they want to additionally format or change whatever the suggestion was from IBM. This is pretty much it. It's a small demo, but a lot of things happen in the back end. So, let me just check my time. I still have it. Good. So, I'll quickly switch to the code itself. So, how does this work? There are numerous steps. Once we load the page, it happens when we load the page and to the point when we get the slackbox with the available options. So, starting with this, what we have is, like we mentioned, so we extended the field type class. We added the regular, we used the regular form API in Drupal Lake to add additional field that is going to show the slackbox, the button that triggers everything else. And then we also have the field... Sorry, this is the field widget. The field type simply adds this checkbox on the field configuration page. The field widget. Sorry. Oh, sorry. Hope it's better now. Thank you. So, yeah. This is the checkbox. Oh, no. This is the field widget. So, it shows the button. It's not a complete solution because the field widget class also depends on some JavaScript work that we have because everything that includes executing the API code itself is done through AJAX. So, in the end, I think the last slide contains all the links including the link to the repository where the custom module, the library that we did is available and we, of course, encourage you to all clone it and expand it and enhance it in any way you want. Okay. So, what's else? So, you get the field widget displaying the button. You upload the image, the button displays. You click on Generate the tag. What happens next? There is a small JavaScript code that we added which is, again, part of the custom module itself. I don't want to zoom in right now because it's a difficult code. It's a regular jQuery with a little bit of JavaScript which is taking whatever the image ID is and executes an AJAX call. It's relatively straightforward. The point in all this is that it's pinging a Drupal control that we created. So, you can have two things here. You can either directly ping the IBM model or you can just simply have something called Drupal backend that will then call IBM system, get the data parsed in any way you want. We chose the latter method because it gives us greater control. So, Drupal backend pings the IBM model, gets the recorded data, and then we can parse it, capitalize the first letter, remove the probability or add it instead of putting everything on the client inside. Whatever method you choose is up to you but this is what we currently did. So, now we are pinging, in our case, the Drupal backend. There's a control that we created which takes the image ID and then collects all of the necessary data and pings the IBM max image caption generated. It's a very long title for it. So, I'll use IBM for short for now on. What happens there? That's the most interesting part and this is the part that Laura was talking about. So, the image is processed. It goes through the encoder, the convolutional neural network, first of all. It's split into many different details. So, in general terms, the image is split into whatever the system can process individually. So, we had a mountain with a sky above it, maybe a little river on it. The convolutional neural network tries to split all of those terms in the penalty. So, it splits the image into smaller pieces and runs through its extremely large database. I think the IBM model has around 500,000 images. They're not very diverse, but they are mostly related to, I would say, nature images. So, that's why, in our case, we got pretty good results and descriptions of the images. So, once it goes through the encoder, the encoder tries to put all those independent pieces together and as a result, you get a couple of different sentences. So, the pairing of all those different layers that we got from the encoder could give us a lot of different results. That's why you also get a couple of different options all with a different probability. The larger probability, the better description of the image is. Of course, the most important thing in all of this is the database itself. Like I said, IBM currently has 500. I think they're continuing work on this, but since this model is a proof of concept for them, they have a whole other slew of other models that they are trained for different purposes. In any case, back to our code, the decoder finalizes everything, puts the different sentences, and puts it into a JSON output that the Drupal control now receives. Drupal, like I said previously, we parse it, we do with it whatever we want, and output the JSON again. The good thing is, you can use the JSON API, the new module that will be shipping soon, and I think it's already available. And you give it to the JavaScript. As a result, what you get is... Switch. Where's the browser? Okay, if you can drag it while I'm tweaking. So what we get as a result is a slack box with options that you can use to... It's pretty sensitive. But a lot of different options. There's one more demo that I'd like to show you, and it's based on what I've said earlier that why the models are important to be trained well enough. The IBM model is mostly about nature, so it's not really good at recognizing something else. Machines won't be... Don't believe me on my word, but machines won't be still ready for at least another 10 years to detect everything that you put at it. It still means a lot of storage and a lot of data in order to understand everything. So I found an image that's going to be interesting for the algorithm system to try and describe. So here we have a close-up of a monkey's face. So let's get available options. So how do we frequency it? What the model sees is a large elephant with trees in the background. The first one, if you can see, the third option is my favorite, so it's completely off point. It doesn't mean that the system doesn't work. It's simply a matter of how well you train it. So initially when we started this proof of concept project is that we asked ourselves, do we really need to create our own model to train it for our special purposes? Or can we just use IBM and hope an image that you throw at it will work fine? Well, for proof of concept at least, the answer was pretty simple. It's important for you because Drupal is heading, and it already is, heavily invested in media. So you guys are doing media sites with a lot of images, not only images in articles, but you're also representing the hero banners, static images that don't represent perhaps valuable information, but they do for visual impaired, maybe they even do represent information. So it's important that you use as much information as you can, not only for the visual users that you can see, but also visual impaired users. The important thing with this is you have a website that's doing something related to recipes, related to food. It would be much more preferable if you create your own model, train it with a whole set of images related to food and recipes so that something like this works in your advantage. So you would create it once constantly feed it with the new images and the model will work for you. I mean, it's something that you can see benefits in the long run, of course. So that's one of the ways how the technology will also progress because the model itself can then move on to its parent. And the different learning models can be joined together. So instead of just seeing this as an elephant, the same model can get the data from another model and then describe the image properly. I mean, I expect the next 10 years will be pretty interesting because the models will grow in size, not only by Google, IBM, Microsoft, and Facebook, but the rest of the community including Drupal. So that's about it. Thank you. Of course, we have time for questions. Yeah, please take the mic. You demonstrated this image caption one by one. Is it also possible to generate captions for a large archive of image fields in an existing website? Absolutely. And this was my first point, how we decided to implement the project. So you have a library that's simply there to initiate API calls. The biggest problem is that one API call per image can be made to the IBM.  The IBM one only allows one image per API call. The way you implement, since we're talking about the image field for this demo, if it's a multiple value field, you'll have one generate alt button per uploaded image. So click on it twice to get alt tags for two images. That's about it. This is a proof of concept. Everything that Drupal has, you can use so you can extend it any way you want, but it is possible. Absolutely. Okay. So a quick question. IPTC data. So most of the large-scale image banks or agencies such as Associated Press convey information using IPTC data, which is kind of lacking. It doesn't have any support for linguistics or anything. Have you tried using IPTC data for the machine learning calibration or anything? Yes, Laura can do this because she's more in the central since I'm from Serbia, I do that have a different part but I'm guessing no, right? Yeah, no, we haven't as far as I know. So again, this is a proof of concept and it'll be interesting to offer this. Here's an idea for you. If you have a company, if you work at a company, you can supply this to one of your existing or future clients. What type of costs are connected with calling the IBM service to get the caption and what are the approximate costs for getting alternative image sets to train on? Like the 500,000 images plus a description isn't something we can really build up by ourselves. So I have to get it somewhere. Yeah, do you want to take this? Yeah, so I know that some data sets are publicly available. The one that was used to train this show and tell model is publicly available and I know that there are some large ones that are. I know that the ability initiative is going to make theirs public as far as I know. So yeah, there are quite a few that are. As far as, you know, ones that are charging, I'm not sure what the cost is. But yeah, it would be interesting to see that's probably going to be something that's monetized more and more. So it would be interesting to see what it costs. Okay, and the IBM service? Is it currently free or? This is for the go one. Yeah. For evaluation purposes probably. Yeah, I think that it may only allow a certain number in a given amount of time. Yeah. So the way they offer it is, of course, you want to use it for commercial purposes, it's ever free. For, you know, not making, I think more than 100 API costs per day is again free. But, you know, with IBM, it's always contact us for more information. So based on your needs, you know, you should always contact them directly and, you know, specify this. The Amazon also, I think it's worth mentioning that the Coursera has a free course on the Amazon AWS model training. So AWS also, as you already know, whenever they are used, that's how they charge it. So sitting, if you have a website there, it's not doing nothing. You're not being charged with anything, but they have an extremely valuable course on machine learning, especially in related to their services. And I think one of the information that you'll find there might be how they value this. In any case, this is technology, emerging technology. So it's definitely not free, especially if you consider half a million images, quality images, you know, 2000 by 2000 resolution, they take up a lot of space. And we're not talking about space, only we're talking about processing power, because it needs to, it's not going once to the same image. It's going a few times because the convolutional, the neural network and the RNN and CNN, the way they work, they need to have multiple layers. So the image is not spliced once, but it's recombined and spliced in another way. It's an extremely time-consuming process, so we still don't have that heavy processing power. Yeah, I think to train a model with, you know, many, many images and captions can take, you know, a couple of weeks. Are the captions always delivered in English? Any ideas for translations, for liquid sites? Yeah. So with everything, even the, you know, the PHP, the Python, everything else is in English. So everything they do, they do in English, first of all. But yeah, this proof of concept, it's only available for English, the IBM one. But you do have other languages that are supported and I'm guessing they are not free at all. Don't exit anyone, but any like available, you can think of, but Dutch certainly might be available for the same, you know. It's charged. One last question or remark. I think it might be good if, I mean, I'm not sure if it's possible, but it's the alt text. So say we use this system and I correct the alt text because, well, there's a gorilla and an elephant. Is that information being sent away to IBM to train the system or are there other providers who do that?  It's the same thing here. And again, I'm guessing we're living in the age of information so anything that you can supply it with, it'll use to train. So it's not doing anything bad with it. It's using it to train your data. Again, it's never written anything so it can be completely written. And I think that if you train your own model, you could definitely set up, you know, if you were using a system like this within CMS, you could set it up so that it did send the result back so that it was taking whatever the users are entering and adding additional information for that image. All right. Well, thank you, everyone, for coming up.


brief slide deck. And I wanted to say thanks to all the organizers for the quick reorganization. I think this has really worked quite well. I've learned a lot about some of the areas of emerge beyond discovery that I was a lot less informed about. I think that there's an enormous amount that the VA can learn. We are developing our big data strategy in the VA which crosses both research and clinical care. You're having built a genomic research infrastructure within the health care system. So a lot of the things that I've heard today are going to be very applicable. We are approaching a 250,000 individuals in MVP and as Neil had mentioned earlier in the day, about 220,000 of those will be genotype using an array very similar to the with some minor modifications similar to the global array or the African-American array that Neil had developed. And then figuring out how to take that knowledge and bring it back to our clinicians. That's always our primary mission in the VA. So I think that that's much what we talked about. It's been very applicable and we have those 250 are consented. It's a trust model that will allow for some evolution and we built in some of the things that we've heard earlier today like the ability to recontact and actually survey individuals and potentially opt in or out of return of information as the field evolves. But there's an awful lot to moving that into the implementation range even within one health care system. So if I can have my next slide, I think that some of the issues that we've heard today are echoing themes and I'm going to be very brief. There's various levels of institutional buy-in and it sounds like you all have gotten institutional buy-in. One thing I've learned about buy-in when we move knowledge from the research side of the health to the clinical side of the health within the VA or within partners, which is the system that I also live in here in Boston, is that there's buy-in at various levels. And I think we have to think about it. We've talked about it today. There's overall buy-in at the top. Then there's sort of clinical leadership buy-in. There's buy-in from the informatists who have to help us with the implementation. And that's one issue that we have struggled with in the VA, particularly because this can be sometimes resource intensive and we may not be able to pay for that implementation aspect. Then there's buy-in at the user end, the clinician end. And so certainly understanding their needs, understanding how they utilize things, understanding their stressors, understanding the fact that they get many reminders, alerts, and other realms of clinical decision support are often in play and we may be competing with other aspects of clinical decision. And then buy-in at the patient level to the extent that these things are accessible in a patient portal. Number two is system heterogeneity. And we've talked a lot about that today. And I don't think we need to go into great detail. I think that certain of the vendors are less amenable to modifying their platform. And so we talked about one solution today, which is to make sure that the institutional leaders that are buying from these vendors speak with a unified voice to try to get, to try to pressure certain vendors, particularly the big ones, that are rolling out now in really cookie-cutter fashion EMRs that are going to be laid out over. And one thing that is being dealt with in partners is that there is a general sense that the modifiability of some of these, what I'm going to say is big EMR vendors is perhaps not what some of the local implementers would like to see. Then how customizable are these and how that could impact the aggregate experience if we want to learn from that in aggregate. If we end up with customized systems where the information, whether it be process outcomes or not, are useful locally. Will that translate across the systems? And how does this compete in terms of appearance and other things with other clinical decision support? The nice thing is that there is, in most institutions, individuals who in the non-genomic world have been involved in this clinical decision support arena for quite some time. I think I might last slide and we'll just finish up and then turn things over to some of the other panelists. So I think that it's important that my only comment here is, and we can discuss in general in the next steps, is moving from process-based outcomes which are easier to measure. You can measure how many times a test was ordered or how many times a report was read, how much information was utilized in a given clinical situation. A little bit harder is then turning toward clinical outcomes. Does this really do what the health system wants to do, make patients healthier? Other system outcomes, such as improving efficiency of care, often put in the context of the health of the individual. And what that means is that we have to phenotype individuals on the back end and phenotype in real time if we want to look at who has less bad outcomes when they've utilized a clinical decision support tool. So I would talk about a lot of this today and there's a lot to think about. And I think it's time to turn to the other panel members. And since they're listed in alphabetical order, I'll suggest we go Erwin, Ken, and Lucila if that's okay. If Erwin is warming up, maybe we can just turn to Ken. I know Ken's on because I see him chatting. Ken sent me a message saying he is muted by the organizer. So, Brandy, could you unmute Ken? Oh, can you hear me? That's it. Great. No, I think everyone's points were really good. But you just have a few comments. So one aspect is, I think, overall, just from a high level view, I think it's important to do, to make CDS for pharmacogenomics and for genomics in general, that's active from an operational standpoint. So my day job now is more operational. I'm in associated CMIO at my institution now. And you sort of see what sort of how the operations and the health system can look at some of these decisions more. And I think there it's really important to think of how can you make this so that the 10 or 20 or 30 second elevator pitch to the hospital admin makes this more attractive. And I think there's probably two components of it. One is to increase the value. So how can this solve a problem that is near and dear and at the top of an organization's operational priorities? And I think we talked about process and needing to go to clinical outcomes. I think also going to cost effectiveness is important. Like, we have a paper that's pending publication in DMC Medical Informatics where we systematically reviewed clinical distance for for them, patient setting, looking for cost effectiveness. And essentially we found there's almost no data. And I think, you know, if you're making a case to clinical administrators why we should invest in pharmacogenomics for distance support in an operational sense in terms of going widely, what you need to do is show that this can be cost-effective in what you're looking at. So I think that's important. And then the other part that folks have been talking about in terms of using standards is that I think that really needs to be decreasing cost. And I think it simply is hard, like the CS Consortium example is a good one, it's hard to do a lot of these things. And if you couple, we don't yet have cost effectiveness data on the impact of these kind of efforts on the bottom line. And also we don't have, and more of the cost of implementation is relatively high and is a relatively big demand on, say, your hospital IT group. I think that's a challenge. So I think sort of from a high level view, looking to show cost effectiveness and looking to come up with approaches, whether it's using standards, services, et cetera, or using available tools in the commercial EHR system without too much need for adjustment, figure out how to decrease the cost of implementation I think would be important. And I think when you have research funded through something like Emerge where you really can, you know, take the time to really figure out how to make it scale, how to, you know, research cost effectiveness, I think that could be a really great outcome if you can, one of the outcomes is we've shown and we have very easily available now information you can provide to your IT group to say to implement this, it only takes these steps and requires 20 hours from this kind of person and you're done. And moreover, we've shown that in this type of a setting, this is going to reduce your cost, whether in, you know, a capitated or an ACO type of setting. I think if you can do that, then it'll sell itself and it'll disseminate widely. Yeah, I would second that that's critical. I mean, as these, you know, big health systems are moving toward, you know, global risk, there are going to be a lot of pressure to use their informaticists to do well proven clinical decision support that's going to improve the health and lower cost improve efficiency of the healthcare system. So we're going to, you know, whatever comes along in this space is going to be competing with that activity. And if you can demonstrate some efficiency, I think that that's a real advantage to getting the attention and resources that you need to get it implemented. So I think that that Erwin was neutered and he's back on now. So Erwin, do you have comments? Yeah, I agree with most all things that have been said. I want to add a few points to that. And one is when it comes to EHR integration and clinical decision support, I think there's broad agreement across the network and beyond on this panel today that clinical decision support is sort of the main vehicle to focus on. And when it comes to clinical decision support, it's a very difficult undertaking, as Mark pointed out in his opening remarks. I think going forward, what we might want to consider more than what we have to date is to actually bring some selected practice leaders, clinical practice leaders into the fold, because one of the key elements to successful clinical decision support, and then certainly also downstream, you know, collecting and measuring meaningful data is that we have a decision support that really integrates exceedingly well with clinical workflows. And so we have now done this in the form of code genetics implementation sphere that was outlined earlier for emergent beyond here at Mount Sinai, but also in with the April one clinical decision support is one of our genomic medicine demonstration projects. And we found that even within one health system, such as Mount Sinai, and having 130 physicians involved in our programs, various programs, these are a number of different practices and practice types that, you know, each group of physicians in the different practice types, whether it's private practice style, faculty practice, or whether it's the clinic, or whether it's a community network clinic, each of those practices has a very different perception of the decision support that is the same text that we presented to them. So I think, you know, going forward, certainly this is sort of probably one of the main vehicles for us to play in implementation. I think what we might want to consider bringing more into this focus is sort of a direct reach into those folks that are sort of opinion leaders, etc., in different clinical practice settings, and that will then obviously be driven from email three perspective from by the point of what what kind of areas we are focusing on or what might want to focus on for implementation. But I think that is something that might be from a very practical perspective going forward. Very helpful in a comprehensive approach to a focus on clinical decision support and having it optimized for clinical adoption and then being able to make meaningful measurements from the implementation of that. Great. Thanks. And Lucila? Yeah, well, I agree with all my colleagues. And I would like also to point out that the modifications to the EHR that clinical decision support needs to make are not that different from the ones that researchers need to make, for example, to implement trials to facilitate recruitment and so on. So there is a lot of need from the research as well as the clinical communities. The clinical community obviously will have much more of an impact in asking for such things. But I think a unified front would be of much influence because when you're selecting eligibility criteria and so on, it's not that different in terms of research or in terms of recommending the right thing to do for a particular patient. One thing that I think is no surprise that pediatrics and oncology have done more in this area, I believe is because of clinicians are more educated about genomic medicine in many cases, as well as the patients being more informed or their parents being more seeking of such opportunities. I think the opportunity to leverage PCORnet, which is newly formed and just had its kickoff meeting this week is tremendous. And the lessons learned from EMARGE could definitely pass on. So PCORnet doesn't make the mistakes or go in routes that you've already shown that won't work. And hopefully vice versa as this clinical and patient power research networks proceed. So I think I'm very pleased to hear that we're moving in the right directions. Great. Thank you. And I think now it's the time for our open forum on this topic. If anyone else has anything to add? This is Eta Berner. I was going to add picking up on what Erwin said is that we tend to talk about clinical decision support or EHR use has to fit into the clinical workflow as if it is the clinical workflow. And in fact, there are many, many different workflows often individual physicians within a given clinic. And we need to take that into account. Right, I think that's important. And I completely agree that having clinician buy in and all clinicians are not the same is critical as the clinical decisions report tools that are being developed. Other thoughts or comments? This is Peggy Pysig from Marshfield Clinic. And I just wanted to comment on something that Ken said. Cost effectiveness is definitely one way of selling this to, you know, operations. But the way that Marshfield has done it in addition, well, we haven't done it by cost effectiveness, but we've taken it actually under patient safety and sold it that way. So that's another approach that may be worthwhile in investigating. Yeah, in addition, hitting quality metrics. I mean, if you're meeting standards, meeting guidelines, it's another, another important consideration. Yeah, this is Mark. I wanted to make two points related to what Peggy said that are, I think that's really important. We have a couple of opportunities within what we've currently we're currently doing to take a patient safety approach. One relates to something that came up briefly on today's webinar, but was really a focal point for the global medicine meeting that many of us were at last week, which is a prevention of Stevens-Johnson syndrome and toxic epidermal necrolysis. While these are relatively rare events, they incur huge cost to the system. And at least for some populations, there's a big opportunity for prevention. So that would be one example of a major problem that could be presented from a patient safety perspective. The other one is if we can figure out the genotyping issues on CYP2D6, hospitals are being beaten about the head and neck on pain control for patients, but also on safe use of opiates. And I think we have something to contribute in that discussion that could be helpful as we sort of ping pong between those two extremes. And that would be something that would be attention getting for the leadership. The other point relating to some of the economic discussions before, you know, when we tend to think about economics, we tend to think about, you know, looking at things from a national perspective and qualities and a very academic approach. What we're really talking about here are economic studies from the perspective of the healthcare delivery system itself, which is a very different methodology and is something that there are very few groups that are really doing, but I think we would have an opportunity to take advantage of what we're doing to look at it from a very different perspective, which would contribute to the conversation about why this should be implemented. Yeah, I completely agree. It will become, you know, as we move to these risk contracts, the economics of the health system that is going to drive, you know, what their EMR looks like and what they're just directing their positions to be doing. So I'd like to respond to Mark talking about the PIP-2D6 and opiates. That situation might be so complicated, and it's so complicated even on the genomic end that an assay that measured how that metabolism happens on a practical level for a patient might be the only way to really get to the bottom of it. It looks, in our superficial way of looking at how complex the genomics are related to how the metabolism actually happens is not that strongly predictive, and hence a more practical approach that's non-genomics might actually be something that would be more useful in terms of the clinical application. And just to say, you know, the genomics is not necessarily the end-all for the way we solve all the clinical problems. This is Mary. I wanted to, well, I guess first of all say, I do think that for PIP-2D6 one can easily identify the 10 percent of the population that are poor metabolizers. I know there's a lot of confusion for some other findings, but the vast majority of 2D6 high-risk patients can be identified with relatively simple tests, and that's allowed our hospital to leave codeine on the formulary for patients that have permissive genotypes, which has really helped with the opiate prescribing problem that you were referring to. But one thing I was thinking as you're talking about using safety instead of cost effectiveness to be the primary driver behind the CDS initiatives, which is also what we've done at our institution, would you emerge institutions be able to summarize what the majority of your interruptive CDS is currently and what the added burden of CDS related to genomic testing will be? I mean we found out as part of our genomic implementation that we needed to do a better job of tracking what all the interruptive CDS is that we have ongoing, but the vast majority of it that's customized is patient safety related and we find that our institution is very willing to support efforts to create that CDS, whether it's genomically based or otherwise. Yeah, Mary, that's a really good point because the whole concept of alert fatigue is one we're well aware of. I think though some of it also relates to not only the current burden within our institutions, but are there other ways to do it rather than an interruptive alert? You know the example for Simvastatin, you know in my view the the you know if you went into a statin prescription and you had the star 5 genotype, Simvastatin wouldn't appear on the pick list and the only time you'd get an alert would be as if you tried to type in Simvastatin and then would say hey there's a reason we didn't show you this on the pick list and so I think there are different ways that we might be able to do this without interrupting workflow and look at that as our only solution. That's true but there's a lot of genomic CDS that will be dose-related not choice of drug related so you're still going to have to have alerts for a lot of genomic findings in pharmacogenomics. Well not necessarily because when we did our warfarin studies at Intermountain what we did was the process for dosing warfarin using clinical features was already in the hands of the pharmacologists and so the clinicians were just seeing a dose and so when we did our randomized control trial it was simple for the pharmacologist to use the genomic dosing algorithm along with the clinical features and again the clinicians just saw the dose so it made absolutely no difference to them the fact that we incorporated genomics in that particular type of a workflow so I think studying workflow design and not just always defaulting to an alert is an important thing. Mark could I ask a different question at Susan? Most of what you all have presented about is CDS and related issues once information is already in the EHR but I wonder if you guys could back up and link more explicitly to the issues just before your panel because you know in genomics and in a merge you've got to worry about this translational process of how do you structure the flow of information into the EHR when is it ready for that move how is that governed I wonder if any of you could reflect on that for a second. Well certainly at Geisinger we dealt with those issues relating to our implementation of IL-28B genotyping for our patients with chronic hepatitis C in terms of treatment regimens and so as we decided clinically that this was desirable and that this information would be usable we had to solve all the different problems which is first of all how do we make sure the genotype gets done because we do not have preemptive genotyping for those individuals and we use a standardized order set that included genotyping to reduce the barriers to trying to remind clinicians to actually order it and then the issue was once it was ordered and we did the genotyping which we were doing in-house we had to develop a manual process to be able to represent the IL-28B genotype as well as the viral genotype as structured data within the EHR so the medication recommendations would be able to run and so we developed a manual solution to that and we now have you know a process that's essentially running in a hundred percent compliance but that's a one off and you know so we have to get over the fact that to solve some of these problems rather than keep building one off. And I wonder how that looks across all the immerged sites it seems to me there's a comparative opportunity there because this is a core question that has to be solved in a more standardized way. So we are looking at that in the that's part of our EHRI outcomes measurements is to be able to look at how people are actually doing some of those things and the hope is that as we aggregate that information that will provide us some opportunities for standardization but as been mentioned the challenge that we have is still ultimately we're dealing with vendors and whether or not they would be amenable to modifying their product to represent what we think it's the best way to do it. So just a reminder we have about one more minute on this topic. This is Ken maybe if I could do a quick comment on that notion of what to do scalable. I think first just one point is vendors actually oftentimes do provide extension points and I know some folks have already exploited those and I think one approach is to just say what's currently supported and let's figure out a way to be able to scale on that and another point is just that this notion of scaling CDS is something that of general interest to the ONC and it's been a priority for for example for development for meaningful use stage 3 proposed criteria. I coordinated one of those efforts called healthy decisions to develop some of these standards and there's still a lot of interest at ONC, CMS, etc. to you know develop these standards to make them scalable to make to solve this problem essentially for all of medicine not just personalized medicine and I think doing whatever is possible to align with that would be a great thing in particular because the idea is then this these would be industry standards maybe part of meaningful use and therefore implemented in all each our systems etc and just aligning forces with these efforts that are really trying to solve the same problem just not necessarily focused on pharmacogenomics. Okay well we'll take this pause in what has been a very good discussion and sorry Justin we're going to move on down down to


The moment you identify with what you know, you become a speck. But that speck is a super speck, because it's your speck. In this cosmos, this planetary system, this solar system as we call it, itself is a speck, isn't it? In that speck, planet earth is a micro speck. In that micro speck, Toronto, Toronto is a super micro speck. In that, you are a big man. This is a very serious problem. This is a very immense problem. In a super, super, super micro speck, you are a big man. You need treatment, yes or no?