business in Hawaii with Reg Baker. We broadcast live every Thursday from two to two thirty in the downtown high tech studios of Think Tech Hawaii in the Pioneer Plaza. We are a story or we are a show that focuses on success stories in Hawaii of businesses and their owners and also organizations that support the small business community and help it to thrive. Today we've got two very important guests from the Hawaii Small Business Regulatory Review Board who is responsible to oversee and advise the governor on different rules and regulations that are being proposed within the state and I'll let our guests talk a little bit more about that but we've got Tony Borsch who's the chairman of the Small Business Regulatory Review Board and then we've got Kioto Kimura who is the vice chair. So welcome your returning guests it's great to have you come back. Thank you. Why don't we go ahead and just start out with just an introduction. What is the Hawaii Small Business Regulatory Review Board? Small Business Regulatory Review Board is a volunteer board and made up of basically nine board members and the board members are I guess appointed two by the governor, three from the Speaker of the House and three from the Senate President and pretty much what we do is we advise as an advisory board we advise the governor on rules and regs that may impact or have a negative impact on small business as well as you know take except petitions from small businesses that have maybe experiencing a negative impact on their business on some of the rules that are in existence so it's twofold but basically as an advisory board to the governor. Right it's an advisory capacity but the small businesses if they encounter something that may be particularly challenging can they come to you and and kind of ask for your help or thoughts on the matter? Of course and and at least one board member is from each county and I'm from Maui so the neighbor island small business can come to their board member on their own. So they don't have to come to Oahu then they can do this on Maui or the island or whatever and and if they want to do that how do they find you and reach out to you and get the information so they can make an appointment or talk to you? Well we normally have our meetings the every month monthly meetings it's a third Wednesday of the month and it's held at the d-bed on the 4th floor of the building that's right on the hotel street. Right that's actually the Hawaii culture and the arts. The old YMCA? Yeah yeah right next to it yeah right next to it. That was the old Chris Hemeter. Yeah the Chris Hemeter took the old YMCA and rebuilt it. But yeah so the third Wednesday of the month we have meetings there and you know it's usually we get started at 10 o'clock it's posted a week ahead on the lieutenant governor's website so you can get the information there you can go to the d-bed website as well and click on business resources and I believe for business resources and the small business regulatory review board will appear on that as well as the meetings and you know agenda. And is there a capability to send an email for for example if somebody wanted they were on Maui and they wanted to reach out to Kyoko and ask to speak to her and talk to about an issue would they be able to do that through the website? They can contact through the email from the website and and somebody is going to contact back to you where you can reach me. Good good good all right because I would imagine that there would be people that would be interested from the neighboring islands and are not having to travel to the San Oahu. All right very good and so you meet the third Wednesday of every month. Is that open to the public? Yes it is. Okay very good open to the public and more yeah it's it's you know we're under the sunshine. Good so no secrets. In fact we encourage everybody come down you know if we don't have enough room they're standing room only that's fine good and we normally have water for everybody. All right very good and Tony you chair this and you run this and and you've got your board there from the different islands and I guess what happens is that people actually have an opportunity to come up and and present or talk. Do they have to be on the agenda in order to talk or can they just show up and ask to speak? Well you should be you know you should be on the agenda. If you got a specific business to address you should be on the agenda. We do take you know comments or or hear people who are there but normally it's got to pertain to what's already on the agenda. Right yeah the proper way part of the rules and regs yeah you should be on the agenda. Yeah it's always better. Yeah and it's pretty easy to for a business to to get a hold of the the Small Business Regulatory Review Board either with the website and or you know telephone but it's we strive to make it a lot easier. I mean we can do it via the the internet via the website and you can do it via fax as well so you know we add telephones. What's what's fax? We need to publish one week before the seven days before the meeting the agenda so it's if somebody wants to something particular to be discussed at the meeting needs to contact us seven days prior. Good okay that's a good and finding the website is easy I mean sometimes you only have to do is just google d-bed or you know and it pops right yeah and Small Business okay all right so it makes it easy to find what are some of the issues that you address I mean there's there's I guess a lot going on these days I guess the legislature just kind of ended they might have a special session coming up a lot of controversial possibilities there. Any amendment of any rules or the new rules needs is mandated to come through us. Wow that's a big job. Yes so it's from agricultural regulation to license issues to workers comp it's it's all over. Medical marijuana too? It came wow yeah and any agency we're getting you know promulgating new rules and regs or changing revisions would have to come before the Small Business Regulatory Board for for our input as far as you know affecting if it affects Small Business so I guess what the agencies do they we have a form that they would fill out and kind of walks them through to see if the rules and regs that they're they're they're putting forth does impact Small Business if it does then it comes to us. All right was that form also be available on the website? Yes it is. All right so there seems to be a lot of resources there they could check into and download the form you get a better feel for you know how the whole process works. Very good now I've been to some of your meetings and some of them are pretty active there's a lot going on there's a lot of discussion and I've seen people lining up in the whole way to talk you know is that normal is it is it usually pretty busy? Depends on the agenda we hear sometimes about CQ Combo Regulation then that very small group is interested in versus if the workers comp rates are going to go up and so many people will come. I remember one time I was there and they were talking about DCCA increasing their fees and that seemed to be a pretty popular topic and there was a number of people there to provide testimony on that. Definitely I mean you know it's great though that we get that kind of input especially from the Small Businesses I mean that's what we're there for so if we can encourage more of that and you know that would be the greatest thing I mean to be heard and yeah we're an advisory committee but you know I still think it's worth getting the input from those that would be affected the Small Business because Small Business is such a big part of the state's economy that you know I mean it's a great economic engine generating you know tax revenue and whatnot from law abiding small businesses right well of course yeah but yeah so it's you know to me I I we welcome that that's great mix mix up we feel good in what we do you know it's all volunteer board so we don't get paid you know Keoko comes over from the native islands it's that's that that's the whole day same thing with the other members coming over from the big island right now we don't have a member from Kauai but but yeah it's it's that and then also you know prepping for the the meetings and all that stuff so it it's voluntary but we do it because you know we believe that Small Business there's a lot to be done for Small Business make it easier you know it's it's coincidental that you mentioned that but in the next issue of the Hawaii Business Magazine I wrote a column about how important a Small Business is to the Hawaiian community and it represents over 50 percent of employment and domestic product in Hawaii is 50 percent you know you've got to have a strong thriving small business community if you didn't have that our economy here in Hawaii would collapse well definitely we agree however they most of the small businesses don't have time to pay attention to those changes even if so that's what where we want to represent them right and it's nice that you've got representation on each island it makes it a lot more convenient for them to be able to get their voice heard and also the the board members you know that that that are pointed pretty much come from different fields so you know we the the idea is to have you know somebody from the the tourism retail construction you know and a good finance so a good cross section so we can really represent you know the the small business community yeah no it's important there's I guess some and we can probably come back to this after the break we're coming up against the break right here but there's some opportunity to serve on the board if people were so inclined and we'll maybe chat about that a little bit I mean how many open positions do you have well good question right now we have what two open one open position but hopefully the governor signs part of the legislative nature one of the bills that was passed was increasing the board from nine members to 11 so we'll have three three cents to fill so you're gonna have to get a bigger room that would be nice right we've got to get more water too that's good yeah and so in 11 so we're gonna be able to you know bring in more industry experience and more talent and really have a very viable strong going I know you already have one but having an extra couple members just allows us more strength and that was the intent the idea being that you know one was to have a chief quorum but more importantly to have a voting quorum when we have issues that board members need to recuse themselves if they got direct financial interest so a lot of times you can't you can't do anything we don't have a quorum so by having 11 hopefully we don't run across that too many times that's a good point yeah it's good that even when you recluse yourself it's good to have enough voices there to really be powerful when you make your recommendations to the governor all right so if somebody was interested in serving or submitting their name to be considered for service on the board how what's the process for that there's there's a website that it's all electronically you need to download your resume and your interest on the state boards and commission website and then choose small business regulatory review board okay to serve on all right and would that be through the lieutenant governor's website or just through DCCA or do you know how or where that would be there's an independent website called boards and commissions okay all right so i'm sure there's a way that we can find that google is amazing you can find stuff why is state boards and commissions there we go okay very good well let me um go and break for a minute we'll come back we'll talk a little bit more about the makeup of the board and maybe we can talk about some of your success stories you know what some of the issues are that you're dealing with this is business in hawaii with reg baker we're talking today with the hawaii small business regulatory review board the chairman and co-chairman will be back in about 60 seconds thank you very much this is reg baker business in hawaii we are talking today with the state of hawaii regulatory review board i keep calling it fairness full disclosure i'm on the national regulatory fairness board of the sba and sometimes i have a fruity and slipped there but it's the regulatory review board for the state of hawaii and we've got tony and kiko here today and and they're sharing with us all of their experience on on how this board works now you've got nine you may be going to 11 but you're looking for a cross section of different industries different islands we've got one open spot for kawaii right now what kind of industry experience are you looking for well we do need i know finance right off the back would be good finance accounting that that area and then farming agriculture would be a good one too that we currently need and that's a big part or a growing part of our growing this one growing part of the small business makeup here and hopefully it's green yeah you know so but yeah so the agriculture uh transportation would be another one you know that's a that would be a big one there's a lot going on out at the airport or or should be going on at the airport yeah we all can agree on that yeah definitely yeah so if if they need to go audience has any experience in any of those industries or have an interest in serving on a regulatory review board um they need to go ahead and go online find that website attach a resume fill out the form and submit it right and the qualifier is basically either you know a current owner or previous owner of a small business and define small business real quick well for hawaii it's you know less than a hundred employees basically i know on the federal level it's 500 right so you know for for hawaii you know it's a hundred then and of course a hundred employees in hawaii today is really it still represents a big percentage of the businesses i know at 500 it's about 97 98 percent at 100 it's probably 60 or 70 percent yeah you're right that's an educated yes but probably in the ballpark all right very good um and explain to me a little bit about some of the successes that you had i i've heard about the the oahu liquor commission and they've been very responsive yeah they have been very pro business especially for the small businesses and they they propose this amendment to their existing rules but some of the small business raised some concern and they they listened and they changed the amendment that that's really unheard of for the government i was just gonna say that that's amazing you know so we upload their efforts of listening to us you know and and they were responsive and they understand the purpose of the the department is to keep the community safe not to hurt the community and if as long as they can keep the mission they can be flexible right and that's a good attitude to have it's more of a partnership and cooperation working together is it safe to assume that all the organizations and agencies you work with act that way i wish i could say yes i mean that would be the ultimate um and you know pretty much um you know every monthly meeting it's what i kind of interject into that if it's the agencies coming forth is that you know if they can engage the stakeholders early on in the process rather than at the tail end you know when no surprises you know and that's why i guess you know one of the things i don't want to get off track but we got to look at how you know the the way the rules and regs are put forth now the process we need to look at that that's going to be a statute change but they need to look at that and maybe change it because nobody wants to you know you do all your your research put put your proposal out for public hearing and you get all of a sudden because you didn't engage the stakeholders you find out there's a lot of problems with this by that time you're already done 99 percent of the work you think you're going to go back and want to change any of those rules or you know that you put forth the best intentions but you know and doing the job that should be done but you know you forgot about the whole intent is you know we still got to work with the small businesses right i mean going back and redoing things is not only a little embarrassing but also expensive you know it takes time and effort to go back and rethink some things and and then once you make that acknowledgement then you start pulling in the stakeholders and you might find out that there's other areas yeah yeah i think you know and not to politicalize anything but that's one of the issues when they did obama cares that they just tried to come up with this plan and somebody said read it and you'll find out what's in it and when they read it they found out there was problems and going back to redo it is something that we've been fighting with for what eight years and you know you learn from your mistakes right so the idea is hopefully it does that doesn't happen again and i think it's important that you know the more we can you know collaborate work together we know we got to have rules and rags but we got to make it so that it's it's doesn't you know impact one industry or one group of in this case small businesses more than it should you know and that's the key because we got way too many rules and rags for a small business person to you know some of it make a lot of sense some of it you know if you ask me i if we're not yeah i'll tell you exactly what i thought about it but you know and that's how we volunteer our time because we want to you know minimize do away with what no i at least advise the governor and you know that it just it doesn't make sense or impact businesses you know heavily that it shouldn't be you know getting good effective collaboration tends to improve the final product i agree you know and what you end up with is going to be a much better result than if you try to do things independently without getting that input now it may drag the process out a little bit but if it's done efficiently it doesn't have to be that much more effort unfortunately we still have a lot of rules and regulations that are not unnecessary and i've heard maybe not always consistent so what what maybe a set of rules on a wall maybe different it's supposed to be the same state law the legal law is but like i gave you an example in oahu but in maui every board member even for the non-profits needs to submit all the records of fine 25 dollars or more ever since you are 18 at like my age or your age do you remember do you remember you have library fine when those years are kind of foggy so something like that it doesn't prevent from securing public safety by missing one library fine when you're 19 right it kind of sounds like somebody's got an agenda yes that maybe isn't you know directly involved in their duties maybe at the liquor commission it's so for some other reason they're trying to stall the process or stop certain things from happening and that's unfortunate our goal is to make this state a bit easier place to do business that's what we are shooting for you know and the and the reach of the small business regulatory you boy you know we we're under the d-bed state agency but we also advise or provide input to the counties as well and that's why you know like the city county of honolulu county of koai maui county they put forth you know things that affect small business comes to us as well and and gives us the opportunity to give our input like it or not you know yeah that makes the the review board kind of uh unique in a sense because not only is it a state type of board but it also has city and county activities going on advisory advisory but nonetheless it still it sounds like a pretty important board to be on you know to hear about all these changes and all these things that are going on and and hopefully take some steps to make things more workable for that small business community that's the intent and that's why we you know we um part of the board and um you know it to me it's small business needs representation um it's it's it's uh whatever help we can provide uh you know we're like the you're like either advocate for small business we're the abuzzment for you know that's sorry for small business and um and hopefully we can make an impact does do you have the opportunity to collaborate with other organizations so that you can take the voice and make it a little bit louder i mean i know that's kind of like the brand name for the chamber of commerce is the voice of business you know so i see an opportunity for some collaboration there does that happen yes like we we try to go speak to them as much as possible in in all counties that's like we now we just had that uh large conference of small businesses and we participated there that's good that's good so that was another platform for you to get some feedback from the the small business community on Maui on on some of the issues that are out there yes and did you get some of that yes all right everything kiko did a great job representing the the small business regulatory review board as well as dory was part of you know debate who who's the executive director of the board but they did a great job of Maui and we you know we continue we want to do more like that going forward you know we want to get the word out who we are what we do and that way you know maybe we can have standing room only at the um hearings that we have is especially if it's matters that will affect you know small business so absolutely and what we need to really work on is getting somebody from kawaii we need that yeah we needed a person from kawaii for the last six months yeah it's been vacant so now kawaii has a chamber of commerce out there have you reached out to them we have i think mark is the executive director yes yeah in fact mark's been to a couple of our meetings yeah so okay well i could put some pressure on that too you know we'll get somebody from kawaii yeah i had a guest on the show i guess uh last year it was the pi lady of kawaii and she makes all the savory pies and the sweet pies she's got like three or four different shops so we weren't invited yeah well there was only enough pie for one we can reach out we know some people all right but we're gonna have to wrap up right now it's great to have you back on the show again i'd like to try to make this a regular type of recurring update and particularly if there are issues out there or if there's a particular issue you want to get more feedback on maybe there's a way we can have a show and talk about that i'll just kind of help get the word out but this is business in hawaii with reg baker we broadcast live every thursday from three from two to two thirty uh from the studios in the pioneer plaza downtown hololulu we're looking forward to seeing you again next week until then aloha


Welcome to Spotlight. I'm Colin Lauver. And I'm Liz Wade. Spotlight uses a special English method of broadcasting. It is easier for people to understand, no matter where in the world they live. This program is about a very famous artist, but it is also about his mental illness. It mentions suicide. If you are thinking about suicide, please call a crisis centre in your area. You have probably seen this painting before. It is a painting of a night sky. Stars shine over a small village. The moon glows yellow. But this is no ordinary painting. Swirls of white and blue fill the sky. Each star is as big as a sun. It is clearly a real sky, but the way it is painted makes it a mystery. There is some kind of magic in that sky, but you do not know how it got there, or why. This painting is called The Starry Night, and it is by the artist Vincent van Gogh. He lived during the 1800s in the Netherlands. Some people say that van Gogh was a genius. He influenced many artists who came after him, but van Gogh was also unwell.  He died without much money at a very young age. Few people knew about van Gogh when he was alive, but today he is one of the most well-known artists in the world. Today's spotlight is on the life and illness of Vincent van Gogh. Vincent van Gogh was born in 1853 in the Netherlands. His family was wealthy. Many of them were art dealers. Some were Christian preachers. When he was young, van Gogh did not impress many people. He had mood swings. Sometimes he was very happy, and sometimes he was very sad. These emotions were very strong. They often annoyed family and friends. He was not able to control these emotions, and the people who loved him did not know what to do. As he grew up, van Gogh did not know what to do with his life. For a time, he was an art dealer, but he was restless and quit after a year. Then he wanted to become a preacher, but the church did not accept him. He often lived without much money, and he sometimes lived on money that his parents gave him. Van Gogh did very badly when it came to practical things, but he soon discovered that he was very good at making art. He began to draw and paint constantly. He believed painting was his calling. He believed he was born to paint. Van Gogh had a brother named Theo. Theo was an art dealer in Paris, France. He believed his brother was a great artist, so he funded Van Gogh's art. Later, Theo invited Vincent to live with him in Paris. Living in Paris was very important to Van Gogh. That was where many other interesting artists lived. There, he was able to talk to and learn from them. It helped him become a better artist. There, he created his own style. But Paris was also where Van Gogh's mental illness really began. He had always been very emotional. Often, he was very depressed. He abused alcohol. He especially drank an alcohol called absinthe. But in Paris, things began to get worse. Sometimes, he would become terrified for no reason. At other times, he would lose consciousness. He would forget where he was. After a few years, Van Gogh tried to leave Paris. He found a house in a town called Arles. He hoped leaving Paris would help him get better. Instead, it made things worse. Paul Gugain was a famous artist. He lived with Van Gogh in Arles. Van Gogh admired Gugain. He wanted to work with Gugain. But Gugain began to dislike Van Gogh. They fought often. Van Gogh told his brother Theo, Our argument is at times full of energy, like with electricity. At times, we end up with tired and empty heads. It is like an electric battery after discharge. Soon, Gugain told Van Gogh that he was leaving. This changed something in Van Gogh. Gugain was very important to Van Gogh's plans. He did not know how to deal with the change. Later that day, he approached Gugain with a razor blade. Van Gogh did not hurt him. But Gugain felt threatened. He left soon after. That night, Van Gogh began to hear voices, even though he was alone. They tortured him. To make them quiet, he picked up the same razor blade.  Then, he went to a brothel. He gave his ear to a prostitute. Then, he disappeared.  There, doctors tried to treat him. They believed that he had epilepsy. Epilepsy is a disease that causes fits. During these fits, people lose control of their bodies. But, it can also give people hallucinations, where they see and hear things that are not there. Soon, Van Gogh began to recover. He stopped drinking alcohol. And, he started taking medication. He left the mental institution. And, he began to paint again. During this time, Van Gogh made some of his most well-known paintings, like the painting described at the beginning of the program. He painted similar things to other artists, like flowers and fields. But, the way he painted was very different. He did not just paint these things. He painted what he felt about them. And, he used colours in ways no one had seen before. People began to recognise his works. But, Van Gogh's mental illness did not go away. He had several mental breakdowns. He went back to the institution several times. He hallucinated and feared being poisoned. And, he was often very depressed. He did not feel like a success. He wrote to his brother Theo. I am so angry with myself, because I cannot do what I should like to do. At such a moment, I feel that I am lying at the bottom of a deep, dark well. I feel completely helpless. Van Gogh's mental illness was very painful for him. One day, it became too much. He went out into a field with a gun. He shot himself in the chest. He did not die.  He was bleeding from the chest. Van Gogh died two days later. He had committed suicide. He was only 37. After Van Gogh died, his paintings quickly became famous. Museums began to display them. Art collectors bought them for a lot of money. Newspaper editors called him a genius. He influenced other famous painters like Henri Matisse. Today, Van Gogh is one of the most well-known painters in the world. But, for all this, we will never really know what illness Vincent Van Gogh had. Some scholars agree that he had epilepsy. Others believe he had bipolar disorder. People with this disease have extreme periods of depression, but they also have moments of high activity, or mania. A manic person often feels afraid of imaginary things. He may hear people speaking that are not there. Both of these diseases seem to describe Van Gogh, but there are many other theories. Many people even think he was such a good painter because of his mental illness. They believe his strange colors were because of madness, or they believe his swinging emotions pushed him to paint. This may be true, but these things also pushed him to his death. He felt so strongly and loved what he did. He fought through many years of difficulty and changed the world of art. But Van Gogh did not see this. He died thinking he had failed. Before he died, he told his brother, If I could have worked without this terrible disease, what things I might have done? Mental illness can be very serious. If you or anyone you know feels suicidal, please find help. There are many options for help. You can call a suicide hotline in your country. Or talk to a friend and ask them for help. More than anything, you should know that you are not alone. You should know that you can get better. Or email us at contact at spotlightenglish.com. You can also find us on YouTube, Facebook, Instagram, and Twitter. The writer of this program was Dan Christman. The producer was Michio Ozaki. The voices you heard were from the United Kingdom and the United States. All quotes were adapted for this program and voiced by Spotlight. You can listen to this program and read along with the script on our website at www.spotlightenglish.com. This program is called The Hidden Life of Vincent Van Gogh. We hope you can join us again for the next Spotlight program. Goodbye.


Hello, my name is Charles Hadley King. I'm a senior research associate at George Washington University and technical lead for the biocompute project. Today I'm going to be discussing publication of biocompute objects created from galaxy workflow invocations. I'm going to spend the first few minutes of this talk, going through a few slides about the development of biocompute objects and the needs that biocompute objects fulfill. And then I'm going to do a demo of some recent integrations we've submitted to the public galaxy repository. Biocompute is a standardized template for reporting how a computational experiment was done, and also includes why it was done. The original need for biocompute arose from United States regulatory agencies specifically food and drug administration, but its application is far greater than just regulatory science. The standard reporting template supports fair data standards and encourages good data prominence. So it's something that we see is applicable to many other areas of science communication. In 2014, the genomics working group at the US FDA can be into special session to discuss workflow communications, particularly as it pertains to NGS or sequencing data. They came away with four main aspects a solution should adhere to. The first is that it'd be human readable, like a gen bank sequence record. The second is that it also be machine read, which means it has structured information, predefined fields and associated meanings of value. Everyone is explicitly clear on what every single field means. It should contain enough information to understand the confrontational pipeline to interpret information contained therein should be complete enough to serve as a record. And it should allow reproducible experiments. The final takeaway was that it should be a meeting that we should have a way to ensure the information has not been altered since it was first created. Out of that meeting bio computer was born. Currently, it's an IEEE approved standard for communicating bioinformatic analysis workflows. It does this by acting like an envelope for the entire pipeline. It's essentially an aggregate of all of the information that one would need to understand the pipeline. And in doing so, it also incorporates other standards. It is human and machine readable. It is categorized by domains, which I'll get into in a little bit. It adheres to and encourages fair principles. It's also fully open source. It's adaptable. Preserves data provenance and has a new ID perversion. Bio compute streamlines the reporting without enforcing any specific tool platform overflow strategy. And machine readability enables customized views and development of other tools associated with the report. So here is a bio compute object color coded to highlight each of the different domains. There are eight different domains by which are required the provenance list contribution name identifier state of creation description domain contains information to describe the pipeline. Keywords external references. Steps specific tools use our domain is where you can record sources of our parametric domain specifically lists each parameter that's been used in the workflow. Execution domain contains information that could be identifiers day of creation description domain contains information to describe the pipeline. Keywords external references. pipeline steps specific tools use our domain is where you can record sources of our parametric domain specifically lists each parameter that's been used in the workflow. Execution domain contains information that could be used for reproducibility, i.e. where a script is located external data endpoints so other resources that you need access to to execute the workflow. The IO domain input output contains exactly that your inputs and your outputs. The user domain is for user defined fields that are either required or optional. This is somewhere where you could insert your own defined fields to include additional information that might not that might not be present in a base bio compute. Execution domain, which we see as one of the most important aspects of the bio compute that's where the scientific purpose of analysis or computation is recorded. Free text field so a creator can put in a description of what they're hoping to accomplish the analysis, they can link out to other papers or previous studies or analyses. To review the key features of a bio compute object. It abstracts away workflow based on commonalities. So it's platform tool protocol independent usability domain being free text description and scientific purpose data provenance. You have data manifest track processes from beginning to end. It's a track user attribution. So, authored by contributed by viewed by. So you know which person contributed what type of content to the object or the analysis. There is what we call a verification kit. So, with a full bio compute object, including an error domain and the IO domain. The input and output files. This can give you a sanity check, given a specific set of input files and the inherent or known error is the output of this analysis claims to have gotten valid. Bio computers are extensible. To the extension domain and fully open source. Bio compute was part of the open source pilot program for IEEE. So it's one of the first standards that they published that was fully open source. Over the development process. Many different groups have helped with the development of bio compute both conceptually and technically. And a few of them are listed here. Probably about 400 or so different individuals have participated in one way or another. And helping to develop the technical specification. So let's move on to the demo here. I have a instance of Galaxy local instance running right now. It's based off of the code that we have on our bio compute work of Galaxy at bio compute objects slash Galaxy. Currently, we have a poll onto the main development branch. Possibly to be released in September. We have a release of Galaxy. But for the time being, you can either take it from our GitHub. Or you can run it at our Galaxy portal link here. So hopefully everyone here is familiar with workflows and how they work in the Galaxy ecosystem. We chose workflow invocations as the entry point for bio compute objects because most of the information we needed to create a valid bio compute object was already available in that portion of the Galaxy code base. So I'm just going to show you to, you know, construct a workflow. I have a pre constructed workflow that I've used. We're going to run this workflow and select run workflow. And this is going to work here on my local machine. So we'll take you over to this. So we're done. And you can see here. In the particular run ID. We have options and option for bio compute objects. It's also available under workflow invocations. And you can see it from this same workflow a couple of times to test it out. So we'll go with the one that I just ran. Previously, I think the last year's GCC, we had integrated some the ability to download a bio compute object. So, you know, that's still there. That parses the workflow invocation and pulls out a good deal of information, but there's still a few pieces that you know a user needs to input and they can't be pulled out automatically. So to address that we've added a export field to this menu. You simply fill out the URL or BCO database API and give a specific user API key select the table and the groups that you want to apply it to and hit submit. Now I'm going to come back to this in just a second. In our bio compute portal. We aggregated all of the resources we have for bio compute objects links to the hive instance on AWS galaxy instance a page for different community organizations associated with bio compute check those out. We also have a page where we list all of the different resources and applications I'm just going to sign in with my account and can see here a few objects listed. This is our beta version. So these are just test objects here to submit an object from galaxy. You need to acquire your API token. So once you have an account and once you've logged in you'll see this bio compute object server section at the bottom and you just copy the token here and put it into the form just to show we have a few draft objects here. So I'll go back to galaxy hit submit and then refresh this here and you can see there's a new use or new object here galaxy bio compute object development test. So now I'm able to open the form here and you can see the information that came from galaxy has been populated into this bio compute object now to and user can fill out the form here if it's filled out and valid you can select the server to publish it to assign permissions to the associated groups and publish draft. Now we should see our galaxy bio compute object here in the bio compute viewer and this is publicly accessible for anyone to see as I'm showing you with the chrome incognito tab right here anyone can view this object. So this work was done by myself and Christopher Armstrong the development lead for bio compute project contributed significantly to this I'd also like to thank Dr. Jonathan Keeney assistant research professor and the bio compute lead Dr. Rajan Mazzunder my PI at George Washington University and then Janisha Patel bio compute


So while the sound is getting set up, I think we should kick off in interest of your time. So good morning, everyone. Everyone had a good keynote session in the morning? Okay, today I'm going to talk about enabling continuous delivery in enterprises with testing, what that really means. And this is a case study of a project that I was working on, how we got onto the path of enabling CD in one of the enterprises. A very quick introduction about myself. I'm Anand Bagmar. I'm working with ThoughtWorks since five and a half years now, been in the testing field for about 17, 18 years. I love testing. Anything related with testing, I would love to get involved with. I have been getting involved with. Enough about me. There's my Twitter hashtag over here, as well as some information how you can find more contact details about myself from the about me page. Okay, so enough about myself. This is really about what you are here for. And before I start off on my rant of what I think the content is, I would like to know from you guys, what do you expect from this particular session? How you can continuously deliver, okay? That's a very loaded statement, but we'll try to address that in some ways, okay? So testing is sort of part of the DevOps cycle and how is that really related? Seeing that. Any other thoughts? Okay, so infrastructure requirements and other performance related aspects when coming to continuous delivery and how does testing fit into that, right? There was some challenges with continuous delivery, okay? And how do you manage the dependencies around that? Okay, what is the role of testing in continuous delivery? Okay, to make sure you actually continuously delivering and not delivering one thing and breaking what has been delivered earlier, okay? So in enterprises, maybe some sections of that enterprise are in the CD, on the path to enable CD, whereas others might not be. So how do you really fit in together these different aspects? Requirement changes, how to handle that? We will not be covering that. Enabling continuous delivery in BI world. So you could potentially take examples from this in your context, but we won't be covering that also. Sorry. Tools used for continuous delivery. Yes and no sort of. But I think we've got a fair mix of expectations over here and the point is we will not be talking about any of these. This is a magic show. We are going to prove that a triangle is equal to a pentagon. Confused enough? We'll see that towards the end, if we get to the end what we really mean by this. So this is a story, a true story for that matter, of a core banking implementation for one of the larger banks in Australia. They were on a path of continuous delivery and how many of us are from the banking segment over here? Excellent. Please don't shoot me if I say anything wrong. A quick show of hands again, how many of us are testers over here? Any testers? Okay. Developers? Excellent. Who am I missing? Of course, BAs, few, PMs, product owners. Who else am I missing? Stakeholders, business, managers. Okay, excellent. So we've got a complete mix of what the team is sort of structured of in any particular domain. Okay. Banking domain, as we all know, especially the ones who are in the banking field know, or the finance field know, it's usually complex. But regardless of the complexity, any organization has got specific objectives or the same objectives overall, right? Those objectives are, they need to make money by providing some value. To do that, they need to get their product or services offering out to the customer base in proper time. Otherwise, it is not going to be able to provide value to the customers, to the user base. But as we heard, if some of us were here yesterday, just in one of the talks, I think Jeff Patton mentioned that, is just getting out on time is not sufficient. You need to have proper quality. The feature set that you are delivering to the end user should actually be valuable and usable by then. And these three things go hand in hand. If any one of this gets missed out, potentially it's not going to have the impact on the segment that you're talking about. Now another reality is that organizations are spread across the world. And for various good reasons, right? Globalization, the cost factor, it's very expensive in one place versus the other. You want to spread that. You want to have 24 by seven availability or keep that machine rolling continuously at all times. The team sizes are just so large enough that it cannot fit under one roof. You have to split across in different floors, different offices, in the same city, multiple cities, different countries, continents, whatever it might be. Team sizes do require that kind of a split. Mergers and acquisitions happen, which means that the organization you're merging with might not be in the same place. So again, you are spread across, right? And of course, talent is available all across the globe. You want to tap into that rich talent pool from the best possible conditions. So there's nothing different from this perspective what this bank was going through. They were split across, rather the main business of course was in Australia, but they also had business in New Zealand. And to help them in the implementation services, there was work happening from India and China as well. So there were multiple locations, lot of locations in Australia, New Zealand, and quite a few locations from other regions also, which was helping deliver the product to their customer base. Let's talk about what the scope of the program really is. And that is what truly adds to the complex nature of CD. Hopefully, you all will be able to relate with this, even though you might not be from the same domain. So if you look at this weird-looking diagram, it's not an Apple, it's not a Amoeba or whatever other thing we might have learned in biology. The gray area is the complete core banking platform for the bank. This has got a lot of systems inside it, but the important ones are there are some external dependencies or external interfaces, integrations that need to happen with existing systems in the bank, which are not directly related with the core banking. It might be certain things like payment processing gateways or various other factors, right? Compliance related, it could be anything. The blue icons represent external systems that this platform needs to integrate with, external from the scope of the bank. This is something, maybe it's government organizations or irregularly authorities, which is doing independent validations or compliance text, whatever it might be again, right? So these are external offerings or external systems that this core banking platform is closely integrated with in order to provide whatever the bank needs to do. The intention of this core banking implementation is to replace this gray box, which is a legacy core banking platform, the mainframe system essentially, with an off-the-shelf product from one of the many providers that is there. You want to do a complete transformation from the gray to the red. Of course, we are buying an off-the-shelf product as we might have seen from our experiences. Any off-the-shelf product can rarely be used directly in your own context. What it means is you need to customize it in whichever fashion required, whether at superficial configuration levels or actually change the product the way it works in order to fit it to your purpose. So there's a lot of customization that needs to happen in this whole process. Along with this customization, because you're changing the core engine of how banking works, you also need to change certain aspects of these external integrations that are there, internal to the bank or external to the bank also in order to fit into this new way of the new platform. So it is a pretty complex thing. This was a very, very simplistic view of the core banking platform. What it really means is this is the architecture, a very high-level architecture view of the different systems that the core banking platform entails. Red is the product, the new product that is going to be replacing existing systems. Green is the existing bank system that get affected by this, which need to modify to work with this new platform. Blue is the external interfaces that you need external systems that you need to integrate with. And there are also some grays which are there in the scope but they don't need any modification at this point in time. So we understand the scope or do we? What this diagram represents is there are 130 systems which means 290 interfaces in this context for this whole core banking transformation project. It is a huge scale. The very aggressive best case estimate for this to do this complete transformation was five years. Realistically speaking, it's probably going to be around seven to 10 years which is again very aggressive. Now, for those who have been on the transformation journey whether replacing one system with the other or whatever, can you really do a big bank approach in that?  Does it really work like that, especially in the enterprise context? No, right? What you have to do is in phased fashion you start taking down, replacing one system with the other and you start off on that journey of changing one piece at a time in some logical sequence and eventually you'll be able to pull the plug on the old system and now there you go, you have implemented everything. By that time, of course, there'll be new systems that need to change because again it's taken quite some time. What are the execution challenges in this particular context? There are multiple partners involved in helping the bank get on to this journey. With multiple partners and we're talking about enterprises, right? This is not just one smaller services organization that is involved. It is much more than that. Large partners, large vendors involved. And with each partner, they have their own policies, their own way of working, their own constraints, how quickly they can adapt or change on this path. So the bank is one. There are at least three different partners involved in this. So getting all four aligned together, it is a big challenge. The stakeholders involved. Imagine we are talking about 130 systems. This is purely from the core banking implementation perspective. There is a huge level of governance involved in all this, which means you multiply that at least three times. That is the partner stakeholders also getting involved in that governance model. So there are a huge number of stakeholders and getting everyone aligned in what is to be done in which fashion is tricky. Not impossible, it is tricky. Now the biggest challenge is agile. The bank wanted this to be implemented in the agile way. Why? Because they wanted to see incremental value come out of this change. It's not a big bank approach. I've given this to you, see you after five years. It is not like that. This bank is, yes, they have money, but they don't have an endless pocket, right? Any organization would be prudent in that sense. So how can you get incremental value? One of the good ways is agile. Now how many of us in the room over here work in the agile way or some variant of agile? Well, we are in agile conference, so I'm guessing almost everyone. Now, if I put you all together or give you a question sheet of sorts with very explicit questions of what agile means, do you think everyone will come out with the same definition? The same way of working, the same principles, the same practices, principles, yes, but the practices, it's not going to be the same, right? Everyone has got, unfortunately, a different interpretation of what agile is. Likewise in this case also. There are various stages of agile adoption and understanding. One is about, I don't know agile, but I want to learn it, so I'm a learner of sorts. The other is I'm on a path, but I'm a checklist-driven agile. I have got a list of things, if I'm doing stand-up, if I'm doing my iteration planning, my show cases and everything, I am agile. Is that really being agile? Not necessary, but you are a step ahead of someone who doesn't know that. And then you get to being agile. You truly understand what is required, you understand the principles, you understand the context, and you pick up the practices which are going to be best suited in your context to implement and deliver as quickly as possible. So there are various stages of being agile adoption, and Martin Fowler has written about the agile fluency model also, which talks much more in greater depth about what it really means. But the point over here coming back to this context is the bank wanted to go agile. They had a certain conception of what agile means and how to implement it, but their partners, some of them were traditional waterfall-based companies. They used to execute in that fashion. Some of them were on the agile track, but again, different understanding. So how do you get them all aligned together because if you want to deliver this core banking platform transformation, you all have to be aligned as one team, one thought process, one clear understanding of what is to be done. Huge challenge. He spoke about the money. So there is money available. It's not an endless pocket. You have to be prudent and you have to see, start seeing value out of what you're doing incrementally and not wait for five years to know if it has worked or not. You cannot just keep spending for five years without that kind of feedback. So that is a problem. And for various reasons, trust is a major, major issue because again, different policies, different constraints, different way of working and distributed nature when you are not co-located in all possible ways for all possibility members, that is a huge problem. How do you build trust and keep that trust going on for that sustained period of time? There are some other challenges also involved in this. We mentioned about it's not a big bang approach that you just flip a switch and now you're suddenly on a new system. You want to coexist with the existing system and start replacing it one by one. Now that is a major challenge because in 130 systems that you are already having, you are adding the complexity of many more systems and not just saying that, okay, my payment gateway is, I'm just going to replace from one to the other. You might change it for certain parts of your core banking implementation but the other parts still need to use the older system because everyone has not yet migrated over or moved over, right? So coexistence is a huge problem. Likewise for data migration. The data has to be there, it's banking, right? I cannot, it would be great if my statement balance just doubles up, hey, I'm happy, but what if it reduces? I'm not happy. So data consistency is extremely important from this perspective, especially when you're coexisting to make sure that the data is correctly stored. The scope, it is so huge, it is difficult to take a thin slice and say, okay, this is what I'm going to do in the next three months or six months and I'm going to deliver. Because of so many stakeholders involved, so many systems involved, getting all of them aligned and remember, it's not just the core banking implementation team, we're talking about so many other external systems also. They also need to do some work in order to fit in the new platform. So how do you get that aligned? That is a big problem. Defect management. With the partners, so many different partners involved, so many different systems involved, ownership of all the systems is not in one place. So if I'm taking off the shelf product, if there are issues with the core product itself, how do I raise those issues? How do I get them prioritized and get them fixed so that I can get that into the system as quickly as possible? Likewise, if I'm customizing the product, it's some other team potentially doing that customization, how do I identify that and get those issues fixed? So it's a huge problem about how do you really maintain, manage all these issues, prioritize them correctly and get it going. And lastly, the team distribution and integration. Distributed nature, the hugely integrated systems with mainframes and databases of various different sorts and so many other systems, it is a huge challenge to get that coordination done. So these were various different challenges from the implementation perspective for this core banking project on a path to say, yes, we want to do CD. Along with this, as if the existing challenges were not enough, because it's a bank, there are a lot of non-functional requirements that we had to cater for. Security, no-brainer, right? Banking system, it has to be secured. Performance, it has to be performed from various different ways, right from the integrations of different systems to the end user experience what is really happening. Now it's a hugely complicated system where the end user is not people who have accounts in the bank, which could be just savings, current account, whatever it might be, but there could be loans and various other insurance policies. There could be various types of customers and you need to have the system performed on that basis. Auditability, if anything goes wrong, this is where the government policies also, each country policies also kick in to say that if I'm in Australia or wherever, I need to also conform with these other things. And if something goes wrong, I need to know exactly what happened at different points in time so I can trace back to the origin and figure out what the problem was. Compliance, very big thing and different for each and every country. So even though the bank is operating in Australia, they might have different compliance requirements in New Zealand, for example. How do you cater to those? How do you implement those? How do you validate those? Big problems. And accessibility, that becomes a huge thing. So these are just the top five non-functionals that I've listed but there are many more that we need to think about in this whole context. So how do we really get to having good code quality in order to start delivering value? The way the system worked and this is a very, very simplistic view of that whole platform or the team structure, there is a core team where this is the off-the-shelf product that we've bought and because it's such huge in nature, the bank is working tightly and in collaboration with them to say what the requirements are and how it should be done. The core team, the developers would write code, put it in their subversion system which now because we are working as one team in some ways, would also get pushed towards bank subversion system, get synced manually in some form, manually or automatically triggered in whichever way. Once this code is of decent quality only then it needs to get pushed into the main subversion system and that's where the code would get compiled again and tests would run against that. Now, against this particular subversion system, this is where the customization teams would start working on it. Now I have the off-the-shelf product, I need to add my customizations on top of it and it's going to go in the same subversion system. Likewise, the integration and configuration teams would be working on the same thing and the systems team that, okay, I now need to say how do I print a checkbook for this particular user from the new system? It would connect the printers to it and say, okay, can I print the correct checkbooks over here? That kind of thing. Or tellers in the bank itself, how do they connect their systems and see the data? So that becomes another systems team to see that type of integration. And of course, the NFR teams, because it's such a huge thing, ideally, you would want every one of these to be sitting together as cross-functional teams and working on delivering the functionality. But for such a large scope, there has to be separate teams also created to look at the whole and not just part of what has been done. For the scope of this discussion, we are not going to focus on the NFR teams or the system teams. We are going to focus on the core product team, customization, and integration configuration teams. So from this kind of a complex system, how do you really get early feedback and ensure that the quality of your product is good enough for you to take it to the next stage? For you to potentially say, yes, I'm ready to go live or get into, get at this particular feature in front of my internal users, bank users, so that they can start providing me early feedback. So remember, the objective is this, right? Nothing has changed from this perspective. So how do you really achieve this? The first stage of getting to continuous delivery is, of course, continuous integration. You have to get this itself set up. Anyone over here who's not aware of continuous integration, everyone is, I'm going to skip through. A very quick thing about it, it's integration verified by an automated build, including testing, could detect integration issues as early as possible. This would happen on each check-in that developers would do. The process would be, anyone checks in code, it goes to a subversion system that tests automatically and likewise for any developer from any team across wherever they might be sitting, any change is done, you would want tests to run around that as quickly as possible. Continuous delivery, we'll get to that part. How is automation integrated as part of CI and why do we need to do that, right? We'll touch upon that briefly. If we don't get a chance to answer that explicitly, let's talk after the session, so we can cover that part. So continuous integration, it's a natural extension of continuous, sorry, continuous delivery is a natural extension of continuous integration. The whole intention is to make deployments boring. That is a boring process, that is a very error-prone process and if you can automate that part, you have taken care of a lot of issues that can potentially happen in your deployments and moving code from one environment to the other, for example. So again, a link over here from Martin Fowler's article on continuous delivery, you can read more about that, but the intention is that you quickly iterate over what is your business idea and the color is looking really bad thanks to the light, but there's some business idea, you take that idea, build on it, design it, do whatever process you need to follow to implement that and deliver it to your customers and the cycle keeps repeating, right? You just want to keep doing that as often as possible, as quickly as possible so that your feature set, whatever you're working on, gets to the end user. So how do we get to continuous delivery? This is what we are really here for, right? We understand the principles of agile, I hope. To implement those principles, there are various different practices that can be there. The principles might be tens in numbers, manifesto has got four principles that would potentially lead into many more well-defined things, what need to happen to implement the principles in the manifesto, the tenets in the manifesto, but those principles can have hundreds of different ways to be implemented and which way is right for you depends on your team, your context, your product, your domain, your team skill set, your team distribution and various other things. One of the practices that is essential is what I want to ask you, do you know what that is? What is one of the practices that is essential to make the team successful on the path to CD? Continuous improvement, communication, collaboration, early feedback, how? Exactly. So essentially it is automation and I remember I'm saying it's one of the practices, right, it is not deep practice. All the answers you said were correct, but the answer I was looking for is automation. To get to continuous delivery, you need to have good automation in place, otherwise what happens? Think about you're making one change in that big complex ecosystem. You would have to manually regress each and everything in order to release that. Now, would that be a viable option? You've checked in your code to CI, but if you don't have a backing of automated tests for this kind of environment, ecosystem, is that really going to make sense? How much time would you need to regress all those? So automation is one of the key factors, the key practices that need to be implemented well in order to help in early feedback and know what is the quality of the product. Do I need to move back, roll back some changes and fix them properly before moving on ahead or I'm good to go to take the next step, right? Likewise, what is one of the other practices that can make the teams unsuccessful? Automation, why? Automation done wrong, explain. Can anyone explain? Pass but not pass, excellent answer, quantify that. So it's essentially saying assert true equals true. It's always going to pass, regardless of what's happening in your product itself. Other problems related with automation? Low coverage of tests? Falls alarms? All scenarios were not covered. Maintenance, sorry? Data, sorry? Baselining the data. Very important especially for such a complex system. Automation coverage is not complete, right? Sorry? Too long time. Okay, now we're getting to the crux. Duplications. So there are various other things, right? Based on our experiences over here, potentially we could come up with a unique reason for each person in the room over here. That could be a reason why automation has gone wrong. And what happens when automation goes wrong? That's how we create a market for manual testers, okay? Unfortunate thing, but true story again. So what is test automation? Test automation is a safety net. Everyone knows how fishing is done, right? We have fishing nets. Fishing nets are of a particular grid size. Can I use any grid size to catch any type of fish?  Does it make sense? If I do that, I'm going to catch a lot of junk or debris along with that fish. If at all I catch the fish. What is important is you need to identify the right type of test which is going to give you the right type of coverage. The right type of confidence in your test suite and the results related with that test suite to say yes, this is the feedback. This is not a false alarm. This is not a mis-scenario. It might be the test is out of date, but it has actually caught the right issue. The product functionality has evolved, but I have not updated my test for it. A valid situation. So you need to identify the right type of test. Now, what is the right type of test? Very subjective. For that, the test automation pyramid is a great reference to see what can be done about. How do you identify the right type of test and where it should be automated? Anyone not heard about the test automation pyramid? Good show of hands. So I'm quickly going to go through this one. Test automation pyramid was a concept introduced by Mike Cohen in his book, Succeeding with Agile. Martin Fowler has again written about it in a very simplistic terms of what that means, but I think all of these definitions are sort of lacking what the pyramid means. So let me quickly explain. A pyramid by definition is a pyramid. It's a equilateral triangle. What it says is there are various different types of tests in the pyramid, and these type of tests are of course dependent on what is the product under test, right? These names listed here as types of tests are for reference. It really depends on what is the product that you are testing and if a certain type of test is going to be applicable for it or not, right? So let's assume that these are the type of tests applicable for my product under test. What the pyramid says is the base is the widest, right? So we need to have the maximum number of tests at the lowest layer of the pyramid, which is the unit test. As you move up the pyramid, I'll have more and more higher level tests, integration, JavaScript, view test, web service test, UI test, and on top of it, there is a explicit mention of manual slash exploratory test. Now I am very hesitant about calling these manual tests because when we say manual test, typically what we mean is there is a detailed test description written, click on this, enter this data, click on this, enter this data, and those are the worst kind of tests that you can have. What is essential is if you have the right test automated, you need to really focus on the exploratory testing, things which may not have been automated because you did not think about that before. In certain weird combinations, you are testing out a particular functionality that is exploratory. You may not have automated those kind of things. So this is what the test pyramid says. Now what happens is as you move from bottom to top of the pyramid, the cost of implementing each test keeps growing. Why? Because at a unit level, I have the code checked out on my machine. I'm running the test against the code base, not necessarily having deployed it to any environment. It is against raw code. I just build it, compile it, and I run tests against that. I do not deploy it anywhere. As I start moving up, so my unit test might be just based on the classes that or modules that I am using, just writing tests against that. For integration tests, I need to start pulling in other modules or components together, and now I need to start running tests against those. So the cost has gone up. I have to wait for someone else to have finished certain things and made it available to me. It might still run from my machine itself, but now I'm dependent on someone else. And that is delay in feedback. I have to wait on someone else. So this is lesser in numbers because remember the pyramid is going tapering towards the top, right? So these are lesser in numbers, but more costly to implement. As I get to JavaScript, again, likewise, JavaScript might not be just one JavaScript that I am using. There could be multiple JavaScripts coming together, and that's how I need to run the test. For view test, web service test, UI test, you need to get more and more of the system together and deploy it for view test, for example. I need to have a UI component on top of it to run the test, which means I need to build it, deploy it maybe locally or elsewhere, and then I'm going to get to it. For UI or web service, I need to have the environment deployed in, the complete code base deployed in some environment. Only then I can run tests against that. So there's a cost involved, and especially if it's a much more complex system, you need to have more and more of these things coming together, the correct environment provision in order to get that. So the cost keeps on growing. Likewise, the value of each test needs to keep on growing also. The unit test is talking about granular code change that you have done. You've changed one parameter of a method for an important logic. You need to have tests around that. Where says, when you go to the view test, you need to think about it from a UI perspective, how is that really coming together? So the value you keep on abstracting of what the test is asserting against, validating against. Likewise for UI. And the bigger problem is time. In order for me to get an environment up with web services and UI deployed, it is going to take much more longer if I've just made one method change in my code. I need to wait for the complete build and deploy cycle to be done to get to this stage where I can run the test. So it takes a long time to deploy. What the pyramid does not really say though, and this is a missing link from existing documentation in my belief, is exactly this. The impact of the inverted triangle is the product, the impact of each test on the product. Unit test is talking, covering the very granular bits of the product, versus as you get to the UI or exploratory of the web service test, it is covering the breadth of the product. So the tests need to be focused in that fashion also. That is very important. The last aspect about this is from the, and this thing is actually messed up, showing correctly on my machine over here, but the view test till the unit test is the technology-facing test, versus the web service UI and exploratory is looking at the business-facing test because you're covering the breadth of the product. What is the business functionality I'm trying to validate? That's what the focus should be. So this is what the test pyramid says. However, if you look at the reality, there are two types of situations I've typically come across. One is called the ice cream cone anti-pattern. It's the same test pyramid. However, it's completely inverted. Why? Because I've got minimal unit test coverage. My maximum focus is on building that massive UI test automation framework, which is very slow, which is very brittle. It doesn't provide the right feedback. It is not up to date with the product functionality, all the issues that we spoke about from functional automation side. And as a result, what happens? There is a larger effort of manual and exploratory testing on that. It's a typical anti-pattern. Over here, I don't know why we still do UI automation. We should just scrap that whole thing and focus on manual in that sense. It would be easier. There's another pattern that we typically see, especially in large organizations, called the dual test pyramid anti-pattern, where there are two test pyramids. Can anyone guess what they might be? No, these are all automated, right? But there are two test pyramids for development and testing, right? Developers are involved in automation, but because of the silos created in the way the organizations function are structured, the developers do not really talk to the testers as much. So they end up building their own test pyramid. The testers are held up building their own test pyramid, which is focused on the specific layers that they're talking about. However, there are two problems over here. One, there could be potentially a lot of duplication across these because there's a silo, there's a wall in between. They're throwing it over the wire, okay, I'm done, now you test it. The bigger problem is that you might miss out on very important scenarios to be automated. That to me is a bigger problem, right? So with all this information about automation, which is one of the key aspects of getting to continuous delivery, if you have all the right type of automation done, so how do we really get to continuous delivery? And this is what I started doing for the bank to get them on the path. Now again, remember this is a very simplistic view of the whole thing. For reference, let's look at the legend. And by the way, all these slides will be available on Slideshare or from my blog. It will also be the link from the conference agenda also. So let's look at the legend, the pyramid. We are focused on five different types of tests. Unit, integration, web service, functional UI, and manual exploratory tests. That's the pyramid layers that we are using. So given this program of scope that we have, what we want to do for the bank, first thing what we have is there's a developer environment where the developer is actually making changes to based on the functionalities required to be implemented. So in the developer pyramid, there's going to be non-existent UI tests because I don't really care about UI at this point in time. I'm making core granular changes. I'm making unit level changes mostly and integration tests, maybe some JavaScript test or sort or not. So my developer environment should focus on these types of tests, mainly the unit and integration. And of course, manual exploratory testing because I'm doing some changes. If I can validate it as quickly as possible, that would be great. Next, when the developer has done this testing and is happy with the changes, the developer commits code into the subversion system or version control system rather, you get into the stubbed environment. The stubbed environment is, I've got all other integration stubbed out. I'm just focused on getting all the developer code from my team into this environment. Everything else has been stubbed out. What type of test can I run over here? I'm going to repeat running the same test, but potentially I can validate some UI over here because I've got all other changes also coming through. It's a separate environment. I'm not blocking off my machine to run the UI test necessarily at this point in time, right? So we've got limited UI tests. Then once my stubbed environment is done, then I say, let me move it on to the semi-integrated environment. Semi-integrated environment, I might say that, okay, let me, from the context of the bank, I'm integrating with other systems from the bank, also at this point in time. In this case, I do not need to run my unit test again because that has already been done in two places. I don't need to repeat that. However, now I'm going to do UI JavaScript integration at this point in time because I'm integrating with these other systems. When I get to the completely integrated environment, over here, I do not want to run my unit test. I do not want to run my integration test because that has already been done. Rather, this is limited in number, not completely excluded, right? Because now I'm integrating with the external to bank systems potentially over here. I want to test those integrations also. And once all that is done, when I get to my pre-prod or UAT environment, I'm not going to run my units. I'm not going to run my integrations. I'm going to do the full validation from web service and UI layer. And of course, in each of these stages, we've got the right type of manual slash exploratory testing done, okay? With me so far. So, what does this leave us with? In order to get early feedback in a complex environment, you need to think about what is going to be the right processes to set up. What are the practices to use based on these processes? And what are the tools that should be used to help implement these practices? Typically, we would go the other way. Hey, wait a minute. Our organization already has an enterprise license for this tool. Let's use this tool. And I know this practice has worked in some other project before. Let's use it. We've got a center of excellence, which recommends these practices. We are going to use it. And then how do we create a process around it? It's eating the other way around. What is required is understand what is to be done. Set up a process based on that, based on the processes, what tools and practices, and what tools will really help you achieve the goals. You might end up with the same state, but it is a much more thought through approach. Based on all this, you identify the correct and appropriate number of environments.  You would potentially need for a stubbed environment, they needed five servers, physical servers. Not really required at that point in time, because no one had even thought about virtualizing it, which could easily be replicated again. So think about the right type of environments. There will be constraints. Question the constraints. Are these valid constraints? Can you work around these constraints? But the right type of environments if you identify, and the right number of each of them, it will help you tremendously in getting that early feedback by running the right type of tests in that. Okay, doing smart automation, using the right tools, practices, identifying the right test is going to be very important, and more important is you identify the right type of test for specific environments. So potentially a UI test, if you have implemented, you could say this is targeted, not for my developer environment, not for stubbed environment, but anything other than these, I want to run these tests. What that means is you have to categorize your tests also correctly. Is it a UI test, is it a JavaScript test, is it an integration test, is it applicable for which environments? You need to start putting much, much more thought into that and make it configurable.  You need to automate all of these things. So just by a click of a button, you say take this code base, deploy it in this type of environment and it will do everything required for it, including running migration scripts or whatever else is there. The biggest thing is testing cannot work in isolation. You cannot keep the testing teams separate from development activities or DevOps activities because otherwise you might miss out on very important context that is to be carried forward. So collaboration is very, very essential. Test consolidation is equally essential because as the product evolves, I have written my first test, as the product evolved, I have written a login test. Now I am in iteration 10 or iteration 100, whatever it might be, do I still need that login test? Ask yourself that question. If it is not required, if it is covered as part of something else, remove it. Just increasing the number of tests is not going to improve the quality of your test, the quality of your product. Consolidation is extremely important and so is maintenance. As the product evolves, am I being smart about updating my test framework? Have I figured out a better way to implement my test data creation, for example, to make it faster to run the test? You have to constantly maintain the framework. You have to constantly refactor the framework as part of regular activities the way developers would have tech tasks to do refactoring and clean up the code base periodically. You have to do that for your test automation code also and test automation, remember, it's not just functional, it's all types of tests. Test prioritization. If you do not prioritize your test correctly, you will not be able to say, I want to run this as smoke test. So if my login test has failed, it doesn't matter if I'm doing my account transfer test at all. It is going to fail at test, the login itself. If you prioritize it correctly and put it in the correct groups where you want to schedule the test runs to be happening, it will give you earlier feedback because in a large system, you would end up with potentially tens of thousands of unit tests. You would end up with hundreds or thousands of functional tests, but if the first five tests can tell you that it doesn't make sense proceeding, this is the worst build possible, you want to get that feedback within a few minutes. You don't want to wait for two days, dozens of tests to run and give the feedback that you could have got in the first five minutes. Okay? You need to have the common repository where all these tests are stored all different types by all different teams. The common repository is essential. Otherwise, I don't know if I'm writing a functional test which does not make sense to me to have it as a functional layer. It makes more sense to have it as a, maybe a view test or a JavaScript test. I can go directly into the code base and see do I have a test for this type? If not, maybe I can add it over there itself instead of having it as a functional UI automation. That is very important because the more tests you push down the pyramid, the faster feedback you will get out of it. So you need to be doing that. And lastly, you need to have a single dashboard to tell you the status of what is the quality of your product. CI dashboards will tell you all my unit test what is happening, all my integrations what is happening. But how can you get a consolidated view of what is the quality of your product from one place? And this has to be transparent and visible to each and every role on the team for sure because as a manager, I would like to know the quality of a product. As a test lead, I want to know the quality of my functional test, for example. As a team lead, I want to know for my stream of work, what is the quality of all tests for my stream? So if you have one dashboard, one place where all these data is collected, you'll be able to make more meaning out of it. So yeah, the question is who has a say in defining the process, what is the right process to set up? Practices and tools as well. If you keep it in the hand of one person, it is going to be a biased view anyway. At the same time, if you have the group like this sitting and thinking about process, it's going to be just chaos, nothing else. So it's important to identify the right roles, the right people from each role, which is going to contribute to what the way of working needs to be. It starts from stakeholders, from where the business requirements are coming, from the technical implementation team or the IT teams to say, who has a say in how it is going to be delivered. So you start thinking about the requirements itself and how you would deliver it. And you would keep going down in that fashion for all roles to say, I need product owners, I need business analysts, I need developers, I need testers, I need these roles from a mainframe team as well as from, let's say, Java implementation team. Because if it has to be a common process for the team, you have to involve the team with it. And when the team comes in together, the right team comes in together, that is where you can really get valuable input while it would work in one case but not the other. Why it would work in Australia but not in India? Culture aspect also becomes very important. So you have to mix in these type of things. Yet, the team can get really large in such cases, right? Especially if you look at a core banking thing. So again, there's a governance structure of sorts. You start off with the top layer to setting up that governance structure to say the stakeholders talking to each other. This is what we agree at a fundamental level how we different partners are going to help you collaborate and achieve the goal. Now this is at a high level, at a 30,000 foot level. At a 10,000 foot level, who is going to be required to help that? Maybe it's the whole core implementation is split up into five different streams. At the stream head level, you would think about the right rules to come together and do that accordingly, right? So it is a combination. It's not just one person who should say that. And of course it has to be in the best interest of the client, not in the best interest of the vendor, how you can continue getting business for 10 years or 20 years going beyond that, right? That becomes vested interest. That usually becomes a point where the value is lost and it's all about money over there, okay? So in interest of time, is the triangle of Pentagon now? Or who said yes, okay? Or is it really a n dimensional polygon? The answer truly lies in how many environments you have identified to say this is what is going to, if I run tests, the right type of test in this environment, it will give me the quickest feedback. It will give the team working on that subset of change, the quickest feedback to know if that change for those changes are good or bad. And you keep on adding that. So the answer truly is it is an n dimensional polygon and I rest my case by proving the magic words. It starts off slow, starting to think from a, let me rephrase what I just said. It definitely starts off slow, but release to production doesn't necessarily mean releasing to the true end user. It is how can you get the changes you have done as quickly as possible to the next level so that others can start interacting with it. If I have a payment gateway, that's what I'm building an API for a payment gateway. The quicker I release it to at least one partner, vendor or consumer of that API, I'll start getting feedback from that, okay? So release to production doesn't necessarily mean it is going live, truly live to the end users. It might be to internal users. It might be to internal systems, but I have processes and practices and tools well aligned to say that if I make a one line change, I will be able to confidently run the test as quickly as possible and say yes, it is ready to go to the next stage. That is what it truly means. So from the bank perspective, their milestones, the five year program, they split into three phases, phase one, phase two, phase three. They targeted a certain section of the business what they would want in each phase. To implement that, the first release was just, we will release it to an internal customer, internal bank employee. So if a customer end user wants to set up a new insurance policy of a specific type, they would call to the call center. The call center employee would use this internal system only for that one type of policy and see if it can be done. That is continuous delivery. It doesn't have to go truly the end state, right? You prove it works. You prove the environment's work. The basic setup is working and then you start scaling it up. So if I quickly go back to this other slide, right? This one. In a developer environment for customizing of the core product team, this pyramid might make sense. For a mainframe system which comes together in a semi-integrated environment, for example, the UI layer does not make sense potentially. Maybe the unit level doesn't make sense. They are just integration tests in some form that you have written. So for each type of system you are building that you are interacting with, that will define their own test pyramid. It is not going to be the same pyramid all across. So there'll be one band of say, potentially there might be 10 different layers over here and five of them might be grayed out for a specific team. It is possible. It is complex to build that kind of thing, but if you start thinking proactively about it, how can I get quickest feedback for the changes I have done, you are in the direction of doing that. So the question is in this case study, how did we be able to achieve it, this type of strategy towards what stage, what value was really derived from that. So this program is still going on. It is going to go on for another three years at least. It is a massive thing. And the value of getting the early feedback, why did the bank want continuous delivery? No one actually asked that question. Why do you want continuous delivery? As I said, bank has limited money. Even though it might have sanctioned X amount of dollars for it, no one from the bank is or the CFO, whoever who's actually going to sign the check is not going to give you that X amount of dollars immediately. He'll say, I'm going to give you 5% of that right now. Show me something, show me some ROI on that. With this approach of continuous delivery, you start thinking in a way of how can I get value quickly out? I see something, I tell my stakeholders, here's the value you're seeing out of it and give me some percentage more. So I can spend more on that next level of investment. So this approach has helped in terms of saying, what is the thinnest slice? How can I show it to some stakeholders, some user base out of it, start deriving value out of it and then take on to the next level? Yeah. So the question is how do you really set up API for stubbed environments? How do you set up stubbed environments? It's a very core automation related section, session and discussion. The stubs, the way you would create it depends on what you are trying to stub. Are you trying to stub a mainframe? Are you trying to stub an API? The technologies would depend on that. There are plenty of open source and commercial tools available for it. There'll be plenty. Yeah. Okay, so sorry. I'll pause this right now and we can head outside and continue the discussion. Thank you very much.