Hi everyone thanks for joining us today. Today we are going to talk about automation test failure classification using machine learning. This is Anuradha Kondori. I have been working as software engineer in test with Expedia Group for more than two years now and I'm joined here today by my colleague Kirtika who is a senior ops and traffic analyst with Expedia Group. So today we are going to talk about how machine learning could help us categorize the automation errors into multiple categories. So let's get started with the introduction section. So first let's talk about the problem statement. So when we look at the automation failures we are not always quite confident if they can be directly reported as actual bugs or not and I can say this from my experience being in testing for almost five years now. So this doesn't essentially mean that the automation test cases are flaky but it could be due to several other reasons like the environment stability issues or unavailability of inventory or it could be the setup issues, runtime issues or in fact even automation issues like locators and so on. So because the automation failures could have other reasons as well teams tend to ignore the results altogether thus defeating the whole purpose of having automation in place. Also engineers spend a lot of time in analyzing the failures to determine if they are actual bugs or not and with the number of failures being very high it becomes quite a difficult to analyze the root cause of each and every failure and these days with microservices architecture and with CI CD pipelines in place it is quite difficult to analyze the failures for each and every build that happens mostly for every commit and to determine if they are actual bugs or not. So this is the problem statement that we are targeting we want to reduce the time that an engineer has to spend to determine if the automation failure is actual bugs or not by analyzing and looking into each and every failure. So now let's talk about the complexity of the problem that we have in hand. So different teams could use multiple frameworks which are implemented in different programming languages depending on their particular requirement and depending upon the framework that was used the automation failure messages could be quite different and another thing is the test specification methodology that is the way the test scenarios are defined and one of the prominent methodologies which we have been used in recent times it's the BDD way of defining the scenarios and this would also determine how the final automation failure would look like and another thing is customized failure messages at multiple levels of the automation system test causing the failure message to look different. So now given an automation failure it could arise from multiple layers so let's talk about them. So the first layer is the automation test code itself and a layer below that we have the libraries using which the automation was built and in a layer below that we have the framework that was used to build the automation and in a layer below that only in case of UI automation test cases we have the selenium web driver and in the bottom most layer we have the programming language that was used to build the automation test cases. So now depending upon the layer from which the failure could arise the automation failure message could look quite different and also there is some level of customization that is possible at each of these layers for the failure message. So now you could see that it is practically very difficult to find a common pattern among the failure messages that could help us in their analysis or gaining insights. So now let's talk about the desired capabilities for the solution that we would want to solve the problem that we have at hand. So obviously the solution should be able to automatically detect patterns among the failure messages to provide us certain insights or the categorization and like we discussed earlier different automation frameworks could be used and hence the solution should remain agnostic and be able to provide the categorization for different frameworks. Also the solution should be extensible to any future frameworks that the teams could use depending on their particular requirement. And the next thing is when we look at automation failure message it could contain two parts like the automation description or the summary and the stack trace. Now we could argue that we could use only the automation failure as input to the solution and ignore the stack trace but from our experience with the automation failures which we have seen within Expedia sometimes the failure description could not be available and sometimes we have seen that the stack trace could have certain useful information that could be used for providing the insights regarding the failure. So because of these reasons we have decided to go ahead with the entire failure message including the stack trace as input to the solution and thus now our solution should have the ability to work with this entire message which is now an unstructured message and next the solution should be able to adapt itself to any new failures that could be arising out of the frameworks for which the solution is already in use and last but not the least the solution should have high reliability because this would determine the confidence with which the teams would use the solutions outputs of analysis for the automation failures. Now keeping in mind all these reasons we have decided to use machine learning as our solution for the problem that we have in hand and to provide us predictions. So now my colleague Kirtika would be talking about the solution more in detail. Everyone this is Kirtika. I've been with Expedia since past two years and working as senior analyst in Ayurveda. So as my colleague Anuradha talked about the problem that we had and the outline that we have created to understand our problem. So with the problem statement and all the analysis we identified like how can we solve our problem. So we created a solution outline and we could able to with the deep analysis we could able to identify that this problem is a good fit for the machine learning because our data is like that our data is basically promoting it to the classification category of machine learning because we already knew our data we are very much familiar with our data and we have like we know that what we want from the data. So we widely has the four classes in all the test error messages that we get. So one is the inventory. So whenever there is some item unavailable then an inventory error shoots up. So and then there is an automation error as Anuradha previously talked about like whenever there is a non-existing element on the page or something like that. Then there comes the environment. There are many downstream and the backing services that are running to power up the UI and everything. So that is actually if something goes wrong with those downstream services then environment error shoots up. Then there is a product. So it is actually the error that a customer faces on the final UI of our Expedia site. So before moving ahead with the ML we have to plan it accordingly because as you can understand that there is multiple steps to you know solve a problem via ML we can't just throw the problem to ML. So first and the very important thing of this solution outline is we have to collect errors and classify them manually to provide a source of truth to our model. Then we have to identify the correct algorithms. So there is multiple algorithm available for the problems like there are text classification algorithm, there are categorization algorithm, there are multiple algorithm I would say. Then we have to do provide the proof of concept to finally you know upgrade our solution outline into a product. So after that we create a machine learning end to end flow and then automate everything because right now it is a very manual process where a tester has to go there and has to select one of the category on which that particular error message belongs. Then we have to keep adding more pre classified data and keep fine tuning. This step is very important for any machine learning technique that we are using or any machine learning environment if we are setting up because if data gets stale then it is more prone to you know more prone towards biasing and overfitting. Then there is then we at the very end consume the model. So we are planning to you know give it wherever in our Expedia. Expedia is a very big organization and there are there is and we are producing a lot of data. So we are basically will be able to make it robust and scalable to consume the ML model. So in the very first step the data collection step so as you could see on your screen this is how one of the error message that looks like. So this so one of the test framework pulls this message and put it into the dashboard and and tester has to go there and select the categories manually by understanding the message like from in which category it falls into. So this is how and from this this MongoDB this is basically the MongoDB back end. So we are basically collecting our data from MongoDB and we are classifying them manually for this for to provide the source of growth or to create our training data. So this is our data preparation. This is how our data looks like. So this is an error message I could see as you could see we don't have much dimensions in our data our data is like two-dimensional data with multi-class with multi-class problem. So this is an error message. One column is an error message. Another dimension is like prediction where the predicting category or the where the this error message falls into. In next step we have to do some of the data cleaning like there are multiple punctuation multiple html tags that comes in the message but we are to identify like exactly where that message belongs to like for example the inventory if something is out of if something is out of stock if it is not able to you know it is not able to call the inventory for a particular item then there must be a particular error message that is thrown by the app but the thing is that in the in the tests that are running we are getting the whole stack trace so we'll be dealing with the stack trace in when we will mature our model but as for the proof of concept we have removed all the all the extra punctuation and the tags and only kept the important message or the plain text message that was relevant. Next we have evaluated multiple algorithms for on our cleaned data so as you could see on your screen like there are multiple algorithms that we have created that we have tested on like native ways and support vector and so on so we finally was able to stop at the support vector machine with the term frequency feature so it was it was giving us an accuracy of 72 percent so it was good for the proof of concept but as you know it is not very good for the production and it is not reliable for an engineer because a human can do the error human can you know human human's error rate is around 95 or 96 percent of he is working as a job doing a job and and and doing it regularly so why he would use our he or she would use our product so we have to identify like why the accuracy is not up to the mark why it is not reliable and so we have to so now so far we have evaluated the model now we have to evaluate our data like what is wrong with data so as we have evaluated our data was like not in very balanced shape like few of the categories was having very little values while a few of the categories having huge value for example product does not have much does not have much errors instead the I would say automation or some internal error will also shoot up like have the multiple errors so in the end so we could identify that there is an data challenge and there is an issue of biasing and overfitting of the model so next we have to fine-tune our ML model and also you know clean the data so to so for cleaning up for the cleaning of the data we have like used multiple techniques like consensus cost sensitive general techniques, synthetic samples, tree-based algorithms, semi-supervised learning and all so I will go by one by one so cost sensitive learning is like assigning the cost and everything though the model assigned the cost itself on the whenever whenever it classifies that will that comes under fine-tuning but to improve our data to we have assigned we have assigned constants to our to the most favored word in most favored word in an error message so there is usually we have already seen multiple errors and it's a very it's a very matured Expedia is a very matured platform so we know that what what what are the patterns already so next the next step is the synthetic samples so generating synthetic sample is basically whenever the bars are not meeting so we are basically filling up extra space of commons so that we'll be able to you know uh uh level level up the all the categories now there are tree-based and model and sampling it is basically the machine learning technique uh machine learning modeling technique but we have used it to uh to balance out the data by you know uh adding the feedback loop and all so semi-supervised learning is basically so so far we were going through our supervised learning but in the end we identified that few of the patterns to actually tackle complete message we have to go by and understanding the patterns of the message like uh like to and to find out the similarity factor between the two uh message or the two records or the two data points for that we had to go by the semi-supervised learning furthermore the model tuning parts now we have like balanced out our data somehow and we were like uh we are like uh moving forward with the model tuning so uh we used uh the model and sampling at we were maturing our uh technique so we uh ended up with the model and sampling techniques where we combine the model break the model and get the result out of it and then we again you know keep on keep on keep on there is multiple iterations that we do to get our results so uh we have used like random forest voting classifier and so on but in the end we have uh we stopped at XGBoost where uh our accuracy was like very much promising and we were able to uh you know and it was very good for the production so currently we are using like semi-supervised learning for data cleaning and for imbalance part and the XGBoost classifier that could classify our errors and which will actually give us the better recall and the precession values so this is the final uh fine tuning uh that uh that gave us that is a accuracy score of 97 percent and precession of around 69 so this is 69 percent precession is for the product and and so the further we have divided the category wise uh accuracy and all so uh as you could see the hyper the most important thing is the hyper parameter values that we have assigned to the fine tuning and we have identified uh during the process for example the maximum depth depth of the tree is equal to 13 and then uh learning rate what should be the learning rate what should be the feature selection percent etc so there are multiple things that are under that comes uh under the fine tuning part so it is a continuous process yes because we are uh feeding more and more data and the new patterns are also coming new errors are also coming as the new applications are being developed uh continuously so yes we are uh uh to be a keep on so these uh parameters are prone to change and also uh for the improvement also prone to change for the improvement as well so our final uh metrics with the 30 percent split and the two different seed like there are 30 percent will be the training data and the testing data and so it is like the we have got the accuracy of 97 percent which is this is very very very promising so uh so so and it it was good and it was uh it was good to put into the production so that is why our next step is to deploy our model and set up an end point so that everybody couldn't use it so that part will be taken up by my colleague Anuradha Anuradha over to you thanks Kirtika um everyone so uh now we are going to talk about uh regarding the model deployment and end point setup so uh we want our trained model to be exposed as a service to an api end point where the error message uh could be given as input to the api end point and the api end point could talk to the trained and deployed model and get back the inference or the prediction and send it back as a api response we want to expose the model as a service to for its simplicity of usage because the model could be invoked to get the prediction as an api endpoint from any piece of code so now let's talk about in detail how we have achieved this so the primary uh technology that we have used to uh to expose the uh model and to uh train and deploy it is uh sage maker so sage maker uh allows us to uh put our model code into uh uh into a custom container which could be trained and uh run within the sage maker environment so the first step would be to uh create the docker image with our model code so uh sage maker provides a default container structure which already has a flask uh service that serves as the web app and an engines uh server and all we need to do is put in our model training and inference code and uh build the docker image so once the docker image is built there's an invocation end point uh that gets exposed within the uh docker image so once the docker image is built uh sage maker uh provides us uh with training and uh and inference uh scripts using which we could test on local and these scripts uh basically uh mimic the way how sage maker would call the training and inference process within the actual sage maker environment so okay now once we have uh built our image and tested the training and inference part the next step would be to uh push the model uh image to amazon ecr which is the elastic container registry and in the next step uh we want to make our training data uh available to sage maker and we have chosen amazon s3 for this purpose so we put our training data into s3 and uh in the next step we can go ahead and create the sage maker uh training job so the uh the inputs that the sage maker would need is the full path to the uh model docker image that we have pushed earlier to ecr along with any uh tag names which is useful to uniquely identify the particular model image and the second input is the training data path in s3 and the third input is the desired location where the trained model artifacts should be stored so uh now with these inputs once the sage maker training job is created sage maker run state training and the trained model artifacts are pushed to uh s3 so uh these trained model artifacts are nothing but it's a it'll be in a pickle file format which is like a serialized form of the trained model now once we have the trained model artifacts available in s3 in the next step we can go ahead and create this sage maker environment the prerequisite for invoking this sage maker end end point is it can be invoked only within a sage maker runtime environment so as the next step we can go ahead and test this sage maker endpoint within the sage maker runtime environment to see if you are able to fetch the predictions from the deployed model so now as a next step uh we want to make the model available as an endpoint so we have achieved this using aws api gateway and aws lambda so the lambda would have the code to create this sage maker runtime environment uh invoke the sage maker endpoint with the error message input that it gets from the api gateway and the sage maker endpoint would in turn talk to the trained and uh trained model and get back the uh inference uh which is nothing but the and send it back to aws lambda so lambda would append this prediction to the error message that was received as input and send it back to api gateway and api gateway uh sends this back as the api response so here you could see the example api response where uh this is the input error message that was received and uh the environment testing prediction that was populated using the model endpoint so now that we have the uh model uh created let's talk about uh the model endpoint uh usage uh within xpedia so uh within xpedia the automation test cases are are run as part of uh ci cd pipeline which is set up using Jenkins and for every test case run there's an uh there's a test record that that gets created with all the uh information regarding that particular test case run which gets pushed to uh elastic search and can be visualized using kibana so now let's uh look at a sample snapshot of uh the test run data from uh kibana uh in this particular uh data i have queried uh only for the scenarios which have failed and here you can see the field scenario name and the uh status which obviously is failed here and the line of business uh for which this particular scenario was executed and the next one is the error message or the failure message and uh we have here a field called an uh page failure message we will talk about this in a bit and uh we have the framework uh field uh using which the the particular test case was built and finally we have the prediction field and this is the field that gets populated for each test case run from the model endpoint that was created so now let's talk about in detail how these predictions are getting populated so fetching the predictions from the model uh endpoint so for every uh test case run depending on the framework using which the automation was built there's a result file that gets generated uh either in jason or uh our html format and for every uh test case run we have a processing module that gets uh invoked and this processing module is uh responsible for forgetting all the details uh regarding the particular test case run and creating a json record with all the details which is finally pushed to uh elastic search so now let's talk about what happens when the uh scenario fails so when the scenario fails this processing module would get the failure message or the error message from the uh from the result file and now there's another optional uh input called page failure message so in case of xperia site uh pages uh uh again this is applicable only in case of uh ui automation test cases so whenever there's a back end process failure or a back end service failure sometimes there's a custom a page failure message that gets displayed uh to the user and which we call as the page failure message so in case of uh ui automation test cases whenever this page failure message is available and when it was included in the training data uh we have seen uh that the model performed really well as it it could gain certain insights from this custom message uh which is taken from the ui itself so uh because of this reason uh even in the uh inference or the prediction part as well the processing module uh fetches this page failure uh message from the browser for the particular test case run whenever available and append set to the beginning of the error message and sends this uh whole message as input to the modal api and uh and gets back the uh prediction from the modal api and appends it to the adjacent record that was already uh created and this whole record is now stored in uh elastic search so now uh within uh xperia these are the current frameworks that are being in use and for each of the uh frameworks we have processing modules implemented which are able to uh fetch predictions from the modal endpoint and populate the predictions for the uh test case failures so the first framework is scala test which is implemented using scala programming language and the second one is night word js which is implemented using javascript programming language and the third one is kukumbar whose step definitions uh implementation is done using uh ruby okay now that we have talked about the uh modal endpoint setup and usage let's look at some of the uh prediction category uh examples from within xperia test case failures so the first category we have is automation so like we talked about earlier automation uh represents any locator issues or setup issues or runtime issues that are caused so and also please note that in these examples for simplicity purposes i have included only the failure description or the summary and not the stack trace so okay now let's go through these examples from left to right and top to bottom in that particular order so the first example here says that a null pointer exception was thrown which is a runtime issue and hence classified as automation and the second example says that uh element is not clickable uh could be locator issue and uh categorized as automation here and here it says that the automation was trying to access a method for a null uh element again uh automation issue in the fourth example it says that a particular uh value is uh not found from the select list in the ui again an automation issue and in the fifth example it says that it's unable to uh locate a particular uh element or the object was unknown uh again an automation uh issue and in the last example we can say see that it says that the session was terminated uh because of credit exception and the selenium uh web driver so this is an example of setup issues again and uh automation categorization so now let's look at the next category which is environment so uh like we discussed earlier uh automation is usually uh run on pre-production environments or the test uh environments and these could have certain stability issues like capacity issues or back-end service uh performing stability issues uh which might cause automation failures and this is what we expect to be categorized as the environment so let's look at the first example here it says that the automation is not on the desired page which is the uh hotel search page calls it a runtime error but it's an environment stability issue and the second example here uh says that uh even post multiple retries and element was not formed and it's categorized as environment here so like you could see here there could be certain amount of overlap uh within the uh automation failure description like it might look very similar but might get categorized into different categories again this is completely dependent on the training data that was fed to the model uh it could be that the model got certain uh insights or keywords from the particular failure message to to indicate that it belongs to a particular category okay so let's continue with the uh example so here it says that navigation to the trips url failed again calls it a runtime error but it could be an environment stability issue and the next example says that the failure message was malformed again an environment issue and the fifth example uh says that there's no uh room type uh i mean required room type that is uh that is expected by the automation from the hotel uh infocyte page or details page again an environment issue and the last example uh again says that the automation is not on the uh expected page now let's look at the next category uh which is inventory so like we discussed earlier uh automation is run on pre-production environments and the test environments inventory within xpedia is quite different uh than the production uh inventory that is available and in the test environment uh inventory sometimes uh the uh the items could not be available as required by the automation while it is running which could be causing failures so this is expected to be categorized as inventory failures so let's look at some of the examples the first example here says that the uh search result section itself is not visible definitely an inventory issue and a second example says that the uh hotel search section element is not available again an inventory issue and here uh in these two examples what you see as uh prefixed with uh alert error uh these are actually the page failure messages that we talked about earlier which are coming from the uh ui so now let's look at these examples here it says that we have a whole lot of rooms for the particular criteria that the automation is looking for clearly an uh inventory issue and the next example uh says that there are no results on the page although it calls it calls it a runtime error but it's on the flight page basically the inventory is not available and in the next example it uh it says that uh it's not able to find a particular trip that is required again an inventory issue and the final example is that the uh flight search is not working again an inventory issue now let's look at the final category which is product which is expected to represent the actual bugs that are to be reported so the first example says that certain expected sections are not uh uh as expected on the hsr page which is the hotel search results page and hence categorize this product or a bug and here uh in this example and some of the following examples these are custom error messages which are coming from the automation layer or from any libraries that the automation could be using so here it says that a particular section is not working as expected categorize this product and in the next example it says that the particular condition is false along with a message to say uh why the verification has failed and the fourth example it says that an expected message is not displayed on the page and in the fifth example it is termed as an assertion failed error and says that a particular option is not in the expected condition and in the final example it says that the search dates on the ui are not as expected by the automation again categorized as product so now that we have seen the examples based on the current data that we have in elastic search this is how the various predictions or distribution looks like for the automation test failures so the so most of the predictions are our environment category which accounts for around 65 to 70 percent of the failures and the next comes the automation category which accounts for around 45 to 55 percent of the failures and next comes the product category which denotes the actual bugs and it accounts for around 35 to 40 percent of the failures and the last category is inventory which accounts for around 15 to 20 percent of the failures so now my colleague Kirtika would talk about the uh future work that we could further do in this particular space Kirtika over to you so as per the future work uh we have like uh like we have only tested out with the test results that were uh you know focused on the product but there are multiple uh log mechanisms that are being used in our in our uh you know in our platform for example the js error javascript error that are produced and uh and you know fetched by selenium web driver uh there so those errors can be used to you know further train the model and further identify the issue and further classify like where the where the platform is failing and where the and what is actually the error and then there is an exception in a splunk so all the event logs are uh all the transaction and event logs are being logged under the splunk so we are thinking of you know fetching data from a splunk as well and working on that so then there is a haystack trace so haystack is basically the Expedia inbuilt tool it is now open source so it is trending tool basically and the trend analyzing tool so there are multiple failures success rate uh and uh and uh you know the anomalies that are being uh surfaced by this tool and logged under it we are also gonna fetch these anomalies and you know failures and everything and we'll put into our data and see like how the model and the automation will work so basically our idea is to you know use all the data points for the uh and and and uh from the across the platform wherever the error and wherever the failure messages uh are you know producing wherever the environment is failing and everything so we are basically fetching everything and we are making it more scalable more robust more uh more global I would say so uh that is all for the future work and uh uh and we are now open to your questions thank you for the thank you for the thank you for listening and everything uh this is all that we needed to present thank you for keeping calm we have I have seen many Q&As I know you guys are interesting this is an interesting topic I totally understand and uh it's uh it's like whenever we worked upon so uh it was like for us uh to you know uh interesting to implement as well so now let's handle your questions Manoj sure uh there was a pretty uh interesting talk as as we have been seeing on the uh panel yeah I was like only one right cool um the most upvoted question that I see here is how much data you use to classify and you know and train your model initially okay uh Kirtika can I take the question yeah sure yeah okay so uh uh initially we have actually started off with very uh less data we had because uh the like Kirtika talked about the pre-classification is a manual process and uh hence uh uh you know we had to rely on the teams to provide the classification so the rows were almost like 1k to 2k uh and the last thing that we have were around 3k rows but then what we did is like Kirtika talked about we have involved semi-supervised learning where we have put the uh I mean taken all the unlabeled rows which is all the failure messages without the categorization because we have lots of data so uh we have taken that and uh took a promising uh model to categorize it and fed it back as training data thus creating uh more data for the model to work with and this has definitely given us uh given us significant improvement in the performance uh plus I would like to add on it like after uh all the automation that we have did uh we are now getting around I would say with more than 60,000 data per minute for training very interesting very interesting um the next question that I see is from Abraham he's asking hadn't such a tool do you still use some manual analysis if so then when and how do you know when use manual analysis against fully trusting such tool do you miss bugs and what is the ratio that you see uh Kirtika shall I take it yeah sure okay uh so uh like we saw I mean although the uh precession and recall are the scores that we're currently relying on and it's doing quite good and uh so this like we said we want to uh or uh we want to highlight that we are using this to reduce the time uh that an engineer has to spend not completely replacing it because in some cases it could be that although it gives its initial analysis or the prediction in some cases it could be that the engineer might have to still spend some time to see if the category was different uh or even when it is categorized as product which is the actual bug it could not be a bug okay so uh this would still need and we are taking all such again manually set rows where they say it is the wrong value and set it to the right value and giving it back as feedback to the model so that the metrics and the performance could could further improve so uh coming to the other part of the question where you ask like did it miss any bugs so so far we are seeing that the categorization for product is good it is around 35 to 45 40 percent of the data obviously like maybe few intermittent issues could go and be categorized as environment or automation or something like that but apart from that I think it's not missing most of the bugs great I will take one last question as we are nearly there so other questions are mostly in some of the internal details and one question that I see most common is is this shareable uh I have plans to open source or is it already open sourced okay it's not currently open sourced where we were actually planning to do it before the presentation but we were on a time crunch so we would be we are planning to open source it soon with all the documentation required for the setup and stuff as well so probably uh yeah it will get open source soon that is a really great news to hear that um all right thank you very much Anuradha and Ketika for your time and such an interesting talk


The COVID-19 pandemic continues to affect the well-being and lifestyle of people all over the world. It is now known that pre-existing health conditions such as high blood pressure, obesity, and diabetes can enhance the severity of the disease and people's susceptibility to infection. Curiously, however, relatively lower rates of infection and mortality have been registered in the Gulf states, where those very health conditions are extremely prevalent. A new study by researchers from Dasman Diabetes Institute in Kuwait suggests that variations in population-wide genetics could account for that disparity. The researchers compared genetic data for more than 1,300 individuals of Middle Eastern origin, with those of people from Europe, East Asia, and Africa. They were looking for variations in three genes considered critical to infection by SARS-CoV-2, the coronavirus that causes COVID-19. These genes are referred to as ACE2, Tempris2, and Furrin. ACE2 codes for a protein receptor that makes the entry of the coronavirus into human cells possible. Tempris2 and Furrin code for proteins that facilitate that invasion. The team's analysis revealed that one variant of ACE2 that might enhance coronavirus infection was most prevalent among Europeans and significantly less common in individuals from Kuwait and Qatar. Frequencies of this variant appeared to correlate with the low mortality rates in the corresponding countries. A similar trend was observed for the Furrin gene. In this case, however, the team tracked a variant of Furrin that could possibly weaken coronavirus infection. That variant was most prevalent in Middle Eastern populations, indicating a lower risk of viral infection. Genetic variation data for the gene Tempris2 were inconclusive. Overall, the ACE2 and Furrin variants detected in this study appear to have a protective effect against SARS-CoV-2 infection among Middle Eastern populations, individuals whose general health profile would suggest otherwise. Importantly, the study did not account for country-specific factors that could affect the data. These include the extent of viral testing, socio-economical status and access to healthcare services, and the higher burden of other diseases. The findings do appear to deliver an important message. Patient genetics deserve a closer look to understand how the current pandemic might unfold in different populations and to deliver personalized care.


I graduated six months ago and that was also the moment in time when me and five other students here from this university started an IT cooperation called the Vodega. We built software for companies and yeah we tried to do that in a smart fashion. One of our products is indeed in the health care sector. Over the next 40 years health costs will nearly double. Also the population of the Netherlands is aging. Therefore there's a growing problem and that is the capacity matching in the health care industry. We were looking for a way to obtain data from hospitals in the Netherlands on an automated way. Together we came to a solution to this problem. And now we obtain the data and within two or three days after the data are here they're analyzed and they are brought back to the hospitals and they can see themselves how many people have to train. The hospitals can really coordinate their education policy. For me entrepreneurship is a state of mind. We really love what we do. The work that Duodega has done for our organization has a meaningful society. But we also want to be a company that's able to build on our own ideas and bring our own products to market. We are fast. We develop fast. We are quick on our feet. They worked on time and within the budget. Our IT makes our world a little bit smarter. We make sure that the people that are educated are not educated for an employment.


Good afternoon. Welcome to condo insider and I'm your host Jane Sugimura And we are here to talk about condominium issues mainly because one-third of the people in the state of Hawaii live in condominiums and We you know have these weekly programs to talk about issues that affect people who live in condominiums and their management Today I have as my guest Cameron de Costa and Cameron works for C4 management Cameron. Can you tell me what you do for C4 management? Well, I'm in it one of their elite site managers been working with C4 for about 12 years now, and I worked with R&J services prior to that C4 management does pretty much what a site manager Jer does is a daily operations of all the the pool the janitorial overseas contractors for the landscaping If you're gonna do asphalt repair Any kind of minor projects? No, no large projects, of course Pretty much make sure that your building is running smoothly So I've been doing that for quite a while now And you do this for condominiums and for community associations, right? Correct, correct. High rises and associations such as Ewa Beach that has multi-family homes structure and and C4 I understand You do temporary property management so that you can go in and take over for what when the the resident manager Has to take a leave or is on vacation. You can just go in and take over for that temporary Correct, correct. I'm the owner of the company Angel de Costa um Has been doing this for a few years now where we do come in when resident managers go on Sick leave or vacation a lot of resident managers, you know Family emergencies have to leave for two weeks to a month Etc. So we'll come in in the blind take a set of keys and run a building With the experience of our crew We're able to make that happen. We're unique because When you purchase a C4 contract a monthly contract with us, you don't just get the the resident manager So you'll get myself, which is like a resident manager. I'm there all the time I make sure all your contracts are flowing but in the background you still have a Lot of other people that help out like you said. We have electricians we got Handyman we got a lot of other site managers who's experienced different things I'm more mechanical So I will know a lot about your domestic water pumps and your booster pumps your heat pumps and things like that where a lot of our other managers will know more about landscape and Everything else and so this is like a collective effort. So you get everything. Yeah. Yeah. Yeah So you get everybody so that's how we make a building run is we'll bring in different What's needed if certain buildings need certain things a lot of times most people just need Janitorial someone the streets in the chairs someone to clean the bathrooms and check the pool then we'll bring that in if you need oversight of Contractors so you need someone to come in and oversee your employees that are doing all the work Make sure you meet with them every Monday. Let them know what their job scope is for the week, etc Then you'll bring in someone like myself more of a site manager So whatever the association needs we're usually able to provide Okay, and today we're going to be talking about an issue.  No, I did not. Okay. That's probably because you're too young Well, but let me just tell you that maybe 10 15 years ago We started to hear issues. I mean to condominium attorneys Were come and and people in the business managers were talking about Problems that they were experiencing in condominiums where you had elderly people and maybe some of them had dementia And they'd go walking in the hallways, you know half naked or you know they just didn't really understand what was happening and They weren't taking care of their units and maybe there was hoarding and and and and and the situation with these unit owners was affecting other people in the building mainly because if they didn't take care of their units you ended up with bugs and hoarding causes, you know fire hazards and so people started complaining and And a bunch of us went to the legislature and asked the legislature to enact some Laws to allow condominiums to deal with these Situations because we had no authority. I mean, it's not like we could go in and and do anything for these people who were Having these aging issues Especially if they had no family members and many of them had no family members and see that was part of the problem They had no family members here maybe their friends had all died and so they really had no support and So the legislature did enact the statute that said that if condominiums if the association and their staff Do help these people they're not going to be held liable for any actions that they take and one of the things that they They did is they allowed the associations to call social service agencies to come in and you know with social workers and Who would then deal with these residents and help them get services that? Would help them. I mean maybe they need somebody to go and get the groceries or take them to doctor's appointments Or even just to come once a week in the units and you know for all of that You kind of need the management Assistants correct right because you know when you have vendors because that means you have vendors coming into the building and they would need parking and So we would have to cooperate with with the managers to get those types of such You know services to accommodate the vendors who come and help these People and some of them weren't elderly. Some of them were disabled, but you know, they needed the services So anyway, the law was changed so that you know those the the condominium could Association could take those steps now based on your experience. Have you Have you have you? Personally seen any type of situation that would fit within this aging in place Well, I only have a few small incidents. Okay relating to this Many many it's really really small ones such as bugs like you like you mentioned hoarding Elderly not being able to take care of themselves Meaning, you know after they're done eating dinner cleaning their own dishes putting it back cooking their own food and such so a lot of times we'll see them ordering a lot of fast food or people delivering food for them and There might not even be able to go to the trash cans themselves, so they won't take out their own rubbish Therefore it's sitting there, you know days weeks months sometimes even years and it does cause an infestation of bugs and You know normally when we find out about this kind of stuff It's normally other units surrounding it will start having problems with roaches and so What do they do do the residents complain to them? Yeah, they'll definitely complain about it and we'll our first action is normally the building will get on to some type of monthly Service and we'll treat the units around it and and try to minimize it From affecting the entire building try to keep it into that one unit It's very difficult like you said because you don't know, you know You those those homes belong to the person and you can't just enter and go and treat that unit without permission and so That's that's good to hear about this This law and 514 be that does help the elderly and people without family Because we were pretty much stuck at that where we can treat all around it with permission of the units around that want help but when it comes to that individual unit, they may or may not let us in and To get forced entry, you know, it really has to be an emergency and we really have to know that something is wrong in that unit whether the person's you know passed away in the unit or it's it's a real health hazard and To get to that point. It's really really really difficult and there's not too many services out there that will help us when you're When you suspect something Yeah, the bugs do the bugs do they complain about the bugs coming out into the open? I mean Yeah, and I don't know Yes, they do they definitely run run through the walls and and go into the other units Especially when there's not a food source in that unit anymore And they'll start to spread and so what when you get a complaint about bugs and what what I mean What have you you know, what are your on behalf of the association? We normally only can treat this the common areas and that that's because everybody who pays their association fee Doesn't want to pay for you to treat individual units So we only can treat the the association and that only gets you so far and then after that We usually will get the units around it a really good rate whether it's $15 $20 a unit to To offer the service inside their unit Wow We're having treatment of the common area and in that way we're able to get into those units Really evaluate it treat it and hopefully stop it at that but to get into the unit if if the person that's causing the disturbance wants help Then we're always willing to help management will get on the phone. We'll have all the services come out and help them But that's 99% of the time not the not what happens about the Department of Health You know, we just haven't I personally haven't had too many incidents where I had to call on Department of Health The few times that we did we were we we didn't get any help because Maybe the complaint wasn't large enough or it's too, you know, we just didn't have enough evidence So so I mean is but if the department Has there been a situation where the Department of Health has come out? I mean you can call them Not for me. No, I haven't I haven't had them come out for any any large situation before okay Is there any government agency that you know of that you can call with if you know the adult protective services is another one but I think I think they're all kind of on the same line where the person has to want help and they have to Be willing to contact this service and get a social worker or get help For us to just give it to them. It's pretty much impossible Every time I've called and tried to get somebody help I was unable to So so this is a very frustrating situation. Yes very very frustrating and it and you know when when you know Let's say somebody's like you say has a huge infestation in their unit and causing a lot of problems for the whole building It's very frustrating because we can't just pick the locks go in there and Do what we need to do what I mean you can there's a fine line. There's a really fine line and So that's and that's where we don't want to cross the line because we don't want to be liable and we want to make sure that You know we though usually the homeowners Don't want us to come in and they're very clear about that so we don't want to we don't if we cross the line You know there's there's liability and and and and and you know when you have a situation like this like with bugs or With smell like you said people you know have the food and they maybe they don't clean up And do people complain about the smell? Yeah, yeah, so that's one of the the biggest signs is the smell and I Say two out of three of the situations. I've been in we've we've smelt it before we actually knew there was a problem And we're like wow what's that smell and you know it gets stronger and stronger than you're like, okay Some something needs to be done. There's either a roach bomb Hopefully someone didn't pass away in that unit. So you do you do your duty and you try to contact that unit, you know Of course you hopefully you get a hold of that person if you don't then at that point we call HPD We go we we enter and we make sure that that person's okay. We call all the emergency Numbers that we have which is which leads me to the registration of units is very very very very important that you put all of your Family information on that type of form because as management we we use that that's you're talking about the registration Yes, every owner must turn in or tenant who lives in a building fills out for the management Correct correct And that's one of our our largest things we do is make sure that that's up to date every single year and that we have As much information as possible and a lot of people don't want to give it out You know, why does management need to know this? Why does management need to know who my parents are and what their phone numbers are my my brother and sister that live in Vegas? It doesn't you know, they don't live here. They don't need that information But it's really really important especially if your parents are elderly and you don't live on the island And this is the way that they you can get them help. Yes, but right now we're gonna have to take a break So we'll come back. We'll come back. We'll take we'll take a one-minute break. I think Aloha everyone, I hope you've been watching think tech Hawaii But I'm here to invite you to watch me on Viva Hawaii every Monday at 3 p.m I'm waiting for you mahalo Hello, this is Martin Despeng. I want to get you get excited about my new show Which is humane architecture for Hawaii and beyond we're gonna broadcast on Tuesdays 5 p.m. Here on think tech Hawaii Hi, I'm Donna Blanchard I'm the host of center stage which is on Wednesdays at 2 o'clock here on think tech on center stage I talk with artists about not only what they do and how they do it But the meat of the conversation for me is why they do it why we go through this a lot of us are not making our Livings doing this and a lot of us would do this with our last dying breath if we had to that choice And that's what I love to talk to people about I hope you enjoy watching it And I hope you get inspired because there's an artist inside g2 join us on center stage at 2 o'clock on Wednesdays Okay, welcome back, and we are talking to Cameron Dicosta about his experiences with aging in place and you were talking about you know the Frustration that you as a manager have to face when you're dealing with somebody who may have some Mental or emotional problems or you know maybe you know they don't understand what's happening and maybe maybe because they're you know getting older and and and And and and some of the problems that you you mentioned were you know the infestation of bugs and the smell So How do you deal with this? You know what means how I mean what does the association? How does the association deal with these problems? You know it's really case by case because there's the large spectrum of things that could be happening in that unit and like you said the fine line of of Doing your due diligence and doing your job and trying to make sure that person's safe first of all And you know are they in an environment that could be a health hazard to them and the the units around them and and How do you follow protocol based off of your bylaws and such so you know It's pretty simple to do the first few steps of like sending a letter or contact in the unit if you get contact And they're not able to remedy the smell whether it's like set pest control or and if I do what when you when you send the letter You're basically sending them a letter saying we've been receiving these complaints Correct. Well first we do the calling and then if they don't remedy it, you know quickly then we send the letter that way we can make sure we We cover ourselves and And the letter will normally state that yeah You're getting numerous complaints from adjacent units and you need to take care of the situation Or we may be requesting that we enter and inspect and we have had to do that numerous times where We smell something from a unit. We send them a letter We schedule an appointment with them Requesting that we we just make sure that it's safe and it's a clean environment And we go in and we just inspect and a lot of times, you know, they know we're coming and they'll clean it up they'll have their their everything boxed and cleaned and they'll shampoo the carpets and You know do what they can to make it past and we'll come in and we'll No further for the people that don't you don't have a problem with the people you who do have the problem A lot of the hoarders they know they know that they're saving a lot of things and they more than that They need and it's stacking up and they're throwing things in the corner and sometimes it's just it It causes smells because they didn't know that they had something in a bag That was perishable and they just didn't take care of it and it's and it's they're not doing it on purpose And when they're when they live in that unit and they got pets, you know sometimes there's feces and things like that they'd clean it up but maybe they don't clean it up really good and it Occurs every 24 hours, you know the pet might pee on the carpet or poop on the carpet Continuously, and you know, we might be months down the line and we start smelling something So we're like hey, you know, we want to inspect your unit or you need to take care of this smell And then they usually do most most of the time those kind of items They'll they'll come and they'll take care of it most most homeowners are good about that It's the elderly ones that they might not be able to and they might not know what they need to do And they just live with the situation Where like you said their their kitchen is full of You know Dinnerware that hasn't been cleaned and their refrigerator and fridge have they haven't used it in years So whatever was in there when they when they stopped walking or you know, it's still in there And and you you you did have an incident where you where there was an elderly person who was it and into denial Yes, and she was not cooperating correct in fact you weren't even allowed to get in Correct, and you contacted her family members, correct, and the family members will also try to get Help from social welfare agencies and what what happened correct Well, we were thankful in this situation where the the family was willing to help and they did come down from the mainland and Try to talk their way into the unit. It was a little bit of hoarding and a little bit of a little bit of everything on this on this situation she was elderly wasn't mobile and Definitely a hoarder as well. So we had a little bit of everything the family Themselves haven't spoke with with this their mother and over a year as well So it wasn't a good situation, but we at least we had help from family. So the family did come down and try to get in Unfortunately, the mother did not give access to even her own family Kept denying us access made a lot of promises to us that she was going to be taking care of the situation and that she did schedule with different Pest control companies and she even gave us dates, but they never did show up In fact what you found out when when and and and this incident came to your attention because the residents The neighbors were complaining about the noise. I mean about the smell. Yes. Yes. Yes And you found out that there were neighbors who were actually enabling this person Correct her groceries and running her errands. Yeah Like you said the the elderly that live in the building They might have bought in the building when they were in their 30s or 40s and now the building's 40 years old So, you know, they've been living there all this time and they have a bunch of friends all their neighbors So nobody really wants to complain about them. Even though they know there is a problem They they usually shop for these people take out their trash for them do a lot of a lot of helpful things So that's why we get into those situations is because they're they're helping but at the same time they're enabling because You know while they're enabling them that person doesn't seek help Correct that person thinks she has it all under control and they they're like you said they're getting someone to deliver them food and take out their trash And so what's going on in their unit? They feel like They're they just stay like that for years and years and that just keep that just keeps happening and happening Meanwhile, they're not they're not Cleaning or in or out of their unit. They just stay there. They don't see doctors a lot of them So it it gets as they get older and without family on island It really gets tough tough for management and and with this particular person You were the family was finally able to get removed her take her back to the mainland correct What happened to the unit after they were able to remove her? well, it was it's it's kind of a good situation because The family hired a vendor to come in and clean. Yes a bio company to come in and clean and when you said bio company What do they mean? What I mean? What exactly is a bio company? Well, I don't know how much detail do you want me to go into but yeah, like I said people don't enter and exit the unit and so The elderly in this situation. She I guess her toilet must have Broke on her and so she started to urinate in bottles Don't know what she was doing with her feces at this point, but we do know that It was very it was a very Dangerous unit at this point, you know a lot of spores in the air and it's not it's a health hazard for sure And so in fact in fact when they went in they they went in with gas masks, isn't correct? We made sure that it and full bio hazards. It needed full suits and booties and and gloves and Helmets so they went into this unit with full hazmat gear Correct It's a perfect example on how not you know how it's dangerous and you know I think this situation it did lead to her getting sick in the ends and so and what happened to this Resident well, I just found out she just recently passed away from some type of lung Lung problem and so and everybody pointed in the direction of that She was living in her unit with Infested and like you said with all the bio around so with all the toxic And so she shouldn't have been breathing that for all these months or even a year And so you you you were present when when the biohazard people went in what exactly did they have to do to? clean this unit well, of course dispose of all the carpet, you know throw away all of the food that's Left out like you say clean up everything clean the refrigerators. They they removed everything they had to throw out 90% of the unit pretty much work. It's gonna get that whole unit's gonna get overhauled It's gonna demo the everything yeah the bathroom needs to be demoed pretty much all the walls got to be demoed because of The smell gets like you said absorbed into the paint so they had blowers and it's it's pretty bad situation So they took out all the furniture correct they furniture soft items clothing All the clothing closet got everything all the clothing in the closet got thrown out all the bedding was thrown away Correct the furniture if it was usable it was what donated. Yeah, they donated they saved on what they could They they worked it with the with with the family For them to come in and also take what they what they wanted and sell what they thought needs to be sold like nice China Or anything that of value they would keep but all of the clothes and rugs and Soft items couches things like they all had to be disposed of because of the situation Okay, and unfortunately and and and and and now what the family now has to do Now that the unit is in the process of being demoed as they're gonna have to renovated correct and after they renovated they'll probably sell it correct But that's been quite an odyssey Right. Yes Very very difficult and it takes a lot of time to do it to do it right and and make sure that You know you're looking after the well-being of the person in the unit and and and with in your situation You mean it was you and your staff the association staff and it was the association board members There's really a lot and there were the vendors and it was a family member the family who had to Come over and some of the residents who were good friends of the of the woman, right? It took a lot of steps a Village and all of those people tons and tons of people that Had to help and throw in and and it costs a lot of money. Yes lots and lots of money lots and lots of money Which is a shame? Okay Okay, well, thank you. Thank you Cameron for being with us and sharing with us and I'm sure you know You're young to this industry. So I'm sure you're gonna have a lot more stories but and hopefully you'll come back and share them with us and Join us next week and we will have another topic of interest and


Everybody forced here rip in 2023 top Star Wars signature series 20 bucks case number three random hit style type in the hits as we go There's 20 of them 20 spots in the break. We'll do the random at the very end to see who's getting what Big stuff right there Vaughn big stuff. All right. Here we go y'all. All right. We first two cases We avoided some some big we had some nice hits while you avoided some like monsters, so hopefully we hit one here It can be tough as a new series though. Come on Pedro. Come on Pedro a Pedro auto and Jake has to wear a helmet the rest of the night And then all he's allowed to say is this is the way I Don't know I'll have to find a helmet We should get a manual or an helmet for the ship for like for like when I'm breaking star Wars. Okay, but a wheeze Homer Okay, but a wheeze box of signature series with a 70 or 80 it is We didn't have it in stock yet We don't have it in stock. Do they have it in stock of late in gaming? We don't have it in stock on either our sites yet So I don't know what we'll sell it for pretty much right around what market is though Whatever it's going for most other places All right, Jake's boy edgy oh two tubes auto David Accord into your two tubes I mean, yeah, we have the random hit stop breaks. It's just like buying a box essentially So I did the same thing with this box earlier Small paper cut on my on my the knuckle of my pinky. I just did it again and same same exact spot I need to stop. I need to stop what I'm doing for LOM auto Chris Parsons for LOM. I thought he was I thought pager was in this No, he's not I was lied to I was lied to Maybe if master work ever releases he'll be in that I don't think bilber has ever signed No, he hasn't signed ever that'd be that's that's the next person. I think I'd want to see that hasn't signed Hask triplets Paul Warren is hask triplet Well, other than you Blake, it's gonna be drew jones in 2023 bowman. I'm not sure the other names.  Trust me. I want to pull a quad auto or the uh, there's dual and quad Oh, nice one here a bb8 One of one BB8 one of one bryan herring Yeah You'll love to see that red one of one Big dub puppeteer for bb8 bryan herring Do you know how to spell bb8? I could I could spell for you jake if you need me to yeah, I'm not sure if sam or kyle know how to so Just want to make sure you know Yelp best of luck on that one everybody i'm gonna take a picture of that by the way Next up to b1 jaden waldman to b1 That from uh, yeah visions. I do like where it says where they're from to b1 Who is the actor sorry her voice? jaden waldman jaden waldman i think i said that grand moff tarkin Is guy henry grand moff tarkin Yeah, yeah red red. Yeah red is the one of one parallel. I think for every single Well b variation b design doesn't say I would I would only assume Yeah, all red is one of one Ira urso that was nice that one's four of 25 baleen kane 9 of 25 baleen kane lyra urso auto I mean that's just how it's always been. I don't think they're gonna change it a whole lot I mean signature series Izuma auto Andrew kashino izuma. I mean with that. I don't know what they would do what they'd be able to do with it without Having the price raised a good bit That's why they have more products nowadays though at least karsten tiva auto paul sun hyun lee karsten tiva auto You got like finest chrome galaxy masterwork. I think at some point is gonna come out. Hopefully Here we go Moff gideon auto 23 of 25 Yeah, what a case Giancarlo Esposito Kike Homer. Yeah, good luck on those everybody Big time cards. Those are both gonna be. Yeah gus auto Oh, that's cool bendu auto Bendu is tom baker 47 of 50 After that is a fixer auto 46 of 50 also a lot more numbered cards in this uh in this break too Fixer auto 46 of 50 skylar bible or bible very nice This is a fantastic case man. Taylor gray as ezra bridger Ezra bridger base auto here. Taylor gray Uh, no we get case from distributors Ezra bridger Yeah, we're a card shop. We get uh case from distributors. There's a dr. Pershing auto Kind of had his own little spin-off episode that might lead us to more things from mando omid abtahi dr. Pershing auto Four more. I can't believe there's four more. We've got a great case. This was uh The best one by a good amount today twilight major domo auto Everyone's favorite character from book of oba fett david pascuesi Pascuesi a bead auto bead scott capuro bead South parker family guy cards, uh, not that's like big releases or that's released recently anyway Ben kenobi auto gonna be steven stanton is ben kenobi and a imperial freighter pilot Alexander wraith imperial freighter pilot auto Let's go. Yoshi Vaughn Let's go. Yoshi. Yeah, I knew Vaughn's about to say something. Come on, Yoshi Let's go ahead and do the random everybody 20 names 20 hits will randomize them both the same amount of times and match them up to see who's getting what 10 6 and 4 10 times on the names one two three four five six seven eight nine 10 lu y down to matthew s and now the hits One two three four five six seven eight nine 10 Dr. Pershing down to ezer bridger lu y dr. Pershing jason s the moth giddy into 25 That's a big one francis d lyre urso to 25 hano with the bb8 auto one of one Congrats hano drew b with the four lom hilton ben kinobi ted k fixer auto to 50 francis d imperial freighter pilot jmart twilight major domo ted k carcin tiva drew b bead calvin l adrio two tubes drew b with the uh grand moff tarkin auto hano b the ben due to 50 francis d to b1 jason s aura sing drew b iszuma hilton was zeborellios drew b hask triplet and matthew s ezer bridger we'll get these posted to the site we're moving on to basketball now i know baseball baseball baseball