ing with rules it means that we can mask not only the class label but also some sub prompts, some other tokens that help explain the task to the model better and then we can aggregate the predictions via a conductive normal form and get the final class label another approach is not reachable prompt tuning it deals with the verbalizer verbalizer is basically a list of class names which are then mapped with the labels of the class and these names are predicted during the language modeling blog and knowledgeable prompt tuning means that we can predict not only the labels of the class but also some words related to them which are extracted from some external linguistic research so in this study we implemented both manual prompts and prompt tuning approaches that I just described and also we implemented the approach of mixed templates which means that some tokens in the prompt can be fixed and others can be tuned and the data is recent NECorpus which was pre-labeled with named entities and then it was labeled with sentiments and with relationships between their named entities and for the competition it was preprocessed for example there are only used sentences with non-contradictionary cases that X is loved by Y but hated by Z it is contradictory case which in these cases were excluded and also we only dealt with sentence level and not with document level and for evaluation in this research we first conducted three-fold cross-validation and then we tested our model on the recent NEC split which was provided by the organizers of the competition and I would like to say that the baseline model for the competition was rule-built without any modifications and the results are presented on the slide and the metrics which were used apart from just the F score F1 score also the F1 score that was averaged across only the positive and negative class since these two classes are particularly interesting for this task and also I would like to acknowledge the authors of this paper about open prompt since they released a very helpful tool for implementing different prompt tuning strategies out of the box which really helps using them for using any prompt tuning approach for the downstream task just by loading their pre-trained model from Hugginface and formulating the template of the prompt and the first series of experiments is manual prompt based experiments and here we tested two variables the prompt type and the way we handle class imbalance since as I already mentioned news texts contain many neutral polarities and not many positive or negative ones and we used a prompt as just the target word or a question to the target word about how do they feel about X and for handling class imbalance we first of all try to not use any other anything for this then we try to calculate class weights in the during the loss function and also to augment data via back translation and replacing some tokens with contextually close other tokens then the mixed template is present on the slide here the soft are the tunable tokens and text are the fixed tokens which were just the same during their whole training stage and the verbalizer consists of just class names and there then we implemented prompt with rules and here we also tested how the initialization of the prompt can affect the result and in the first template we tried to focus on the fact that there is some participant of the situation mentioned in the sentence from which the model can derive the attitude and in the second plate we emphasize the fact that the sentiment can be expressed both implicitly and explicitly and the last approach is notable prompt tuning and for this we utilized root center lex and root center frames lexicons and in the first tragedy we tried to collect words for the verbalizes in such a way that for both negative and positive sentiments the classes could overlap since there are some words that can have different meanings in different contexts and can have different sentiments in different contexts and in the second strategy we made sure that there are no overlapping and the positive class is always prioritized since in the previous experiments we saw that the results for the positive class are always a bit worse than for the negative one and so it means that for example some words like shame murder have negative connotation that's why it is negative class and so on the results are presented here and we can see first of all that no no method to handle class imbalance showed good result and some of them like augmentation are very time-consuming and they do not give some performance boost so they are not really suitable for the task and prompting approach works much better and the best model that showed the best result during our experiments is knowledgeable prompt tuning with the other first strategy and so we selected this best model and then evaluated it on the data split from the competition and the results can be compared to the third place however the models from the top of the leader board leveraged its assembly methods and these methods are extremely computationally intensive because they are ensembles of transformers and so the prompting and prompt tuning approach works really good only in terms of the quality but also in terms of the computational and so in the current study we researched the task of targeted sentiment analysis in Russia and we tested different strategies for both hard prompts and soft prompts and we also saw that prompt tuning surpasses vanilla fine tuning and manual prompting and the best model that showed really good results was the model based on knowledgeable prompt tuning and that's it thank you for your attention if you have any questions please feel free to ask them thanks a lot Filiana are there any questions Zoom for a change start with the online audience you can raise your hand or just unmute your microphone and as I said if there are any questions in the on-site from the on-site audience just come up to the mic okay meanwhile maybe a very silly question for me since you use prompting anyway so why have you decided to use BERT like encoder models not generative decoder or encoder decoder models and don't you think that using generative models would improve the performance of your approach thank you for your question yeah I was considering that but first I started with the BERT model because this was before the whole thing with chat dpt and so on and BERT models were more suitable for classification tasks and also for the purposes of computational possibilities and then the experiment showed that Rue Roberta was given even better results and then the chat dpt appeared and I also had some experiments with chat dpt which were not very successful and this is just a topic for another conversation why they were not that successful but yeah I think that it is possible that decoder only models when promptuned can be very suitable for this task but I also wonder as far as I know no one from the participants of the competition no one used decoder models maybe they also conducted some experiments but yeah thank you I think that there is room for improvement for other experiments on that yeah thanks and what about decoder models like T5 etc you didn't try them either no I didn't try them okay thanks okay thank you I guess we are way out of time so we probably should move on with the next talk but yeah thanks again Edana that was very interesting and the next talk is supposed to be the battle informational presentations comparing sentiment and semantic features for forecasting market trends yeah and I guess Andrei Zaichenko is here in Zoom yes can you hear me yes we can hear you and we can see your slides so yeah I guess please move on you have 15 minutes floor is yours greetings everyone today on behalf of my team I will introduce to you our paper something's happening Andrei are you still here yes but we don't see your slides anymore can you Andrei I guess he was disconnected Andrei are you here yeah yeah I'm here sorry the zoom crashed for me yes may I also ask you to maybe turn on your camera because it will be much better if you present if we see your face when you're presenting yeah sorry but I don't have access to my notebook I'm presented from the PC and it doesn't have a camera yeah yeah then yeah just please share your screen yes okay let's start again yeah yeah so again greetings everyone I will introduce to you our paper comparing sentiment and semantic features for forecasting market trends in recent years several studies have applied deep learning and on p-techniques to financial data including news articles social media posts and quarterly financial reports in order to predict price movements despite its significance most researchers rely on sentiment analysis as the primary additional feature living little exploration of the potential of semantics and context hidden in a text in this paper we aim to fill this gap in research by testing the hypothesis that semantic features are important for stock price prediction our approach uses sentence embeddings extracted from Twitter data to capture extra information and contextual relationships with financial market trends and then we compare our approach to traditional sentiment-based solutions to evaluate its performance in the introduction to the paper we made a claim about existence of contextual information inside the text that can be utilized and retained using the embedding approach to demonstrate the validity of this claim we created a vector representation of the text using the state-of-the-art sentence transformer model mapping our sentence to 384 dimensional dense vector space after the creation of the embeddings we proceeded with vector clustering and used a bird topic modeling technique to create a hierarchy just Andrej, is it supposed to be that you still show the title slide only? no, no, no I think there is some problem with zoom because we only see the first slide it says zoom quit unexpectedly maybe yes, now I stop the sharing yes can you see the slide? yes, now it works ok, ok, ok can you try to change the slides just to make sure that it works? ok, now I don't know, it crashes right when I switch the slides this app I don't know I try to share the screen share the pdf alone yeah, can you maybe maybe it's easier if you just send us the slides and then I for example can I send them in the chat? yes file in the chat ok, ok, ok yeah it's the first time from me, I'm sorry that's fine I guess that's how things are going on with the hybrid conferences yeah, I sent the presentation yeah, just I'm downloading it ok, so these should be your slides, right? yes, yes, correct yeah, let me ok, just tell me when to go on to the next slide ok, currently we're on slide 3 as you can see on the slide the result in output of the topic clustering in this case you can see the results for the google tweets it produced a list of 20 topics denoted by the distinct circles which were later grouped up into 4 main large clusters of the topics so next slide now we will connect these topics and market data and observe multiple time periods with high and low user activity during low activity period we can see proportionally equal spikes of clustered topics reacting to volatility changes and they are caused by just the tweet number increase on the other hand high user activity leads to great diversity in topic reactions all of them have their peaks and lows meaning that larger sample helps us make a distinction of topic trends resulting in higher correlation that is at least twice as large as you can see in the tables below the graphs at least further speaks in favor of our ongoing hypothesis that great amount of extra information is hidden inside the text semantics that can be used to predict stock market volatility next slide now we proceed to an overview of overall scheme of conducted experiment it consists of 4 main steps first is data retrieval a process we collected a 5 year data set published on Kegel it was containing tweets regarding Tesla, Apple, Amazon, Google and Microsoft companies then the data preprocessing so we deduplicated some tweets in order to clean the data then the covariates aggregation depending on the experiment we either used binary sentiment score or created the embedding vector and used this as a feature for the target variable we chose the close price last step is model prediction after covariates are were fed into either TFT or in linear models we train it and make a 3 or 5 step ahead historical prediction on validation data set dependent on the experiment in order to prevent overfitting and produce better results we introduced a custom loss function DMSE it was built on top of one of the most popular loss functions for regression text the mean square error but we added directional component it was done because MSC just focuses on the difference between true and predicted price in terms of stock price prediction the direction of the price moment is actually a more important factor than the value itself that's why the custom loss was introduced next slide depending on the experiment three main groups of pass cave areas were used containing market, sentiment and embedding features further they will be denoted by the abbreviations where HLOV is market features S denotes sentiment score and E is for embedding vectors next slide two models and one baseline approach were used during experiments to predict the closing stock price the temporal fusion transformer and linear and naive seasonal approach TFT is an encoded-to-coded transform model it makes use of multi-head attention GRNs and LSTM while N linear is a more novel approach of a simple linear layer introduced in 2022 that out-formed multiple transformer models on time series benchmark data sets that's why we used it as kind of a benchmark for our experiment next slide during elaborate exploration of sentiment score we achieved results confirming that the social network sentiment is a great indicator for stock price movement prediction as you can see on figures on the left we demonstrate stock price movement as well as sentiment score calculated in both ways first as a relation between negative and positive tweets and second as a fraction of negative tweets in the total amount visually we can observe a great amount of resemblance between price and sentiment variables in this case Apple which is above has a lower correlation below for Apple in this case the correlation was around 20% while Amazon has 40% when we compare stock volatility with sentiment score in figures in the middle we observe the same phenomena correlation is higher for Amazon by 52% and actually for Apple we can clearly see the time lag between public reaction and stock volatility volatility precedes public sentiment shift but for Amazon the situation is slightly different we can see that sentiment changes are synchronized and even precede the volatility shift in some cases making it a greater predictor of price movement using scatterblot we can further observe linear dependence between sentiment and volatility for both companies although for Amazon the dependence is more prominent we therefore claim that there is a clear statistical relationship between observed values next slide here we present a table of a resulting error score for symmetric mean absolute percentage error for all companies overall TFT model performed better than in linear for all companies the best performing model for Apple is TFT with embedding vectors as a feature comparing the closest instance we observe at least 30% degrees for SMAPE it is also important to point out that for Apple using sentiment score actually didn't help to improve models accuracy for both of the models on the contrary with Amazon we observed different behavior of models performance given the input features both and linear and TFT received performance boost with usage of sentiment score an embedding vector actually yielded higher error values models with embedding vectors show the best accuracy only in two cases out of five for Apple and Microsoft that were companies with the lowest correlation between volatility and sentiment as we said in the previous experiments for Amazon, Google and Tesla sentiment score auto formed our embedding vector approach next slide results could not lead us to a complete conclusion so we experimented further with a smaller prediction window of three days another approach was a different sentence embedding algorithm created by Microsoft mpnet it has twice the amount of dimensions compared with the approach mentioned previously but the accuracy of closing price prediction dropped significantly for our model in the table we observed that for some metrics embedding approach still shows better performance but the difference actually isn't significant and sentiment approach received better results for four out of five metrics like in the case of Apple another company that had nearly similar results for both features as was Google the difference are negligible only 0.04% for MAPE, for Amazon and Tesla sentiment again proved to be a better feature scoring higher for all of the metrics both of those additional experiments they further demonstrated that sentiment as a baseline solution still performs better than the proposed embedding vector approach next slide the results of this study provides further evidence in support of sentiment analysis as an effective tool for predicting price movements in financial markets it showed that the binary sentiment polarity extraction approach outperformed sentiment embeddings in terms of accuracy and training time for some cases the embedding approach proved to be useful on five-day prediction window outperforming sentiment baseline solutions this suggests that the choice between binary sentiment polarity extractions and sentiment embeddings as the preferred approach can possibly depend on the specific task and the prediction horizon as well as the effectiveness of sentiment as a predictor in the given context in the majority of the conducted experiments the sentiment approach outperforms the embedding vectors method this fact might be might be counterintuitive because embeddings seem to encompass more valuable contextual information however sentiments tend to represent information in a more concise way bringing less noise into the prediction model nevertheless the embedding approach still has an advantage that it does not require an additional model for sentiment extraction and the consequent quality verification of that procedure on the other hand sentiment embeddings approach that doesn't require us to verify it and can produce similar results to sentiment extraction retaining more of the semantic and contextual information contained in the text nevertheless the model training time for sentence embedding sentence embedding is significantly longer these findings suggest that sentence embedding could be considered as a robust solution after further works due to its similar performance to the sentiment extraction yes thank you for your attention if you have any questions feel free to ask yes thanks a lot so we have time for probably one question any questions in the audience either online or offline please just speak up okay yeah I have a question thanks for the talk I'm not familiar with stock price prediction field what usually related work do I mean which additional field do they use I guess sentiment polarity score should be somehow known for stock price prediction or yes yes so the current approaches most of them use the sentiment polarity extraction from multiple resources like social networks or some of them use the financial reports in order to predict a long term price movements but the text itself isn't very a researched topic but even if we talk about the sentiment polarity extractions most of them perform rather bad in the in terms of testing on the real data when we try to apply these models to the real stock market movements it only performs good on the historical data but it is actually very under researched topic because the price movements are rather random and it is very hard to find some robust solutions to bring the prices yes okay thank you thanks I believe we have to move on to the next and last talk but yeah thank you again Andrei thank you right and the next and actually the last talk of this session in fact the last talk of the NLP track of eyes this year is whether large language models learn at the entrance stage by Vlad Leonkulikov, Ilya Makarov and Radislav Neyshev and I see Radislav is here in zoom to say something just to make sure hello everyone can you share your screen definitely so here we go you should be up and running now yes please move on 15 minutes of yours thank you Andrei so hello everyone thank you for hosting me here today unfortunately I was unable to join in person but hope next time it will be so we have a lot of different discussions on recent advances in NLP especially in the generative models including transformer based architectures and with my colleagues we tried to analyze why actually they perform really better in some cases when we provide some additional information during the inference stage or is it so called the learning and reasoning effect in the inference stage so main goal was not actually to provide some new approach to make it even better but to understand and explain and maybe provide it to other people the pipeline how to make it work and analyze what are the main reasons of this happening so just in case the main question is how do these models especially transformer based models like GPT or bloom family learn during the inference stage learn in quotes of course so they incorporate some additional novel information which wasn't present during the training stage with no changes either to the architecture and to the model premise and in addition we tried to cover these two questions because they are quite widely known but they are not that strictly defined first of all what is a lender effect learning and reasoning effect and do they actually learn in reasoning in inference stage by them I mean the large language models or they simply simulate some intelligent behavior so there are a lot of different papers of course on this topic I've only brought three of them that for me seems a little bit most relevant for this particular research but actually in the original paper supplementing this speech we have more than 15 different sources then they all are important and there was a first paper which provided the chain of thought prompting in large language modeling in the same approach we do the first one, second one is just original paper on GPT3 and the last one is actually quite old compare it with all the other papers paper on the future learning invention translation from back 2018 but we have a few relevant ideas in the paper here so let's first of all formulate the hypothesis which will help us to move on main hypothesis in this work and we will try to support it with several experimental and literature results is a following, large language models and by large language models we assume all the models which have big enough parameter of space space parameters I will define what is big enough a little bit later we assume that they create some inner language spaces that contain not only the language itself like grammar, semantics and so on but some patterns of rules and reasonings which are implicitly not explicitly embedded in this space so LMS instead of learning something new during the inference stage simply adjust their state of mind to the particular already learned behavior trajectories so main idea once again of this hypothesis is that the majority of the information is learned and embedded into this space during the train stage and during the inference stage the examples of resonance chain of thought and so on only provide some instructions to collaborate the behavior of the model so I will be brief on the problem of you because I understand that everybody on this conference is well aware of all the problems with language models but just in case we assume that we have given a sequence of some tokens from finite dictionary which contain description and maybe one or several examples of the behavior on the desired problem and we assume that these behavior examples help us to solve the problem let it be easy classification regression generation whatever so first of all let's take a look at the data size and the model size for which we actually observe this learned and risen effect because if you take native GPT provided by a breaker party with 300 lines of code then you definitely will not see any calendar effect and that's fine so according to the data we checked in several sources all the sources list once again are available in the paper itself the corpus itself corpus of data meaning should have at least 300 billions of tokens because otherwise it's kind of too small and also the models should be of size billions not millions of cost that it should be approximately 33 or 50 billions of parameters however even 7 billions seems to be enough for certain types of tests for example if you are speaking only about the machine translation and speaking about the paper on unsupervised machine translation and how is it related for the current speech actually translation systems show us that if was speaking about unsupervised machine translation from the time we can see that if we have two different models trained for language modeling on two different languages we can simply try adjusting the internal spaces together by using only small amount of label data what I mean we can have one some kind of language model or just word embedding model like word2vec, fastex and so on and the other one in here on the slides adjusted some kind of strange clouds and the idea in there is the following we assume that all the languages are actually based on the real world real world scenarios outside there and we know that actually word sun and sky are usually much more aligned and they usually come together then words sun and I don't know dimitile or hedgehog or something else so when we provide some examples of words that directly translate one into another for example word gato in spanish I guess or adelian sorry I forgot it and cat in english we simply adjust these clouds of points and then we can get the time of course almost state of that approach with almost no data which is aligned between two spaces so the assumption is the following these models already have created some internal states and then we simply align them with several examples which are labeled for first and second spaces so the spaces are aligned as the markd and now we get the aligned spaces and the translating can be performed rather well that was the idea from the machine translation and later we tried to adjust this idea to the chain of thought reasoning because when we provide some chain of thought reasoning for the model or provide just an example to make the model work better we can see that the model starts working better but the reason might be the same we can adjust the internal state internal language space of reasoning of the model because the model is definitely over parameterized because it contains billions of parameters and it might cause the better behavior there is even an important note because which follows our observations and experiments and it somehow proves our results because when we provide some examples for the model how to work in the upcoming problems like regression or tech generation in specific form or some arithmetic problems even providing the examples with wrong reasoning so we provide example that the model should follow some path of reasoning but the example itself is just incorrect for example it has arithmetic errors or just broken logic and so on even such examples help the model to perform better and to achieve better results but if we change the order of reasoning for example we try to just shuffle the sentences within these examples then the whole sequence is kind of broken and the model behavior fails the model quality fails and the model behavior is not as we expected to provide some experimental proof of our claims not only to refer to some other external research papers we run the experimental setup with 5 different prompt and scenarios the first one was absolutely no prompt so we simply ask the model to answer to our question just like in zero shot learning for example compute something I don't know how many computers are in here and we also had for additional prompt and scenarios first of all standard prompting so we have simple example in here like one shot learning for example question there are 9 computers in the server room each day blah blah blah and the answer is 29 after that we send another question for the model and we'll ask the model to continue the sentence to generate the answer as we usually work with this language modeling approach the second scenario is chain of thought prompting so we provide a question and then we provide a chain of thought sequence so we not only provide the answer but the chain of thought as well so the second scenario is actually invalid reasoning so the same stuff as we did with the chain of thought but now the chain of thought is actually incorrect but in this case we still have the correct sequence of steps so we had original 9 computers then we have for each day we have arithmetical errors and the answer and finally the irrelevant prompting so we have question on something related for example with computers or with vegetables or with something and the answer is just random coming from another example for example so we had 5 scenarios and we used couple of open and closed source models because we wanted to check the behavior either on some on some state of the art models like GPT family or some open source models but in case this work was mostly performed during spring so we don't have recent models like second llama and so on so we mostly focus on bloom if I was speaking about open source models we started using models from half billion of parameters to 176 billions so including bloom MT0 excel and and of course GPT4 because it's kind of state of the art for now so I will provide the example table so it's much easier to see but the main idea was kind of simple models with a small size including bloom up to 7 billions could not provide any useful improvement using chain of thought either correct or incorrect so they're not present in the table and MT0 was also kind of inefficient if we try to provide some chain of thought reasoning while bloom 176 Guancao and GPT4 and also including previous to GPT4 GPT3.5 and 3 models like tax da Vinci 0.2 and 0.3 provided some useful improvements so first two columns in here speaking about the models correspond to open source models while last three correspond to open models which are kind of close and we have percent of the correct answers in 5 scenarios so no Dima no prompting at all then standard prompting chain of thought invalid chain of thought and irrelevant chain of thought and we can see that actually bloom behavior did not improve on the arithmetic reasoning at all using any of the examples except standard reasoning and chain of thought so it improved the behavior a little bit for the Guancao it seemed a little bit surprising so without any Dima it performed much better than with standard prompting but chain of thought and invalid chain of thought provided improvement of the score so once again we prove the hypothesis that actually the order of the prompting is more relevant than the correctness of it itself and speaking about the open air models we can see that we solved them we did not use them with standard prompting at all so we use only with standard prompting and chain of thought we can see that chain of thought improved the behavior invalid prompting doesn't break it and sometimes it even improves for some reason maybe it's the consequences of not big enough size of the tested dataset but for all the other models this size was quite enough because the results were rather stable and even when we provide some irrelevant prompting we can see that it might improve or not degrade for GPT-like models spoken about open air and last but not least GPT-4 provides great results out of the box and with prompting we can either achieve the same behavior or even break it despite we are using some useful prompting so it's standard prompting or chain of thought with correct examples the result was achieved with question and answer and reasoning on the bamboo bill test so we can see exactly the same behavior chain of thought either improves everything a lot or doesn't change it for GPT-4 invalid prompting in this case can break it a little bit but still it's really close to correct prompting but irrelevant breaks it a lot so the conclusion might be the following and I decided to just break it into the couple of questions we formulated in the beginning so first of all what is learning and reasoning effect we assume according to the research we found that learning and reasoning effect is more about finding some similar reasoning patterns in the latent space the model has created during the train stage and then these examples chain of thought or just several short learning provides the model example of how to collaborate itself to find the appropriate maybe a projection of its own parameter space for the desired problem to solve the problem better so it seems like the application of pre-existent knowledge rather acquiring NSNU and speaking about the idea of large language models learning during the inference stage seems like they okay I cannot say that I cannot learn during the inference stage but for now we have not found any exact evidence that they actually learn something new during the inference stage but we can assume that they are kind of exploiting their already existing knowledge and that once again is proved also by the results which we observe when we provide incorrect prompting but preserving the desired sequence so while when the model is just provided it it achieves better results even if we provide it with wrong examples so that's kind of it and if you have any specific questions thank you just from my sorry you welcome just from my side actually the error is rapidly evolving and I understand that if there are some new results this might become a little bit incorrect just every day we got into a really tricky area of trying to explain why all these models work so I would be glad to hear any of the questions including the questions which just question the correctness of this paper as well because I want the discussion to be useful for all of us thank you you're welcome thanks a lot this sort of concludes the ISLP session but maybe we still have time like for one quick question I guess people on site and online are already anticipating the closing of the conference but yes still any questions okay yeah just a brief one from me I'm a bit a bit confused in fact about your final claim or statement that large language models don't learn anything during a few shots in a few shots in a few shots scenarios isn't it sort of trivial I mean does anyone actually claim that models learn anything in a few shots scenarios of course they don't because their weights are not updated so like okay may I then fix a little bit this formulation yeah you're right so we're not speaking about the learning in classical paradigm when we just update the model weights or add some another adapter no by learning we merely meant that the models do not acquire any new knowledge so if they are unable to solve some problems at the if they're not trained to solve some problems during the train stage they won't acquire this ability even if we provide some useful prompts some examples and so on so we only can make models solve the problems they already have seen during the train stage maybe in a little bit different scenario but we cannot generalize them to unseen regions of our fish space or problem space well but we do know that we can just this claim sounds to me a bit self-supporting it's a little bit obvious right okay then it might be a little bit obvious so the main reason we performed actually this research was curiosity to find out whether they do or do not because when we started it was kind of a little bit at least less than a year ago there were several examples like providing iris data set to change gpt and it was able to perform the classification on the inference stage providing some examples and improving the quality of the answer a lot so we tried to make a little bit bigger overview including couple of different papers, different approaches to actually approve or claim that it's not learning during the inference stage so yeah we're not trying to provide any novel I guess result in terms of its astonishing result we're trying to actually a little bit more formally prove that no they're not learning yet okay thanks a lot I guess it's time to thank the speaker again and close the session thank you Radislav and now yeah I step down to leave the space for the closing session thank you Andrei for sharing now we proceed to the final and still very important part of our conference which is try to summarize what we achieved and what were the main highlights so this is allowance edition of ice conference first of all we're very glad that many people made it offline and many nice and really high quality talks were presented during these two days and the first nomination in the natural language processing area award goes to Antonin Lykseev Sergei Nikolenko will now they work on Kyrgyz language so with these I would like to please Anton to come and to make some short one minute presentation and highlight they work but first let's thank you well first of all thank you for the winner I guess now really have to continue the research on the topic to make the dataset even more fine grained and more justified the the the whole purpose of this work was to create the one topic classification dataset for Kyrgyz language which would and is essentially the first dataset for the applied NLP task in Kyrgyz language and then the overall idea why would that be why such a task is that there is an urgent need in some dataset to find out whether the multi-lingual models work for Kyrgyz language and as we've shown they do to a certain extent and outperform the very basic beg of anger approaches by a large margin work is going to be continued and now for sure and more interesting work works I hope are to come because during the this year and a year and a half maybe a large community of volunteers in Kyrgyz involved so this is it and I'm pretty sure that there will be a special time for that but may have some personal remark I would like to thank the organizers who make the conference happen who made this conference happen again and of course our fabulous hosts who managed to do everything like perfectly despite the trying times thank you thank you very much the next section is computer vision and awards go to Razan Didoa Andrei Galitshin, Pavel Astashev Dmitry Dilov and Alek Rogov and this is work on deep learning based one pathology localization with classification with extreme images so I'm not sure whether Alek is for the first time arrived just this morning and already got best-ever award so please this is probably because yesterday I was giving a speech regarding the artificial general intelligence so I think here all the credits go mostly to Razan Didoa who is now the PhD student of the Tenzor Network Slob with the SCOLTECH so we all know that attention is all paid but sometimes you have to look only once so we decided to combine these approaches and eventually we found an architectural approach to address very important medical task of Charles risk trauma detection in hospital we eventually got asked the preclinical trials and we developed the approach that combines the state of the art approaches in the object detection gradient attention mechanisms and they shifted window and blocking it so well I think that as once Andrei Kolmogorov said that really new things lying between something trivial and something incomprehensible thank you thank you very much, Alek the next award on social network analysis go to Sergey Siderov Sergey Mironov and Alexey Grigoryov on work on limited distribution of friendship index in scale free networks please please say a couple of words about your work thank you it's a big surprise for me thanks organizers friendship index was studied a lot in social studies but what hasn't been given enough attention in network analysis so here we did some extensive research on friendship index and its continuation of our work well we we've studied a lot of distributions, how it's distributed what its limits are and well actually I think now it's it's time to put friendship index away and move on cause it's not a great measure it's not a well all solving one and I hope this well you like my talk and thank you very much and congrats again so the next talk is on machine learning goes to Vladimir Berikov on ensemble clustering with heterogeneous transfer learning say please a couple of words about your work thank you very much for it's a lot surprising for me so this work the idea of some additional information which can give some insight to the analysis of data of target data is used and algorithm is based on finding some useful meta features and data from both domains are quite different the features are different for the domains and we should find some structural properties of data and transfer knowledge from one domain to another domain thank you very much thank you last but not least Vladimir Berikov on his theoretical work on subject coefficients for the maximum anti-chain of partitions and related counting inequalities and I really hope that Dmitry will decrypt now what is contribution thank you this is a bit unusual being an organizing member and receiving this kind of award for the theoretical section with something worthy and it was a pleasure for me that the committee evaluated my work by its scientific merits and I was also happy to apply data mining and machine learning techniques to the problems that were posed by such eminent mathematicians like Giancarlo Rotan known in combinatorics Ron Graham Gold and Kleitman so here I just added one small break to our knowledge on the number of maximal anti-chains and anti-chains of partitions in particular and also somehow helped to reduce uncertainty in some unsympathetic coefficients thank you and let me now say once again great great thanks for our hosts we all heard probably about Armenian hospitality but now we certainly all experienced the best of it and this conference this event would not happen unless Habed and Amalia would do so much during this half a year or more and first of all thanks for just proposing opportunity to host the conference and providing all the resources and all the support and our every need so Habed and Amalia let's thank them and computer science and engineering department of A.U. so thanks a lot thanks and of course thanks for all our supporters from Skoltec high school economics those who basically contributed time of their people or other resources to the conference so with this I'm glad to conclude this edition of IST and I hope to see you next time at the conference stay tuned when and where it will be and be sure that we working to make it happen next time


The National Chairman of the Social Democratic Party, Sheru Gabam, has called on the President, Bola Medinubu, to put up a high-level security apparatus in place in the three states of Imo, Kogi, and Bayosa towards the 11th November governorship elections. Gabam made this call at a world press conference in Abuja on Tuesday. He says the President is the only one that can guarantee free and fair and credible elections, since he is the head of the executive that controls the security agencies and all relevant agencies tasked with the responsibility to ensure a safe environment for a successful election. It is the President that is the commander of military and paramilitary and all security agencies that will be deployed in the entire three states where the National will take place, and that is on command. To ensure that the environment is conducive for INAG to deploy sensitive and election materials. If the environment is not conducive, INAG cannot deploy election materials. So the possibility of having an election depends on how conducive the environment is. Unfortunately, other by design or otherwise, the three states have their own pocket of security, but the height of it is Kogi 6, that worries everybody. In the past election in the South Kogi, they have created this panic in the land. If you can remember, we as a party we suffered tremendously in Kogi states. Our offices were burnt down. People were killed and maimed. All parties became victims. Do you think people know as one crisis in our house?


a perspective that I hope will be congenial to you. I have reason to believe it will be. It's basically the perspective that every human being is at every instant that you're not the coma. Mind, brain, society, and culture. And that these four manifestations are each other's environments. And that astonishingly, military trainers have really nailed a lot of important practical wisdom in the area of brain, mind, society, and culture. Because they cultivate all of them in the course of creating a new military service member. So, with that I will launch into my prepared remarks. I'm going to introduce you to very few, possibly only one Greek word that you may or may not be familiar with. And that word is thumos, T-H-U-M-O-S, sometimes spelled T-H-Y-M-O-S because of the vagaries of how Greek letters are transliterated into English letters. I believe that the norms of doing these things requires me to disclose that I prepared this talk originally for a symposium at the Einstein Stiftung in Potsdam, Germany. A great joke if you can get it. And that was, of course, Albert Einstein's hometown. Anyway, Homer uses the word thumos more than 700 times into two epic poems. Plago told us that it was one-third of every human soul. But the reputation of thumos has been quite turbulent over the millennia since then. So, is it that we have outgrown thumos, left behind in humanity's childhood, and in each of our own childish histories, along with the scholar Francis Fukuyama, who I believe many of you are familiar with as an author, and others, I've been trying to put this juicy Homeric word back into current circulation. It's a very odd thing to want to do because for both Homer and Plato, it seems to be something or other to do with killing rage, or at least with anger, but accept to condemn it. It might be suspected of one of those per post-World War I intellectual chatterers who was in Thomas Lund's Doctor Fastis. These chatterers are the folks whose eyes sparkle when speaking of violence. My interest springs from 20 years of clinical work as a psychiatrist with psychologically injured US combat veterans. I quickly learned of what took the greatest toll of their lives, their families, workplaces, and communities, was traumatic damage to good character. The veterans we worked with had been incarcerated. Many have led violent lives since their return to civilian life. Their own violence had lighted their lives, and ultimately, I came to know that most of these men hated their own bad behavior. They suffered great humiliation when they contemplated it, and the damage it had done, the damage it had rained down on their families and themselves. They could, however, recall times when they considered virtue possible. Even somewhat, in the clinic they were destroyed, they demanded, they made self-important claims to have been players in the most significant events in all of human history. They were friends of what they understood to be their dignity. Occasionally, they would insist that they would only deal with the person they called the head of the snake. They could be either the chief of the clinical service, the hospital director, whoever they identified as this person. The, quote, global destructiveness of their fantasies, wishes, and occasionally behavior. Their vulnerability to collapses of morale, which would leave them so apathetic that they would not want to work, even just getting out of bed. They were also like the convry of preoccupations and psychosomatic disorders. All of these were frequently swept together by psychiatrists and done with the psychotrugin narcissist. I will attempt to persuade you to see the word thomas as a suitable bearer for the load that we want. The word sort of has a good vibration to it. You might be wondering at this moment, why does this die out to me? Anyway, so, stay tuned. Traumatic damage to character is a phenomenon that actively refused and denied by American psychiatry. Interestingly, it is accepted by the World Health Organization. An explanation for this strange disparity is that American psychiatry follows the flag that we first see marching with Plato and carrying forward to this day. And here's what's on that flag. If you make it out of childhood with good, good character, nothing in the way of later experience can budge you off of your firm stand on virtue. Plato spoke of good breeding. We would say good genes. I emphasize that Plato was no fool and he said, and good upbringing. Both of these two good ingredients, your character would set up rock hard like cement and experience can change, or at least Plato said so. But his Athenian contemporaries would consider that crackpot. Once you accept Plato's position, you end up viewing someone who misbehaves as damaged goods to begin with. The general position of American psychiatry is that any new behavior after bad experience in adulthood is only the current expression of a pre-existing flaw. Perverse and controversial evidence of, and here this is in quotes, betrayal of what's right. In a high-stake situation, by people in legitimate authority is taken as rationalization merely an attempt of a character-disorder patient to get over on, deceive, and manipulate the clinician for some personal advantage. That's the expression, the experience that a person with traumatic damage to good character has in most mental health settings. Interestingly, in Germany, Professor Michael Linden and his group at the Rehabilitation Center in Berlin has been working persistently and creatively to expand our knowledge of this territory. This territory of post-chromatic character change through a diagnosis they propose called post-chromatic embitterment disorder, the embitterment of personality. They have developed a wealth of new understanding through their work, interestingly with East Germans, whose honorable life trajectories were shot out of the sky by the reunification of Germany. There were such a astonishing historical cleaver stroke in our life. So, Professor Hummer translates the word hummus as the energy of the spirit of the East, Swedish scholar Ernst Artmann, who was writing in German, offered an ilummus as die Ichsel, the I soul, the soul of this I, the my soul. A very evocative, it's narcissistic dimension, but also sparks over into the concept of identity as it currently is used in the phrase identity politics. The conventional English translation of spirit, and that is the conventional translation of this German word, is opaque or misleading. Maybe the German dust commute is better because of its connection with self-respect, but if so, in modern German it is so frequently used sort of to make a joke of it.  I'm here, I expect that a good many people in this room especially are familiar with the name of the, not sure how to get the sign of a professional title to them, but Francisco Cugliano, I see a lot of heads nodding around the room. He's a well-known person who's written on military philosophy and affairs. As Frank Cugliano has pointed out, modern democracies often fail to recognize honor and the desire for recognition as part of the universal and normal makeup of humans. Noticing this phenomenon only in its pathological and deformed states, deformity of hummus is a common and disastrous complication of the primary psychological injuries of war and is what is currently tagged to PTSD, post-traumatic stress disorder, which you'll discover that I'm no great fan of this diagnostic formulation. According to Hegel, all human warfare originates in a fight to the death over honor. A fight for unconditional recognition and acknowledgement, but to skip German phrases, because it's burdensome to people who don't understand German enough, well enough to. So unconditional recognition and acknowledgement by an equal, which only one of them can win. Now this is, there are two ways to lose. Death were they all encompassing dishonor, the social death of enslavement. Honor is a social thermal. Its interior psychic mirror is hummus, guardians. So this whole line of investigation goes back a very long way. Commenting on Plato's Guardians, Aristotle says from the politics, quote, The attitude which some require in their guardians is to be friendly to all whom they know and savage to all who are unknown. This is the attitude of the high spirited temper, and I'll spare you my pronunciation of the long brief word. Going on, hummus is the faculty of our souls, and I'm still quoting Aristotle here, which issues in love and friendship. And it is a proof of this that when we think ourselves slighted, our spirit is stirred more deeply against acquaintances and friends than it ever is against strangers. This faculty of our souls, and still Aristotle, this faculty of our souls not only issues in love and friendship, it is also the source of any power of commanding, any feeling for freedom. It is hummus that causes affection in this, or spirit as hummus is the capacity of the soul whereby we love. It is from this faculty that the power to command and love of freedom are in all cases derived. That is one big load for one concept to carry. And yet, this is exactly the frame that I want to cut the idea of character to carry. Many of you want that too. I have worked enough with military personnel, almost always officer personnel, to know that the word character has huge resonances in that word. Huge resonances. So I object for emphasizing that current psychiatric terminology calls hummus narcissism. Narcissism is simply a new word for an old concept. It calls us from Homer, Plato, Aristotle, and the Athenian tragic poems. Pride or vain glory from Hobbes, a more quote from Rousseau, Desire for recognition, sings of Anarchimus from Engel. Narcissism from psychoanalyst Heinz Kogu, who developed and fundamentally modified Freud's ideas, first threw us to the modern psychojordanic narcissism because of the way the latter term has been pathologized and turned into a general purpose blame word. And I will spare you the usually obscene utterances that one encounters when somebody is anguille, peace, and narcissistic, blank, we all can hear in our ears what the next word was. These thinkers over 3,000 years from Homer to Kogu have seen this feature of mental life as normal and universal, even if it can develop dangerous excesses, deficiencies, or deformities. I believe that Thumbos is a human universal that evolved out of warfare in our ancestral evolutionary past and still explodes in killing rage when violated. Many legal, cultural, and social changes have more or less removed these reactions from the individual realm. And I point out that we no longer teach our children, as the Vikings did, that a man of honor must kill someone who makes a joke at his expense or steals food from his cooker. But this was true of the Vikings, apparently. However, such reactions are very much alive at the collective level, and lamentably they are regarded as both patriotic and virtuous. The modern adult's cloak of safety and guarantor of his or her narcissistic stability is the society's image of, and I put this in quotes, what's right, and the implementation of what's right by our elders, along with concrete social support of a face-to-face community to whom one is attached. Now, I don't pretend to have a universal meter to say which of those is more heavily weighted at any moment. I'm trying to figure this stuff out myself. I don't have it all in my backpack. Narcissism, allegedly the most, quote, primitive of psychological phenomenon, is much inclined on the body, on the one hand, but is just as deeply enmeshed in the social, moral, and political. So here is my definition of thermo-musk, or modern practical use, a definition that still accommodates over and there a style without doing violence. Number one, the historically and socioculturally constructed content embodying ideals, ambitions, and attachments. That's the content of thermo-musk. And then there's an energetic term, the intensity with which these commitments are energized. So strength and weakness and content are in very independent, unfortunately, way bold. Having counter people can shatter old, beautiful sentiments about good and bad and so forth, but who just blow in the wind. So those vary independently. In the normal appraisals, control the emotions and moods associated with shifts and plumes, specifically cognitive appraisals of agency, agency in and of the direction of change in the condition of ideals, ambitions, and attachments. This is a big lump to swallow up all at once, but the basic ingredients are simple and pretty familiar. Are my ideals, ambitions, and attachments, and my attachments I mean to people we love and care about, are they improving or deteriorating in the real world? And if so, how fast and how much? Who is doing this and why? During my years in the veterans clinic, I became fascinated with what I now call moral injury, which Pache Plato lay at the root of the veterans deformities of character. My current preventive psychiatry work was presented to military forces as the prevention of psychological and moral injury. I have made the expansive claim that psychological injury as well described by the diagnostic manuals term PTSD, post-traumatic stress disorder, goes deep into our evolutionary ancestry and one can observe that kind of learning about danger in one's pets. One can observe it in raccoons. If a bear leaps out of a berry patch at a raccoon, that raccoon will probably steer clear of berry patches for a long time afterward. This is part of our pretty much universal inheritance as probably as vertebrates. Main interest is in the prevention of personality deformities, which I now tag with the phrase moral injury. The breadth of the Homeric usages seems both useful and truthful to me. The 700 plus occurrences of the word are almost all very emotionally charged, but with the whole range of emotions you would expect from what I have just, love and pity as well as anger, feel, fear, joy, as well as sorrow, elation, and despair. I want to restore this breadth to any current use of the term. Achilles identifies Hector and all Trojans as the agent of Metropolis's death. For those of you who don't recall, Metropolis was Achilles' battle buddy. His second in command were their regiment and also his foster brother. They had grown up together. They don't use this and just sort of lay to rest question of were they also lovers. I don't know and frankly I don't care if there is extra evidence that Aeschylus was sure that they were lovers because there is a surviving fragment that's pretty torn and honestly I don't give a damn. Plato uses Thumos primarily in an anger-riddled martial context to which Aristotle necessarily had to respond but Aristotle returned to the broader Homeric use as you heard in the earlier quotation and abstracted it to the point that it seems to mean in part the capacity to have emotions at all. Thumos is thus a container for the English word character. Briefly from Homer, the character exists in dynamic relation to the ecology of social power and how that is modeled and remodeled throughout life by how well or badly those who hold power fulfill the culture's moral order. This allows us then to define moral intrigue as the state arises in a person when he or she has suffered three things. One, a betrayal of what's right and that is in the culture. By a person or social institution with legitimate authority that's something in this social system and I emphasize here that it's legitimate authority that has committed the violation. It's not clear to me that moral injury at least in this line of thought is possible in a criminal assault. Now I can see that there might be coaching arguments that that might be not reasonable. So that's one into three in a situation with high stakes for the injured person. So that the stakes obviously reside in the mind of the injured person. Whatever their, whatever the ultimate social and cultural origin of those states we learn to value things the way we learn pretty much everything else those human beings from our experience. Significantly well-throwing up. All three of these are present. The trail of what's right by someone with legitimate authority in a high-state situation the body reacts massively and the way you react is as though a physical attack is in progress. And obviously this neural and humoral response is operated by the brain being surprised. So there you have the human critter as a whole. Culture, society, mind and brain. With none of these have an ontologic priority. Meaning that just one of them is the really, really, really, really, really anxious. And maybe when we're sweet I can tell you I don't know what happened to me on that. So this is a pitch sort of an anti-reductionist pitch here. One of these is epitome of one of the others. It's a way of sort of sticking a thumb and play those eyes. Wanting to get away from any claim these is merely a do-so. Others and environments must exchange suitable inputs and outputs with each other if they are to persist in time. I would add that in the sense of biological evolution they all co-evolved at the same time. When the physically modern at the time, when the physically modern human first appeared in the upper paleolithic probably all four were present. There has never been a time when a river or species has had a third of a language or consciousness, an eighth of a mind or half of a social system. If you take the view that the physical brain is prior to the artifacts that it creates, which makes possible, I point out that this is true but only in a trivial sense. It is a general biological phenomenon that organisms evolve and adapt. The environments of their thrown into what create the environment to which they adapt spare you a brief digression into permits and their amazing nests. So the so-called evo-devo for an evolutionary development community in embryology has shown that it is possible to do rigorous science in this kind of conceptual framework. Okay, I can't blame people for hearing the rolling of their stomachs or the complaints of their bottoms from sticking so long. I have shot far into the outer space of abstract concepts and want to close by bringing it back to Earth. In Achilles and Vietnam, I described a rare state of solitary, rapid killing frenzy that can arise in war when a soldier has experienced the trail of what's right by a commander in a situation involving the death of a beloved comrade. I use the Norse term Berserk term for this. The Berserk phenomenon has riveted people's attention to a degree far out of Earth proportional to its frequency, which is rather rare. Murder rampages anywhere in the world generates a flurry of references to the Berserk state. These are horrible and tragic, but frankly they are not the stuff of my nightmare. What keeps me awake at night is the post-military phenomenon that the Germans experienced firsthand during the 1920s, the Freiburg. Is this a familiar term to at least some of you? I see some nodding a bit. Today we would call these right-wing paramilitary death squads. I fear that the historical, social, and cultural conditions in America are now favorable to the formation of such a path to save will be an illustration of the ways that interdisciplinary analyses of this work I sketched out can be applied to very practical, real-world needs. According to historian Bruce Goodman's at Marine Corps University, which I believe you folks have heard of, that according to Dr. Goodman's news of dear ground, when the visions of the Reichscare, that's the general army of World War I, were demobilized after World War I, they mainly returned from war as units to the geographically compact regions from which they had originally been stood up. Their reintegration into civilian life was fostered by the social bonds that they had formed by training together, going to war together, and coming home together. Their reintegration was even fairly smooth in places where the hometowns lay on the other side of redrawn national after the Versailles treaty. That's sort of astonishing, but it's a historical fact. According to Goodman's, these regular soldiers were poor candidates for attraction into the pride poor and the like. In contrast, recruits to the elite Yeager units were drawn into these formations as volunteers throughout, from throughout the Reichscare. They were demobilized as individuals and scattered as individuals across Germany. These were particularly responsive to recruitment into the elite patriotic ideologies and tightly cohesive group practices of the pride poor. The thought of such a phenomenon in America makes the air span above the background. Here are my practical conclusions. I anticipate that I'm about to tell you a story that puts the elite formations like the Army Special Forces, the Navy SEALs, and so on in bad life. But I'm going to veer into a direction you may not be thinking of. The analogous group at greatest risk of attraction to such formulations is not the demobilized veterans of elite military formations such as the ones I just mentioned. But rather, the tens of thousands of armed private contractors in Iraq and Afghanistan, the trigger force among the contractors. Most of these will have had prior military service but may have few persisting social links to their earlier military units. They will come home from theater alone carrying whatever psychological and moral injuries they have suffered as contractors alone. If they were to ask for mental health treatment, their treatment by the U.S. government veterans health scheme known as the VA would be greatly endowed and their former contract employer possibly heightening whatever indignant rage they carried from their moral injuries. This would render them extremely vulnerable to recruitment to violent political and also criminal gangs. While idle health treatment as a cure-all for such potentially dangerous problems, any incremental societal risk reduction is worth pursuing, especially if it involves fostering stable communities of such veterans in a context that remains open to the wider world. My clinical observation is that recognition by peers in a stable community of peers is the most potent anti-inflammatory treatment for injured umbas, various members of the United States Congress. And I propose anything that might come look like an answer to what I just said. When I proposed extending VA mental health benefits to contract veterans to various congressional staff members that I've worked with, their reactions ranged from intense interest to worse, and I'll spare you the actual language. This staffer wanted the contractors to be liable for providing such benefits, which is hard to disagree with, but you will see that it's a swamp. Reconnecting these contractor veterans to military unit associations reflecting their prior military service may also satisfy their, and this was Hegel's term, Zenzuka Anarkano, a longing for recognition. He earned sociologist group and one of the best books ever written on this up here called The Veteran Comes Back, published in 1944. He said, the veteran comes home angry. He pointed out that the organized groups of veterans are noisy, demanding, and annoying, but that their mutual support and recognition assuages the most dangerous excesses of their anger. The typical American psychiatrist faced with the angry, quote, narcissistic veteran usually cannot see beyond the end of the pen with which he writes out a prescription to modify some chemical or other in the veteran's brain. We are so much better able to do something constructive with and for our fellow humans if we sought to save all four human avatars, brain, mind, society, and culture at once. And I hope that you take in the whole model of doing that that I attempted to portray in this talk. Now, I very much like to hear your comments, questions, and criticisms. We're talking about your time. I think this is a really important topic and my own feeling is that it's one that has only began to gain some traction or awareness within our institution. I think we have a better understanding of what the war does psychologically to the veterans and I think this workshop is a regulated type of a spot like that. My question is going to talk a lot about, you know, the tired of the cafeteria, the drawings, writing, whatever you are. What I'm wondering about is how do we prepare the next generation of veterans? This all seems to be kind of an expo cycle, you know, a retrospect. What is the prospect for a rural injury? How do we address this so that we can better help the next generation of veterans come up with a kind of a working product? My belief is, and I cannot prove this with a sharp scientific principle, my belief is that the key to prevention lies all in the hands of military leaders and policy makers. And the touchstones of their films and their thinking must be how do we improve cohesion? That's a face-to-face quality within military units. Leadership which folks in this institution spend a lot of time thinking about and I don't think I have to explain. Cohesion, leadership, and training. Now, those three things are like motherhood and apple pie to military professionals. Everybody's toward it. And the question is, what are we doing to ourselves inadvertently or sometimes because emotions are aroused or ambitions are inflamed, we manage to do things that are profoundly harmful. Hopefully we'll be interested to hear what leadership has got to be. Expert, ethical, and it's our tall hierarchical institutions that we're talking about. Every leader needs to be properly supported. Obviously if a leader is totally unsupported from below, the thing is a complete mess. So, I really support an expert, ethical, and properly supported. So, there are my slogans. You get no expert charge. I think you probably put them out. And that's a big battle. What I mean by that is... I didn't quite... Lead faculty. Lead faculty, yes. Lead faculty. The big buzz... What's the big buzz in the DC area right now? I'm one of these policy makers. I'm one of the top rats. It's near peer competition. What that means is against China or Russia. And the Army, the Navy, the Marine Corps, the Air Force, they saw the things that I'm reading and what they're about. It's publishing and what they're talking about. It's great, great, great. Yeah, they're talking about training the body.  You know, everyone is talking about getting into this end-of-sit war that's going to be huge, the likes of which we haven't seen since World War II. Because of the Chinese, eventually. And I just don't see how, if that happens, God forbid that happens, I actually put that line in our cultural tribulations. God forbid this happens and the other may take out. God forbid that happens because if it does, this country is not going to have where we're all to deal with all these millions of Americans that are going to have to go into service and be trained to kill. To be trained to, you know, get up to these stands and leak out and get all the top rats down. And I don't really have a question. I just want to pitch that out there to you. Because this indices on, you know, this is why we have military top brass and not only do we deal with better issues, but we're interested in getting ready for the next conflict. And what's scary to me is everybody's saying the next conflict is going to be a Chinese, right?  Wait, think of that as a possibility. Hopefully not, but just like I said, I don't think we have where we're all to, one, pay for something like that, conflict like that, and two, deal with ramifications of that and for generations afterwards. Well, it makes my heart sink. It puts a lump of lead in my stomach to contemplate. It seems to me what you're describing is a land war in Asia, which America has. Well, they do that next week. Well, so that's in there, too. In fact, in general, this is after all the Naval War college and it's part of your job description to try to do your very best to think through plausible scenarios for future wars that they might, in the United States, might be compelled to be fattened. I prospect, and any decent person bought to shudder at such a prospect is fascinating to me that I have yet to encounter a real warlocker in uniform. Plenty of submitters who I would characterize as warlockers and people have actually been a great percentage of people who actually been trained for war below the Air Force Academy. There was a very profoundly respected Marine Pearl who was also part of this gig. And he and I were billeted in the very bossy Bachelor of Officer's Quarters at the Marine FAA Air Force Academy. And I said something that fairly unconsidered and offhand way about the human practice was not deeply considered or my part to even use that term phrase with his eyes wide and said, I dream of that. I dream of that. He had led troops in Korea, I think pretty much all of you in this room know what another elite fight that he fighting in Korea was. I was astonished because this fellow was generally known as the Marine Tough Guy in the Mississippi's practice. The only way I don't have to imagine not in the city, not in that city, I don't have an answer to the geopolitical complexities and dangers that we are required to come. I know that's not a satisfactory answer. I was expecting to have a definite answer so I just wanted to put that out there. I mean, I don't know what I see at the very beginning of this but I'm going to focus in the sense that you know Fenerbahce, Transics, Nationality, Race, Generation, Social Women's League I think that kind of perverse the Chair kind of barn you know where there's a cultural gas that exists and then Fenerbahce is our shrunken population. Well, I hope that we're going to shrink a lot of things at a point, but I will allow where it is that soon, based on how everybody's talking in DC or Fenerbahce, some of you are talking here that we're going to have a pretty big new generation of veterans that have just participated in some of the Najiva and other small minorities in every medical way. So it's just a impressive thing about our guests. I think that it's easy to talk about the value of pushing the value and the need for it when you don't understand how it's going to impact you. So, again, the point of understanding where veterans are going through is that as you said, sir, it's the leadership that's constantly making decisions that they need to understand the repercussions of the decisions that they're making and what that actually means to the individual. Because how we're going to have an 18-year war right now, I would say that we were in force and, you know, that's going to be a big ask if we're going to actually, we're going to fight some year-to-year battle. I think after, as I said, 18 years, I don't think they would have to have something that is largely lost on our policy makers. And so, getting this message out, educating people and bringing that signal by is going to be absolutely imperative to making sure that you don't fight yourselves into fighting the war that we cannot outspot from. That we can't possibly do that. But we're going to be a great strong person, so... A land war in Asia certainly fits that description. I mean, if that's what people are forcing which is what I understand to be comments, questions I I haven't gotten any arguments on that disappointment. I have a question. Your description of some of the risks that minor contractors will be there to mobilize a cohesive military unit that's so escalating to the war. I mean, is there a... I'm not going to answer that for long now, so it's very evidence that there actually is higher rates of incarceration higher rate. Do we actually see it? I think that's a very important question and I am an unlicensed philosopher. You might even suspect me being an untethered philosopher because I the number of plausible hypotheses and conjectures that I can generate far exceeds my capacity to research them. And I don't have a clue as to what the answer is if anyone has gathered the data that it worries them that no one actually would be able to do that. That is a product of some of what I think that probably my ignorance of this is the reflection of the fact that for this invitation, it's a great pleasure to be here.


Welcome to today's webinar everyone. My name is Jeremy and I'm a research audiologist from the audiological science department here at now. It is my pleasure to be giving this presentation on behalf of my project team. In Australia we like to start off our speeches with an acknowledgement of the country. So before I dive into this presentation I wish to acknowledge the traditional custodians of the Eora Nation, the land and water on which we meet and I pay my respect to Elders past, present and emerging. And with that I would now like to start the presentation. By the end of this webinar we hope you will gain an increase in knowledge of our approach to this research project as well as an increase in understanding of the unmet needs of clients and audiologists that might be addressed through Connected Hearing Health. Throughout this presentation I will be using the terms tele-audiology and connected hearing health interchangeably to mean the use of technology to bring new ways of delivering hearing health care services and improving communication between clients and their audiologists. In 2019 a team of now researchers from the departments of digital signal processing, communication sciences and audiological science set out to investigate if there were any unmet needs of clients and audiologists in the tele-audiology landscape. The team engaged in a design thinking sprint which is an intensive group learning exercise that span approximately six weeks. During this time they left behind their own assumptions and biases in order to empathize with this population and gathered information about barriers and facilitators to tele-audiology services using surveys and interviews with stakeholders to define the problem. These stakeholders include individuals with lived experience of hearing loss, audiologists, staff members who facilitate tele-audiology sessions and hearing device manufacturers. The sprint team uncovered a number of findings some of which were unmet needs however in the interest of time and relevance to the pre-assessment project I will only present two of these and they are the effect of tele-audiology on the client-audiologist relationship and efficiency. By that I mean savings in time, travel and number of face-to-face appointments required. If you're interested in more details about this sprint I encourage you to tune in to next week's soundbar webinar which will be presented by my colleague and the tele-audiology sprint leader Nikki Chong-White. So the outcome of this sprint resulted in some very ecologically valid data and it also resulted in defining and prioritizing a set of unmet needs that now could translate into research actions to address them and one of these research actions resulted in our pre-assessment tele-audiology project. The pre- assessment tele-audiology project was built upon the foundations of this client and audiologist wish list. What became clear through our literature review and the sprint data is that a lot of tele-audiology services are being delivered for the latter part of the patient pathway but there is a gap, an opportunity where connected hearing health services hasn't received much attention. But what should we be doing in this pre-appointment or pre assessment space? With more questions come more discovery to be done. So the overall aim of this project was to find out what clients and audiologists currently do, want and or need between the time a client takes action like booking in an appointment and arriving at the clinic. The next question we asked is what role does connected hearing health play in all of this? So this time a different research team set out to do some more discovering but with a slightly different focus because we were now looking at a very specific timeframe within a patient's hearing journey. This involved a two stage investigative approach. Firstly the team conducted 15 ethnographic observations of initial hearing appointments across seven different clinics in rural and regional settings. It was a fly on the wall experience and this was important because while any clinical audiologist could describe to you what happened in initial appointments it is very different from experiencing it and capturing the nuances of the interactions and all that happens in the room. It's the idea of stepping into the shoes of the client and the audiologist that required a very different mindset. In fact I'd say that it was a blank mindset.  So not everyone knew what initial appointments were like. We left behind our assumptions and biases at the door and just observed what happened in these appointments. And these observations led our team to appreciate that first appointments are very, very busy. This case history taking, otoscopy, hearing assessments, discussion of results and sometimes a bit of counselling was involved and in some cases a hearing atrial as well. Also a lot of rapid rapport building took place, decisions to be made, some clients had questions for audiologists and others didn't and some people obviously have done some Dr. Googling before they got there. Now what if we can allocate some of these components to free up some time in that initial appointment? What will clients choose to do after they take action and book in the appointment? And what would audiologists do with that extra time if they had it? More importantly my research team asked what role would connected hearing health serve in all of this? And with that we moved on to stage two of our investigations. The second approach we took to further our understanding of this pre appointment space involved conducting in depth interviews with audiologists and clients who had less than two years of experience with hearing aid use. We designed this interview guide in consultation with our public and patient involvement panel to ensure that our research remains relevant and has the potential of translating into clinical practice to create impact. The interview guide was developed with the combi model of behavior change and it consisted of open questions that encouraged our participants to recount their experiences and to explore with motivation and opportunities to manage behavior change might occur. For example, a question we asked our audiologist participant was what would you like to do or talk about with your new adult clients in that initial assessment appointment that you don't already do? And why? The answers will then enable us to see where connected hearing health solutions might fit in and potentially what those solutions look like. For those of you who are not familiar with the concept of a PPI panel, it is a consultative panel made up of stakeholders who has direct experience with issues we're trying to solve. For our PPI project, our PPI panel is made up of individuals with lived experience of hearing loss. They are not research participants, but they're part of the research team and their role was integral to co-designing of our research questions and methods. These interviews were then transcribed and our team conducted thematic analysis of the transcripts to extract common themes. And here's what our people had to say. For the client interviews, we found three recurrent themes emerged. And these relate to preparedness, expectations and trust. I'll now share some of these interview segments with you to illustrate the point. On preparedness, client one reflected on and compared his or her previous health care experience with their hearing appointment experience and said, Yes, information would have been brilliant. Really important. I know when I had an operation in 06, I was given a lot of information to make choices. So I felt that prepared me a little and I think now I go into every situation with that expectation. I'd like to have time and I'd like to have a buffer to take it all in and think it through. So clearly client one was anchoring his or her expectations that they would have some information about what to expect in their first appointment and would appreciate having the time to digest new information. On expectation, client number four said, I had no idea it would be so hard. I thought it would be like putting on glasses and walking away. To us, this demonstrated that the clients like our participants have encountered a mismatch between his or her expectations and experience with the hearing journey. So perhaps this is something that could be addressed before the client arrived at the clinic. On trust, again, this quote came from client number four, who was answering an interview question about whether he or she felt comfortable discussing the hearing issues with their audiologist at the first appointment. And client four replied, I do think it's important. It's a bit like a doctor. You have to have someone you feel you can trust. And we know that there's a lot of literature out there indicating that successful hearing outcomes for adult clients are associated with the strength of this client clinician bond. So it seems to us that there is an opportunity to address preparedness, expectations and established trust. If we consider capitalizing on that time when clients are doing nothing but waiting for the day of the appointment to arrive, which sometimes could be a day or two or even two weeks in some cases. What if we can address these while they're waiting to see you? And how might that happen? So I hope you remember what those three client interview themes were because we're about to experience a deja vu moment. This slide presents the three key themes from the audiologist's interviews, which are preparedness, motivation and rapport. On preparation, audiologist number one said, I will then ask whether they have had a hearing test? What do they think of hearing aids? Have they done any research on hearing aids? Do they know anything about the cost of hearing aids? And are they prepared to consider a hearing aid? The quote continues for a little longer with the audiologist asking a hypothetical client some more questions and then it ends with some assessing their motivation. What's interesting here is that the audiologist equates the term preparation with client motivation. Whereas when clients mentioned preparation, it was usually in relation to wanting more information to help them make more informed choices or wanting some more details about what to expect in that first appointment. So this highlights the need for some clarity around what both parties mean by the word preparation. On motivation, audiologist who said, one of the things I think was interesting for me, it is important to try and work out early what is a client's or what is a person's reason or motivation to come? Are they a reluctant dragged along participant? Or are they yep, I'm really motivated and keen to improve my hearing help. If you have that information fairly clearly, it does tend to frame that discussion. So what we're hearing here is that audiologists prefer to gauge a client's motivation and reason for attending an appointment very early on so they can plan ahead for those discussions in that first appointment. Lastly on rapport, whoops, rapport. Okay, again, we have audiologists too who said I'd like to explore that bit a bit more and not feel rushed and cursory. That's potentially valuable. It's almost like a mini counseling sometimes and you can't do that in a hurry. If you really need to get to the nitty gritty and help tailor things a bit more for the client, then sometimes a bit more time would be good and not feel that you're having to rush. And there's that next person waiting out there. So we've captured that answer from audiologists too when we asked them if you could change something about the way your first appointment to currently structured. What would that be? So it seems we're staff clinicians want to spend more time building up that rapport and trust with their clients, but they don't currently have that extra time to do this. Again, what can we do in this pre-assessment space and how? So to sum up the bigger picture, the findings from these interviews revealed a converging needs and wants of clients and audiologists to be better prepared by the time they arrive at this initial appointment and build that trust and rapport very early on. And what we're seeing is an opportunity for audiologists to reconsider the clinical pathway. So while clients embark on their patient journey much earlier from the moment they're aware of their hearing issues and making that appointment to see an audiologist, what can audiologists do to match the start of this journey? What can they offer to their clients before meeting them for the first time? Or what do they currently do to facilitate client preparedness? And how do you establish this trust and rapport even before the client attends their first appointment? And how can you enhance this via connected hearing health means? Lastly, this type of research would not be possible without our research participants. These are individuals with lived experience of hearing loss, their communication partners, their families and audiologists, all of whom generously gave us their time and trust to champion the cause to help improve hearing health and wellbeing for many people with hearing difficulties. So if you wish to take part in shaping our research, here is our QR code to sign up to our Hearing Health Professionals data piece. And it's a very quick five minute process. I'd also like to shine the spotlight on some upcoming soundbites webinars on connected hearing health.  So please keep your eyes and ears out for the soundbite schedule for these talks and many more. Last but not least, some acknowledgments that do. I'd like to say a big thank you to our research participants, our PPI panel members.  Thank you also to the tele-audiology sprint team that generated the research project that we're currently doing as it was based on the perceived needs of these clients and audiologists. And thank you also to my wonderful, wonderful project team, Mel, Niki, Margot, James, Jess and Liz. And all of this will not be possible without the wonderful now soundbites production team, Scott Brewer, Jess Cooper and our lovely Xiao Ying. You're doing a fantastic job.