Please be welcome to this fantastic kid here. I learned a lot from him, even though he's only since two years playing around with iOS. I mentioned as well the first untaturing case that I had with iPhone was something like with iOS 5.1 and with every update you had to do the whole shebang again, of course, that's what I remember. So, please give a warm welcome here to Little Lilo. I spelled this correctly. Yes. Little Lilo is really a very fascinating geek. He actually hacks any kind of OS device to my opinion, any kind of it. So throw it to him and it comes back in pieces. He's as sharp as a knife. Please give him a warm welcome and applause. We're going to untatter iOS. Yay! Okay, so yeah, I'm Little Lilo as already introduced. I'm interested in IT security in general and I started to look into iOS about two years ago. And then I met three awesome guys, Sigourza, Steg and Ivy Sparks. And we basically started chat a lot and in the end also released a jailbreak for iOS 10, on toilet, not spyware at all. So for everybody who isn't familiar with the jailbreak scene, it's like rooting on Android, so it basically is a tool that gives the user the capability to tweak their device and that's mostly done by basically installing a jailbreak. In this case, you just go to a website and then slide there and it will install a package manager on your phone. And then the end user can install tweaks, so just little dynamic libraries and they get injected into all the other system processes and then they can obviously modify their memory and that for example allows customization of the home screen or something like that. And basically with the release, we also formed a team that jacked there for 20 team and it was created and with that the Twitter account and there are also a few demos there if you want to check it out after the talk. And yeah, I basically had this pipe dream that in my life I really want to achieve an untatter jailbreak using only some sort of P-list or other safe file corruptions. So basically there are different kinds of jailbreaks. In modern jailbreak, the most common kind are semi-untattards which means that the user has to connect their device to a PC when they first install the jailbreak and then they side load an app onto the device, so it just installs itself there and then on each reboot, the user has to go into the app and press a button to jailbreak the phone. And basically after they reboot, they lose the jailbreak and have to go through this process again. And with an untatter jailbreak, the jailbreak gains persistent installation and then they will automatically jailbreak the device on each boot, so the device already jailbreaks jailbreak, boots jailbreak. So yeah, the untatter can be divided into four stages. In the first stage, that's the config file or database. Stage, I will go over how we gain execution of the boot. Then the config parser bug that gives us the right-to-wear primitive. And then I will talk about the main bug, the ASLR bypass, which allows us to get into ROP. Then stage two is basically a kernel exploit and completely written in ROP. And there I will go over the two main kernel bugs, so the KSLR leak and then the Ray C-33 we use to get the kernel read-write. And I will also talk about the KSLR weakness and the root domain user client memory leak, which aided us in exploitation. And after we get kernel read-write, we can remove a few restrictions from Apple. Mainly, we can bypass code signing and that gives us the ability to load an unsite dynamic library into our process. So now in stage three, we are basically in C via a dynamic library and there we are stashing the kernel task port to host special port four. So iOS has this concept of ports, task ports. And basically, if you have a send-write to a task port of a process, you can read and write its memory. They are most used for inter-process communication, basically. And where modern jail breaks use these, stash the kernel task port into a special port four so that other user mode applications can then retrieve it and with that retrieve the send-write and then they can just read and write process memory. And after that, I just fix up everything and then spawn stage four. And that's basically running in a separate process that has a few advantages I will go into later. And it's basically performing the whole post-exploitation process, including launching substrate, the framework that's used for tweak injection and then performing LUE start, to restart all launch demons on the system and inject the tweaks into them. So yeah, in stage one, we basically need to get executioner of the boot. And when iOS boots, launchD is the first process that's running on the system and it basically then loads a dynamic library with a list of executables that are launch demons and there are a few flags associated with them. And if the run and load flag is set, launchD will then spawn all of those launch demons afterwards. And we basically just attack one of those launch demons namely Raccoon. So what is Raccoon? Basically Raccoon is part of the IPsec package and it's a VPN client used to interact with an IPsec VPN server. And the problem is though that the IPsec tools project has been abandoned since 2014. So they obviously here state on their website that the development of the project has been abandoned and that IPsec tools has security issues and you should not use it. Apple still decided it was a good idea to ship it after 2014 and they now maintain their own fork on opensource.obl.com. And yeah, basically the only thing that's important for the talk is that Raccoon has a configuration parser. So on startup it will just look for a file on disk and start to parse that. And that's written in YAK, so completely written in C. So memory corruption might become a problem and it actually is a problem as we can see with the config parser bug. And for that one we have to go back a bit. Basically in 2011, Poch2G released Corona for iOS 5 which was also an untethered. And there he used a bug in Raccoon in the configuration parser. And Poch2G found this one an old bug tracker and basically the IPsec website had this bug tracker there where users could just report problems with their programs and one user in 2009 I think reported a bug that if he would use this specific config Raccoon would just sack fault but nobody looked at this for two years. So in 2011 Poch2G realized that this was actually exploitable and then used in a jailbreak. And after that obviously Apple got a bear of it and then they also said that they fixed it and it got a signed CV 2012, 3727. And yeah, they say that the issue was improved bounce checking and just improved bounce checking. So let's look at the patch. That's the function before the patch and that's the function after the patch. And if this was too fast for you, here's the diff. So basically there is no difference. The bug is still there and Apple didn't patch it. What really happened here was that basically this is obviously a configuration for parser. So there are different statements that nearly do the same thing. There are these DNS4 statements to parse an IP address and there are Wins4 statement followed by an IP address and for both of them they have to parse the IP address. So the function for parsing that was just copy pasted from another one and yeah, basically Apple fixed the bug in the other function, which I don't get because basically the engineer there had to look at it at the PUC Poch2G provided. Then they had to realize, okay, the bug is in there, fix it and then they had to recompile it and never test it against the original PUC because otherwise Raccoon would have crashed again. Yeah, it's basically an off by one which allows you to overwrite the index of an array and Poch2G gave a talk about this and we can basically reuse his strategy from back in 2011 to get the right repair primitive. So yeah, again, the one will function. Here at the top you can see the Yuck syntax. So an address wins list has to consist of either an address wins statement or an address wins statement followed by a comma followed by an address wins list. So that's a recursion there. It can be multiple address wins statements in a row and then below that an address wins statement is to find this containing an address string and that's just a regs to mention IP version four address and if the parser finds that it will run the code between those two brackets, so they get the pointer to a global struct and then they check if this nminus four index is bigger than the maximum wins and if so they return an error the problem here is that this is the off by one. It shouldn't be checking if it's greater than so but it should be a check for greater or equal than maximums and after that they use this in a pattern function to basically parse the address which is in this variable into the nminus four array at the nminus four index and then post increment the index and that will also become in handy when exploiting the Spark. So yeah, how is exploitation done? Basically there are here the globals. There's this LC config pointer in the globals pointing to a heap structure that the parser also uses and then there's this dns four array followed by the dns array index and then the nminus four array followed by the nminus four array index. So we can just use a modcfg statement followed by this wins four statement with the IP address to parse the IP address in the first array element and we can repeat this process four times to then move the index out of bounds so now it's pointing to itself and the cool thing is about that is that they thought it would be a good idea to use integers for those array indexes instead of unsigned integers so we can just point them, you just use negative number to bypass the bounds check. So yeah, we will override with minus one to point it to the dns array index and then we use another win statement to override the dns array index with a negative number and the cool thing about this is that because of the post incrementation it will move it back to zero so we can repeat this process as often times as we want and we will use a negative number to basically point it to the LC config pointer and then we can use two dns four statements to override the LC config pointer and point it anywhere in memory and after that we can use a timer statement with a counter and the parser will just try to write it to the LC config structure so it will dereference our pointer and write it there and the interval statement will basically write to UN32 which is followed by the UN, where the counter statement is written so we can turn this into a 64 bit write what we are primitive and write anywhere in process memory and that gives us the ability to basically exploit the ASLR bypass so this is the main bug of the whole untatted and allows us to basically have this primitive and it's also the reason why I decided to give this talk this year and not last year because there the bug was pretty fresh and only patched in iOS 12 and it gives an attack of the ability to exploit zero click more easily because they just need to write what we are primitive and that's why I held back on it. Seagruses are founded after looking into Apple's patches of the Corona ASLR bypass and basically from POTCH2G's presentation we as a team learned that ASLR is always bypassed when a writable region is larger than the maximum slide because then there's always a writable address in process memory and you can use this to brute force the slide and Seagruses found out that there was also the same problem with the cache in iOS 11 and a few versions of iOS 10 actually so what's the cache basically? The cache is a pre-compiled blob containing the dynamic link libraries from Apple so in each release they just move all of those into one big file to optimize load times of apps and keep memory pressure low so it's a pretty big file and for that on boot the kernel needs to calculate a slide and slide it in memory but they only slide it once on boot because it's used in every process and so Apple defines an area of memory for the cache with a base address and a size and that's hard-coded on iOS 11 the cache got so big that the maximum slide is now smaller than its data segment so the condition here is satisfied and ASLR can be bypassed basically and the weird thing about this is that actually the same thing also happened in iOS 7 back then the size was defined as 500 megabytes and the cache got bigger than 500 megabytes but before that it was also in a few versions possible to have the same condition that the data segment again was bigger than the maximum slide and Apple was actually also aware of this because ASLR talked about it in one of his talks but they just increased the maximum slide to one gigabyte and didn't add any asserts and now in iOS 11 it again got so big that the same thing happened and we believe that they didn't even really notice that up until iOS 12 where the cache was now bigger than one gigabyte so the kernel just couldn't fit it in the memory region and panicked and because of that they just thought well then we will increase it to four gigabyte while developing so we might see this reoccurring in, I don't know, iOS 15 or something like that. Exploitation is also pretty simple. We can just put forth the slide now and there are a lot of function pointers in the data segment so we decided to go with platform memmove because we can read it via Lcopy from the config parser and the basic strategy behind this is we write a slit ROP chain to an address we can always write to for a specific slide ABC and then we overwrite the platform memmove pointer at its unslit address plus the slide we just chose with a gadget that would pivot to our slit ROP chain and then we call it Lcopy and now there are basically two possibilities either we guess the right slide and then we jump to our gadget and pivot the stack or we guess the wrong slide but obviously nothing happened because we can just write there so we can go back to step one and try again and then with that we can just put forth the slide till we got the right one. The catch with this however is that the rightward way is pretty slow and it also needs a lot of storage so for a 64-bit write, I need about 120 bytes in the config file and because there are so many possible slide values the chain I have currently is around it's only two ROP gadgets but the config file is already 12 megabytes I think and because of that it takes around 10 seconds to run if it's a really bad slide so the ROP chain has to be as short as possible and the other problem is that if we have a bad slide we will basically smash the whole data segment and we can't really recover from that so we had crashes in malloc and stuff like that while developing basically. There are some solutions to that we can have a really short ROP chain in stage one and we can achieve this by basically just opening the big cache file to get a file scripted to it and then we can map it at a static address and then get many gadgets there because the file is obviously code signed by Apple so we can just jump there after mapping it but the problem with that is that nothing is set up so malloc and other functions aren't working but as I said earlier the current cache has to smash data segment so we don't really lose anything and after having the cache at a static address we can use the open and MAP functions from that cache to basically map stage two into the process memory and stage two is just a really big ROP stack so yeah. And then we are basically in ROP but we can only use raw syscalls and not much more because everything else will either use a function that uses malloc or will use malloc on its own and the other problem is that basically Erno is also broken so every syscall which fails will crash the binary so the first thing I do is basically map a few pages so that Erno works again because we have a free use discourse that might fail and then the other big problem is that we now need to implement a whole kernel exploit in ROP so I start to write a framework around this awesome gadget which is present in all the cache versions I looked at basically at the top you can see that all the high registers are moved into X0 through X7 so all the registers used for the calling convention ARM and then we branch link registers based on X27 so also a high register and after that we load all the register values back from the stack so we can just chain those gadgets together to call any functions we basically rob into the lower half and then chain another one of those afterwards and then they can load all the values call the function and then load all the values from the stack again and that's how the whole ROP chain works so yeah, I also used another gadget to basically add two registers together and another one which stores X0 so that I can just store the return value on the stack and later reuse it and for loops I use a gadget which just returns if X0 is zero so it's basically just a red instruction then and otherwise it will tail call a function using function pointer from the data segment and because I can control the whole data segment I can just put a function pointer there that will then jump to an epilogue and misses line a stack with that and then I can call long jump with two different register values and because of that pivot the stack to another location and when we basically didn't jump outside of the loop I just emmept the part of stage two which has the loop in it back over the old stack again and then I can just reuse the stack every time and then pivot up using long jump the problem with this however is obviously that it's pretty slow because I used MAP as a score but this can be solved by just unrolling your loops like for 10 iterations and then emmapping the file so that I get more loop iterations basically per second which is important for the race so now we will go over the kernel bugs so the KSLR leak is a CVE 2018, 4,413 by Panigall it was fixed in iOS 12.1 and it's luckily reachable from the Raccoon Sandbox because Apple Sandbox is most useful in processes and the Raccoon Sandbox is really tight we didn't have that many bugs that would work in Raccoon but luckily this one does because Raccoon has access to the CCTL functions and this one is in the CCTL ProgArcs X function the ProgArcs X function basically allocates a page using Kmalloc but it doesn't zero it so it might contain old heap data and then they copy the process arguments to the front of the page and now if you supply a wrong size from user space and there are a few other conditions that have to be met for some weird reason it copies from the end of the page into user space which I don't get by this even good through code review but there's a graphical illustration there's basically this page and this full of uninitialized data and then they put the process arguments in the front and copy out uninitialized heap data from the back into userland and we can obviously just spray specific heap objects with kernel pointers in them and then free them again and use this primitive to maybe leak them and if we find a kernel pointer in there we can just calculate this kernel slide so yeah and then we come to the WAC-23 as I said the main problem with the antenna is the Raccoon Sandbox so many of the kernel bugs that would work in iOS 11 from an app didn't work from Raccoon but luckily on the Halloween Spark it told me about the lightspeed bug from Scenetic which is reachable from the Sandbox it's a double free in Kellogg-16 so the kernel allocator is based on zones and with different sizes so there's for example Kellogg-16 and then all objects in that zone have a size of 16 or less bytes and there are obviously also other zones for bigger objects and basically there's the structure in there that then gets doubled and so there's the syscall that's handling async file events so a user mode application can just call in and tell the kernel, hey please write a buffer to a file and then move on with execution and it uses a kernel thread to perform those and for that it basically allocates a structure to contain some metadata like where the buffer is and to which file it should write and the kernel thread obviously has to free the structure after it's done because it just gets encroved into the query for the kernel thread then that wakes up and maybe sees it has a new job then performs the operation and then it has to free it, otherwise it will leak but the problem here is that if an error occurred while setting the structure up the second field in the structure will be zero and then the structure also isn't encroved into the jobs list so the kernel thread will never wake up and look at it so the syscall has to free it because otherwise it will leak and the problem here is that we can basically reallocate the structure before the syscall checks so what happens here is that the syscall allocates the structure and fills it up and then it gets added to the list and then the kernel thread is so fast that it runs while the syscall code isn't finished yet and basically gets the job done and freeze it and then we can spray all of we can spray the objects pretty fast to overlay with that structure and then the syscall finishes and checks the field and sees that it's zero because we just replaced it with an object that has zero at that location and so it frees it again, leading to a double free and yeah, we can obviously exploit that so for exploitation I just spam the syscall in one thread which is pretty hard to do in Rob but I just call thread create with a pointer to long jump and then pivot the stack to that location and then in the other thread I spray mach messages with mach port null in it and the thing with mach messages basically they can be used to do inter-process communication and you can also transfer port writes from one process to another so in this case we just send an empty port but you could also place something else there and that will create a structure in Kellogg 16 containing zero at that location and then if the mach message gets freed we can replace it with something and basically point it to somewhere in Kernel where a fake port structure lives and when we receive the mach message again it will basically think that this is a real port and treat it as such and with that we can create a fake kernel task port so yeah, but for that we obviously need to replace it and we need a heap spray and most commonly our surface is used for that as a kernel extension but because of our sandbox we are so limited that we don't have our surface access so the question is how we actually spray and their root domain user client comes to rescue with a memory leak so actually this function secure sleep system options is reachable from this Raccoon sandbox and Apple has a way of basically passing data to the kernel via XML so a user land application can just pass the XML to the kernel and then they will use this always answer as XML function to turn the XML back into C++ objects which the kernel can then use and if this sounds dangerous to you it actually is there where a few bucks in that but in this case we basically this just makes true with the OS dynamic cast that the data, the user mode application supplied is an OS dictionary so that it can use it afterwards and the problem here is that we can basically just supply an OS data object or an OS array so this OS dynamic cast will fail and un-serialized options will become null but the original point of return from OS un-serialized XML will get lost and so we will leak that memory and yeah we can just use this for spraying so then about those two primitives we use to basically exploitation the case law weakness there are these two CTL buffers and they are located in the kernel data region and because of that they are split with the same slide as the kernel text region and this means that as long as we know the kernel slide and we already do that from the KSLR leak and we can control the contents of a CCTL buffer we can get data to a known address and we can easily do that with Raccoon because it runs its route and so we can just switch our CCTL buffer and for example place the fake port structure there or we later also place a fake trust structure there but I will get into that so yeah now we can use the primitive to basically spray 10 OS data objects pointing to the CCTL buffer and then we just receive the mach message again and check if the port is null and if that's the case we basically place the structure and then for the non-SMAP version we can even get the case like by traversing a few pointers but that's not needed for the SMAP version because we already got it with the KSLR leak but non-SMAP devices we also don't need the CCTL buffers because we can just place the fake port structure in user land and then we get the kernel slide this way and with the kernel slide and this fake port we can create a fake user client and from there we can create a call primitive and then we can use this to overwrite the DSS trust cache pointer and point it to a buffer with two hashes for stage three and stage four so basically Apple has two ways of doing code signing either it has a user land demon that verifies third party applications or it has this so-called trust cache which is a list of hashes from all of their system applications and as soon as the process is spawned or a dynamically linked library is loaded and they will basically first verify if the hash of that file is inside of the trust cache and if so they will just trust it binary blindly because it comes from Apple basically and now when we overwrite this trust cache point and point it to our buffer we can basically place the hashes of stage three and four there and then the system will think those are Apple binaries and we can just load them so yeah and for that we need to use the ghetto DL open we can't use the real DL open because that uses malloc so we just open stage three to get a file descriptor then we attach the signature which now succeeds as the hashes and trust cache and then we can map it as read execute and jump there and then we are after two months of writing dropchains we are finally in C and we can write code more easily and the problem there is that we still don't have a working cache so we are still limited to the basic functionality and because of the ghetto DL open linking is obviously not working so we just rely on raw assembly for assist calls and I also pass some function pointers which I already used for stage two so for example open and then map to stage three from there we remit the kernel task and stash it into our special port four so that other user mode applications can use it and then we can basically escape the sandbox by zeroing the sandbox label in the process structure so that we can launch a new binary because otherwise the raccoon sandbox doesn't allow it but in the kernel those process structures basically have this label which tells the kernel which sandbox to use and by zeroing it you can just tell it to not use any sandbox and then we can launch stage four and with that get a working cache back and that's the big advantage from like having a separate file we now have the full cache functions working and can do work more easily and then it's just a call to poison spawn and then a raw exit syscall to exit the daemon without crashing because if launch D would see that one of the launch daemons crashed and the specific flag it said it would try to restart it and then our rope chain would run again we obviously want to prevent that so we use the exit syscall to exit it cleanly and then we are in stage four and from my side that was just basically to block all signals so we don't get killed by launch D because when launch D sees that the launch daemon exits it will send a sick kill to all its child process and I need to catch that otherwise stage four would get killed and then I just called the post exploitation framework which was written by Sparky and basically that does the following it first elevates the process to root with kernel credentials then it performs a remount of the root file system because on stock iOS the root file system is mounted as read only and we obviously need to mount it as read write to modify some files on there and then it sets the nonce so that the user might be able to downgrade to an older version if they have blobs verifies that the bootstrap is in place from the installation and then injects substrates so the framework that's used for to perform tweak injection and it's plug in into the trustcast and starts them so that they can start to inject into newly spawned processes then it spawns all the launch demons associated with the jailbreak unloads our own demons so that we don't respond it by accident run the kernel exploit again and then performs an LD restart to basically restart all of the launch teams of the system so that subset can inject tweak stem and with that the system is fully jailbroken and we can perform a few cleanup steps but yeah basically the end user has a jailbreak system then as a little side note while we were testing all the demons we got killed by jetsam a lot so basically jetsam is this kernel extension from Apple that is there for memory management and they basically want to make sure that the user mode application doesn't use too much memory because they don't have that much on an iPhone or all the iPhones actually so there is this list and jetsam if jetsam sees that a user and process uses more memory than it should use it will just kill it so we change the values in the pillars and the launch demons to actually allow the launch demons to use more memory but the weird thing about this is that this actually got accepted by jetsam and we had no more crashes while Apple actually tried to mitigate that beforehand so because jailbreakers always would modify those configuration files on the launch demons they start to move all of them into a dynamic library to guard them under code signing so that jailbreakers couldn't change them anymore but when we tried to figure out the launch at the launch order of demons we dumped that develop and there was also a playlist embedded for jetsam but Apple was still using those files on disk so I really want to look further into this because it seems like Apple isn't always ignoring those configuration files on disk and yeah then thanks to the whole team Sigus as Sparky and Stack for bouncing ideas back and forth and writing the remaining part of the jailbreak then for Poch2G and Synetic for the kernel box and basically also a big thanks to Zorik for Substrate and the whole jailbreaking framework and for Swagger, Parakeet, Geek and Smegas, and Ninsha for testing a few things and keeping me motivated and for Jonathan Levin for his books basically because he bought a few awesome books about iOS and that got me into it two years ago and yeah and in the future I think exploiting kernel vulnerability with other cache functions and only in Robb really is a pain and I probably won't do it again because I spent most of that but yeah the other big problem now is that with A12 so the newer iPhones pack so point of authentication kills most of these types of exploits because the problem there is that you would now need an ASLR bypass and the pack bypass to get into return oriented programming and it's pretty unlikely to basically have both because pack bypasses are even really rare and yeah I only know about this one ASLR bypass so yeah you would have to get pretty lucky also untill ring gets progressively harder Apple just fixed another good idea ahead in iOS 13.1 basically the idea was to use printf with the string format modifier percentage N to get a Turing complete machine because printf with this modifier is basically Turing complete and then use that to develop a pack bypass basically and get into Robb but now in iOS 13.1 I think Apple actually removed the percentage N modifier so you can no longer do this and yeah so this idea is also gone and yeah in the end I was able to complete my pipe dream so I guess I will need a new one so watch out Apple and that's Spice are there any questions? Thank you little Lilo for his fantastic work you know I suppose we're gonna hear more from you in the future maybe let's see I'm pretty sure are there questions here in this audience no one who wants to hire this guy now right away no one no one can you describe me what changed actually two times in these you know all the OSX OES OES versions what they changed to make this possible you know I told you like I started the the Turing challenge actually at 5.1 well they added a lot new mitigations and also obviously patched a few bugs like for example those ASLR bypasses that POTG2G used in Corona got patched and this one also now got patched by accident but yeah I mean like some bugs are still there for example the bug in Raccoon the config bar, the bug is still in ODE but yeah I don't really care about it and the kernel bugs got patched by Apple but for example the synthetic one they also patched wrong by accident and now it always leaked the struct but I think they also fixed that now your team you're mentioning your team you're working another in your room of course no and how are you structured how are the roles divided how is the well we are just like four people so and we have this group chat and then we are just hanging out there and bouncing ideas back and forth and maybe working on some stuff like close contact with the Apple developers no, not at all nothing no more I mean I reported one bug to their bounty ones or like actually just to them, not their bounty and that one got fixed and it was all fine but yeah for now I don't report bugs at the moment if you have time you have time left now actually you're looking for a new project isn't it yeah yeah and I might report some of those bugs in the meantime but I mean with the presentation they also know about them now so they might fix them they will be listening now and then at least probably I hope yeah is there anyone who has really we're sitting on a question here none of you it's already noon you know noon past so could be that none of you you can ask them something maybe someone else something maybe they can help you out with certain challenges that are there well I don't really have a question either I have my own research project now where like I do stuff at the moment and look at other things for example the the bootrom exploit came out now and so I yeah started developing on the check-rain team with them and that's what I currently do basically you're great man little Lilo thank you give him a warm-up applause


of the disability access advisory committee to order on April 20th at 11 30 in the morning. And what's the, could we do a roll call, make sure who's here. Okay. Myra here. Marty at least link. Yes, here. Ruth Smith here. And no other members present. So at the moment, Xander Crowley, Sarah and Darren, and oh, come on, my brain isn't functioning. Tori Dixon are not here. And hopefully they will be. But this meeting is to talk about Emily Dickinson homestead application for variants for a very large and impressive construction project, restoration project. And we have architects here. Who do we have? So, um, let me for one second. So we have, oh, Sarah is here. Well, let's just let Sarah join. We have Jean Wald, who is on, well, I was gonna say the historical commission, but that's, that's not of relevance. I don't think, but you are the president of the Amherst, of the Emily Dickinson Museum and homestead. Is that correct? I'm, I'm the executive director. Thank you. Sorry. Yeah. And we have Shantae Andrew Hagan, if I'm pronouncing your name correctly. Pretty close. Shantia. Shantia. Oh, thank you. I knew it. Yeah, it's a mouthful. And you are a historic preservationist. Is that correct? Yes. Yes. I'm consulting with the Emily Dickinson Museum on the project. Great. Okay. And so I guess we expect any of the architects, are they, are they planning to be here? They're not able to attend today. They're not able to attend. Okay. Well, you sent us what I think is the most thorough application for anything that I've ever seen. I used to be on committees that looked for, you know, put out RFPs for architects. And I've never seen anything this complete, I have to say. It's pretty impressive. So I think has everyone on the committee read the materials that were presented? Yes. Yes. Yes. Okay. So do you want to give us, since we've all read it, and it's very long, you don't have to go through all of it, but you are looking for two variances, one for the front door, front entrance of the Emily Dickinson, and one regarding the need or not need for an elevator, right? That's correct. Yes. Okay. So is there anything that you want to talk to us about specifically before we have our general conversation about it? I'm here prepared to just hit the high points with, in verbal description and also high point with a PowerPoint, some illustrations, if you would like to see. Okay. That would be good. Sure. If that would be helpful to the group, that's fine. Great. Then I will go ahead and pull this up. Okay. All right. So we're here to, as you explained, to request variances. We had variances in a previous project approved in 2016, but they are now expiring because of the value of this pending restoration project, which exceeds 30% of the assessed value of the property. So I'd just like to give you an overview of the museum property, the project we're bringing to you for your consideration, the request for variances, as well as existing and proposed ADA improvements. So the Emily Dickinson Museum is comprised of two historic houses connected to the Dickinson family, the Homestead, where Emily Dickinson lived and wrote nearly all the poetry we know of, and the Evergreens next door, which was the home of her brother and sister-in-law. The current project is focused at the Homestead, and it has a number of historic designations. It was determined to be a national historic landmark in 1966, has been listed individually on the National Register of Historic Places since then. It is a constituent property in the Dickinson National Register District. It is under the requirements of a preservation restriction with the Massachusetts Historical Commission because of various state grants we've received for its preservation in the past. And it is also a part of the Dickinson local historic district. The project we're undertaking is the most substantial project we've ever done at the Emily Dickinson Museum at the Homestead. And it's comprised of a couple of elements. First is mechanicals, the replacement of a heating ventilation and cooling system within the main block of the house, as well as rerouting and upgrading the electrical service entering the Homestead property. This portion of the project also includes the installation of a new furnace generator and chiller. The generator and chiller will be at the north end of, it's apologized for this map, it's difficult to see, but at the north end of a garage on a pad, so it's out of public view. And the installation of a transformer due east of the location of the garage. So at the other end of the property on the boundary with 20 Triangle Street, so it's just north of the lot on which the Homestead sits. The outside mechanicals will be enclosed with a seven foot high painted wood fence on the east and west and a nine foot high fencing on the north side. The north elevation of the garage becomes the south wall of the mechanicals enclosure. So part two of the project is partial interior restoration of seven rooms within the interior of the main block of the Homestead, that's the original 1813 portion of the building. And these, all of these areas have already been generally interpreted and open to the public, so we are wanting to take them from their roughly 20th, mid 20th century appearance back to setting in a presentation that Emily Dickinson herself would have been more familiar with. So the specific spaces are the northwest and southwest parlors, which run the entire depth of the house, then the two large stair halls on the first floor and on the second floor, a transverse hall between the dining room and the parlors, then a room at the top of the stairs, the northwest chamber, a passage and a closet between that northwest chamber and Emily Dickinson's room. So that would be returning it to its configuration in the 19th century. So the interior scope of work includes the removal of this 20th century strict flooring to expose the original wood floorboards. And then those original wood floorboards would be covered with period style floor coverings. There will be repairs to the interior doors because when the strip flooring was put in, the level of the floors was elevated, meaning that the bottom of the doors had to be reduced in order to clear the flooring. There will be the reinstallation of two pocket doors that separate the parlors. On the left, you can see the opening between the north parlor and the south parlor. And in that opening, there used to be two pocket doors that could close those two rooms off from each other. There will be, in the center slide, reconstruction of the front entry door and its surrounding masonry. We found, we were able to find the original 19th century door in the garage behind the house. And it's still in very good condition and we'll be able to put that back in place so that that front entry looks as it did in the 19th century. Let's see. There'll be a restoration of transom windows in the transverse hall. Removal of non-original architectural elements at the main staircase. So the main staircases here on the right along with the original front door stored in the garage were the original balusters from this staircase. So we will be able to reconstruct that staircase. Let's see. And some other items such as reconstructing the chamber boxes, I'm sorry, the fire boxes for in certain rooms where the floor level is going to change. And then to reinstall a mantle in the bedroom at the top of the stairs, again an item that was found in the garage. It's really interesting why and how the people who occupied the house in the 20th century took those things out and just kept them on site. A bit more of the work is, let's see, reconstructing a staircase that leads from the second floor to the third floor. Replacing interior storm windows. Removing the 20th century staircase from the second floor to the third floor. And installing insulation in several areas of the third floor. So that's a very long list. So it's quite a bit of work and we're really excited about what the outcome of this project will be and I wish I could show you something about what that looks like. But the variance requests we're asking for, their sole intent is to ensure that the historic character of the homestead is preserved. Whether through maintaining the existing architectural building elements or restoring and reconstructing missing elements features and finishes to recreate the appearance and function of the building to what Emily Dickinson would recognize during her lifetime. So the restoration approach to doing that means not introducing some modern elements that would make accessing the entirety of the house easier. There are, however, ways that the museum provides wide experience to the property and its history. But these are the two variances we're requesting, section 25 entrances and section 28 elevators. So what are the accessibility measures that we have been placed so far are an accessible exterior parking space directly behind the main entrance to the homestead and signage for that. There's an accessible exterior walkway and entry door and door hardware on the east side of the homestead rear addition. So that that goes into the same space that all visitors enter. There is an accessible interior toilet room. The interior doorways are wide enough for wheelchair access to the entire first floor, a water cooler in the public tour center, and an additional handrail on the main staircase from the first floor to the second floor that is ADA compliant. So that staircase has two handrails. We also have materials for visitors who can't ascend to the second floor in print, in video on an iPad, and are staffing in that area for during our regular tour operation. There's always someone on the first floor to talk about, describe, and interpret the second floor spaces with those who remain on the first floor. So we're going to keep all of those improvements in place and our restoration program brings opportunities with it for additional interpretive techniques that can serve a variety of visitors. So ultimately we'll be introducing sound, directed sound into the spaces. We'll have some projections to assist with visual interpretation. We expect to have some tactile materials, one of the big changes in this restoration is going to be introduction of carpets and window treatments, and all kinds of textile materials that give us an opportunity for for tactile interpretation. Following the restoration there will be a 360 photography of the entire building and every room within the building. So that will have the ability to have a 360 tour of the house of all the restored spaces for those who aren't able to reach the second floor. In addition, this year our plans include installing a compacted accessible path between the homestead and the evergreens that will facilitate wheelchair and scooter transportation between the two interpreted houses. And then ultimately in a future phase that's pretty well down the road six to eight years from now we're intending to reconstruct the barn that used to stand behind the homestead which will be a fully accessible universal access building. So that lies in our future. So that's that's our basic program and that concludes our formal presentation. And I guess the last thing I would say for now is that we're wanting to renew basically renew the variances approved in 2016. And this Access Advisory Committee offered a nice letter to the Massachusetts Architectural Advisory Board to support that request for variance. So I'd be glad to take questions or respond at any point however you would like. We have any questions? By the way, did Xander ever get here? We don't have Xander? No. No. But I believe Saren, I forget who was the last one who slipped into the meeting. Saren is here. Yes. Yeah, but Tori is not and Xander is not. Tori's here. Yep. Oh Tori is. Great. Okay, cool. I have a question about the garage or the barn or I forget the thing you're going to build in the next six to eight years. Yes. What was that again? I forget what you said. Oh yeah, in the 19th century a large barn stood behind the homestead and our goal, an important goal for the Emily Dickinson Museum is to reconstruct that building so that we can remove all of the administrative functions from the homestead and be able to restore the entire house. So that barn would then become a visitor center, have exhibits, accessible bathrooms, and program space. Okay. All right. That's you answer my question because I didn't know if it was going to be part of the museum or if it was going to be for administration or but you answer my question. Thank you. And the virtual tour for upstairs that is you can only view that if you're in the home like on the first floor or can that be viewed online? Yes, actually we've just received a grant from Mass Humanities to develop additional materials that that would already exist into a virtual tour that can be viewed on our website. We have kind of scattered sort of scattered information now but this project will really make that a coherent virtual experience for anyone. So I've been inside but I wasn't able to go to the second floor and I don't think you had. This was several years ago so my sister went upstairs and explained what the upstairs looked like to me. Yeah, I wish you had. Well, I don't know for sure when that visit was but we do at this point we have printed photographs and interpretation of the second floor in a notebook form and also in iPad form. So I'm not sure I don't remember if that was there then but this is maybe eight years ago. When you say interpretation in printed in iPad form what do you mean? Yeah, thank you. The term interpretation is the term we use in the historic house museum world to describe what information a tour guide would convey to visitors and that same information is converted into into text. So it would include information about Emily Dickinson's life, examples of her poetry, information about her family and relatives and the different influences and impacts on her as she became a working poet. So when you have it in iPad form are the people who work there familiar with how to or did they create the program such that it will read out loud or is it only visual? At this point it's only visual. Okay, I have a question about if if there were an elevator to get to the second floor well I guess the question before that is when you take people to the second floor when you will take people to the second floor will you be letting them go into any of the rooms or will you be letting them only stand in the hall and look through either a plexiglass door or you know stand behind a rope and look at what's in the rooms? They are able to go into the rooms fully. Okay, so do you have any sense of how wide those doorways are? Let me see if we have a have a I'm looking at the at the application here at the at the plans and let me see the door let's see two one the let's see one some doors are 30 inches wide another is 32 inches wide. Okay, so the 30 and 32 okay and making them wider would not be feasible it well I mean it would be different. It would it would be in conflict with our restoration objectives to okay you know to have the to have the house restored as it was in Dickinson's time. Well right she didn't know about air conditioning right I mean she it's not exactly the way it was in her time because she would have sweltered in the summer and nobody's going to and I know you're going to do it because it'll help preserve the materials and you know in the in the building I know you're doing it not because it was period air conditioned but because that's how to maintain you know without molding and all that other stuff so I get the climate control aspect but she didn't know about air conditioning right um well I mean yes our you know for us the objective is is preservation of the collection right no I understand that and and preservation of you know historic fabric is an important preservation no I understand I anybody have any any other questions I have I have a question I I kind of remember vaguely this was being brought up to DAC and we reviewed the plans and I remember also we talked about an entrance from the triangle street am I correct Sharon I believe that was not in relation to the homestead and that was in relation to the building the office building which is a separate building that's located on triangle street and and the Emily Dickinson Museum did provide a walkway from that building sort of the office building leading back to the homestead where there would be an an ADA parking spot and and I'm sure Jane could pull up this I plan to show you show that and then when we looked at the original plans a couple of years ago there was no elevator being planned right no correct and that was for the homestead at that time that was because volunteers or employees would not be going to the second floor was there sort of if memory serves me was there justification at the time so now plans are being changed I'm sorry that's not for the homestead that was for the office building sorry oh yeah so from a few years ago their variance request had nothing to do with the homestead building in how I had to do with the newly purchased building that's for their office administration sort of I see so when you said that the DAAC approved variances and you just want to extend them I'm confused about that in light of what Maureen just said in 2016 we requested variances these very same variances for the homestead in connection with another project in 2018 that involved the Triangle Street House so what we're what we're asking is is is if we could have the same variances that were granted in 2016 for the home when it was a much smaller project it or was it the same project it was a it was a different project it was a different restoration project at that point okay but now you've exceeded the 30 so actually what comes into play is very different no it was the it's the same circumstance in 2016 we exceeded the 30% oh you did anyway okay and now we are exceeding the 30% okay I'm a little confused because you're not planning on putting an elevator right because it would it would it would be too expensive well it would be too expensive but it would also interfere with the historical yeah exactly their plan of the yeah right there's not so uh the goal of fully restoring the homestead means that there's not a logical space for an elevator that could move between levels without without requiring a stack that would disrupt historic fabric and might even change the roof line right what about the outside one you wrote about an outside one right but that also would interfere with the historical right yeah the outside of the building that's right that's why it was that's why we considered it a a a poor option is that it would really really change the change the appearance dramatically yes workhouse I believe Marty had a question raise your hand was a really a question was more of a statement to put in an elevator in this house would just be it would destroy the layout of the house yeah and it's it would be very lot I mean you'd have to come back to us for a variance probably for a lula instead of a regular elevator because to put in a code compliant accessible elevator would take a huge footprint out of the house um and it will affect the roof line and trying to put it on the outside there's no place on the outside of the house because if you puncture from the outside of the house into the house you're going to be coming into a room not into the center hallway so it's it's just going to really screw up the the layout as far as I can tell just for your information I'm an architect and the other uh what was the other thing I was going to say about it oh we you didn't really talk about the front entrance but the front entrance as I understand it is not used except as an emergency exit um so it's not going to change the current pattern patterns so in this case I really I I would am for um accepting this I think this is the best feasible option and I played around with the program that you have for for showing what's going on upstairs and that was really quite interesting being able to really do a 360 you can look at the ceiling you can look at the floor um I think that would that's great and I know you're going to be putting in period fabrics and things you might want to think about in part of your display having a touch an area where people can actually touch the fabric so they won't be tempted to touch the fabrics upstairs yeah I also I also share your opinions too I I'm okay with no elevator because I don't want the the historical building damage in any way but the virtual view of upstairs I think that will be sufficient from my perspective so I will have no problem with with going along with this request and this story I I agree um I don't want the historical part of the house ruined for an elevator and I think the virtual tour is a wonderful idea yes because when I did when I visited before I I felt I was disappointed that I couldn't go upstairs but my sister was able to tell me what was there so but being able to actually see it on screen would be wonderful um somehow though could it be audible for people who cannot see it and um open captioned for if if there's any explanation of anything for people yeah yeah thank you yes Ruth whoops do you have a any any kind of comments I don't know if Ruth is muted she is muted um ask I'll ask to unmute yep Ruth you should be able to talk okay I didn't have a comment at this time oh you don't okay so you don't have you don't you don't have anything to say about whether you approve of it or not well yeah I pretty much approve of what they want to do okay Elise I approve of it as well I think it's great okay um so I'm perfectly happy to not um I'm perfectly happy to defer to the people who cannot get to the second floor um because they don't because they don't want to ruin the historic character of the building and I'm perfectly fine with not putting in the elevator because the people who wouldn't be able to get to the second floor um say that you've done something that seems sufficient to give them the information that they want I'm a totally blind person and I have to say my problem with almost all museums and it will continue to be a problem with this one is that first of all you almost never let blind people touch anything if you're going to let blind people get into some of the rooms and find out where the bed was and what the dresser felt like that would be nice I hope that you're not going to say oh no you may not touch it because everyone else can see it so if you're going to put maybe gloves out if blind people want to wear you know some gloves so they can touch the the the furniture to see what they what it is actually like that would be great and I um I mean I think it would be really important um because I've been to lots of museums where they say oh yes we have an accessible tour don't touch this don't touch that don't touch this we have a little booklet here and you can look at you know we have a you know a facsimile of the wallpaper see what the pattern feels like that's one of the patterns there are five in this place but that's one of them and um I can tell you that it's um it's no fun to go to these places so the fact that I can walk to the second floor is very nice but I hope that you'll be able to let people who walk in with a guide dog or a cane touch the things and I understand about body oils and stuff and all that so maybe you should have a box of gloves and say you may touch with gloves thank you Myra the other thing that I want to say um is what Tori said which is that that iPad thing when you go into some museums you can um purchase maybe for a dollar or two dollars this thing it's like a little tiny box and you can you know you press number seven and you know and and you listen through earphones and it tells you all the stuff about it and it's pretty descriptive they they are not created for blind people but they're really really thorough and it sounds like that's what you put into the iPad but what you put into the iPad so far has no audible access um so if you could get somebody to figure out how to get that either through a remote device or through the iPad itself to have audible access that would be a long way toward full accessibility um so those would be the two things that I would hope that you would do I have no problem with the front entrance at all you have you know you you don't even have a second class entrance you have a the regular entrance that everybody uses is accessible and I applaud for this and I think this is a great thing for the town and it's a great thing um that that you're all doing is any of this money coming from Amherst college or is it all privately raised it's private money really yes so they haven't put anything into it wow that's pretty impressive that's a lot of money and I would I figured that Amherst college was putting some of the bill but they're not no no it's a private gift yeah didn't you receive a donation from someone's life is that state that's correct it was a bequest that was um had to be put in well had to be put into an endowment and the distribution of income from that endowment fund is restricted to use for buildings grounds and collections so this this particular project is um perfect perfect for those restrictions that's pretty cool yeah all right so does someone want to make a motion about these variances but I just want to clarify so if the if the ipad uh tour for the upstairs is going to be audible then need some kind of caption and it's usually referred to as open captioning okay for people who are deaf or hearing yes thank you thank you for the clarification is the ipad thing the same thing that's available in the text book or is it different uh it it's the same thing it's the same thing but the um what is to be further developed is the virtual tour on the website so that that will actually be more complete it'll be a fuller more complete uh tour for the website that we can then um we can we can place on the ipad okay as long as that website tour is accessible and it talks I mean there are ways to make computers talk you might do it through recording that comes up when the picture changes I don't know how you want to do it but if there's text on the screen the text has to work through text to speech so with those changes I mean you're looking for variances um there's some well I I would say that the things that I that Tori and I brought up are not in your variance request um and I hope that that means that you will do them because that's part of access as well and I don't know how to write it in that you should also make the ipad um presentation and the website presentation uh accessible through auditory and caption can we put that in yeah so so the DAC is providing recommendations to the MAAB and I believe Jane is going through the AAB public hearing process next week so these recommendations will be provided to them and it will be up to the AAB to you know they'll they'll consider your recommendations as part of their approval process I think Elise has uh has raised oh please go yeah yeah um well one thing is I do want to um I think Marty brought up a really great point not Marty um oh god I'm going blank a brain fart anyway um I the yes the touch thing upstairs is with the gloves is very important to me as well but um when I know there are people who have hearing loss that don't do well for background noise and I was wondering where the ipad would be located um for looking at those and listening to those virtual tours would there be a lot of like coming and going and background sound that they have to deal with I don't know if this makes sense but I think it sounds good idea yes I understand uh your question um it's yeah it's available in the um first floor front hall uh there is um occasional movement between um um a hallway that leads uh from one side of the house into the parlors on the other side of the house so that goes through that large front hall and that's that's usually the only um um the only intrusive noise okay um yeah question uh might be would something like I don't know disposable headphones with that I mean is can you suggest a um solution I'm not hard of hearing I just um but I you know I'm just thinking of others that are and um and I do also have a thing with about background noise myself um I maybe headphones can can focus you more you know um I know that headphones could be expensive um but that could be an option I don't don't know what to suggest I just you know I'm just thinking of that as it might be an issue okay maybe you could have people um they could go up to the if you made it apparent that people could borrow or rent you know put a deposit down on a headphone and now that everybody's cleaning everything right um maybe you can just you know you can have three or four headphones and clean them um you know when people return them you know would disinfect them and people do that now so um but I think that's a really good idea so does someone want to make a motion that incorporates um the things that we've said and let me just comment I think that cleaning head headsets in between would be much better than having disposable um headsets or whatever you were referring to um yeah and that can be done easily with Clorox wipes yeah you want better quality sound you know if you're gonna have headsets you don't want crummy ones mm-hmm yeah but there's so many cleaning products out now I mean you can just alcohol them you can do whatever you know that's what I do mine yeah yeah so does someone want to make a motion I'll make a motion okay go for it second well you gotta make the motion that we approve this with the um with the recommendations that we have mentioned do I need to mention them all again I think Maureen has to know what to write down so so far I have um you know the the DAAC is fine with the variance requests as submitted um and you recommend that the applicant provide audible and open caption interpretation devices both for the for the website and on site and um and for the on site accommodations they should provide headphones be offered uh and um and have allow blind people or persons with low vision be able to touch fabrics on the first floor oh on any floor or I mean yeah wherever you're on any for you oh yeah yeah yeah yeah on um yeah throughout the uh in building how about that and um if I may um perhaps um you know on the website there should be um I don't know if the word is disclaimer or not but a um a a um a note saying those who need accommodations you know please call this number email us so you can um because some of this I've often seen especially in museums is that there are some museums that have state-of-the-art accessibility accommodations but no one knows about them including employees and that they just sit in closets for years and um visitors and employees don't know about them so to um educate the employees and to um you know where it makes sense to make it known on the website at the front door or whatever at the where people buy their ticket or what have you that to make it apparent and maybe even your voice message for the the homestead about providing accommodations okay thank you yeah that's good and I I really want to make it clear I I understand that a lot of touching can really destroy things over time so I'm very aware of that but I really I think there are ways for you to make it possible for people to touch things and not ruin them and gloves I don't know if they have those cheap cotton gloves or if they have those you know to use just like the ones that everybody's using now you know I don't I don't know if that would damage furniture I don't know but I would hope that you would have a way for people to touch things instead of saying oh no don't touch that don't touch that we have a little book here no I think the the gloves are easily acquired and uh are are you know inexpensive easy to use and I think that's a good that's a good uh suggestion a good accommodation okay all right so do you have something you can read Maureen or should we just trust that we're you're going to put it in a readable form with all the stuff that you very nicely took down yeah so I think yeah those items uh you know that I just said audible and open open caption interpretation be provided for both the website and for onsite um and that you know for the in addition for onsite there'd be headphones offered that uh for the interpretation and um those with visual impairments be allowed to touch fabrics in the building and be provided gloves and and to um and you know to provide a notification to the public uh about these accessibility accommodations well I would change fabrics to features but other than that that that's great thank you that's very thorough okay we have a second from lease right second okay all right were we gonna roll call this um well I'll call it a lease here yes seren yes Ruth yes Tori yes um and uh marty yes and I'll vote yes so that's cool thank you this is great you have um do you know when this is going to be completed do you have a timetable I forget it we're aiming to reopen um next spring so it'll take about a year okay yeah so you think you think maybe like by memorial day ish or oh I uh uh we're hoping for march in 2022 okay that's cool right so the museum will not be open at all during renovations that's correct yeah it's a pretty uh pretty extensive project and it's going to be a big mess for a while oh this is great has it been closed during the pandemic yes it has yeah we've been closed since last march 15th just like everything else absolutely that's right yeah well I want to thank you all for for hearing our our request and for this what's been a really helpful discussion for us uh in planning to move forward and we have uh so your suggestions are very helpful and we have um this next year or two to continue to work them into our into our tours and our experiences for visitors well you can always come back to us about the accessibility issues for the website and stuff because we could let you know thank you yeah thank you I mean that would be really good if you would come back next spring and say this is where we are this is what we've done and what do we need to continue to do does it work all right because sometimes sometimes um the people are told that they work and then the people who have to use them find out that they don't right yes okay thank you great so thank you so much thank you thank you thank you and that was all that was on the agenda for today so uh I I'll let Myra yeah so are you guys are uh readable to adjourn yes Maureen I have a question sure and uh I can never log in through my laptop to our zoom meetings I can only through I try to do it with my laptop for quite a while and I gave up and I can only do it on my iPhone so is there something that I am missing or um you know you might it might be helpful if you download the zoom app to your computer to your laptop um so you are able to um critically uh sometimes people have expressed that they have issues using zoom on their personal computer or laptop using the browser but I can provide you a link to download the zoom app and that might help your situation I'm not really sure what the specific situation is for you but want to try that and see what happens I would I really would but I think our meetings are the only ones I cannot do man you know I'm a member of Zanta Club we always do webinars and I have no problem whatsoever so it's interesting with this thing yeah I don't know that's interesting that's good to know um well let's try that let's uh I'll send you a link um to zoom and it'll let you download it to your laptop and we'll see if that works if that doesn't work maybe I could ask one of someone from our IT department to okay help you shoot you okay all right all right thank you so much do we need a motion to adjourn uh I don't think so but no okay and we're not we're just adjourned okay bye see you next time


Welcome everyone. So, in the previous lecture you may recall what we had seen various classes of policies. So, first class of policies that we defined were these deterministic policies and out of that the first one was a Markovian policy, a Markovian policy was one where at the decision rule at any time t mapped the state that was at that time t to the actions at that time t such that at any in any state you always took actions that were available at that state. So, these were Markovian policies then we looked at what was called history dependent policies in which the decision rule at any time t mapped the history that was known up until that time t to an action and the constraint again was that when you looked at the history up until time t that contained the state at time t and therefore you could only map this entire history to an action that was available in that state at time t in the state s t at time t. So, if you have forgotten the history is comprised of all the states and these the sequence of states and actions available that were taken up until time t minus 1 and the state that was you that was realized at time t this was the history at time t. So, the history dependent policy mapped this entire vector of states and actions to an action at time t this was a history dependent deterministic policy and the above was a history dependent Markov policy then we looked at what were called randomized policies in a randomized policy the decision rule was not deterministic, but it defined a probability distribution on the set of actions that were available. So, the randomized Markovian decision rule was one where the decision rule mapped the state at time at the decision rule at time t mapped the state at time t to a probability distribution on the set of actions on the set of actions and one always had to obviously pick actions from the set of actions that were available at in the state that you are in. So, the probability distribution of the that was induced by this decision rule was denoted by q sub d t now q sub d t is the probability distribution that is this particular mapping to evaluate the actual probability that you get of choosing a particular action you need to input the state s t here. So, it is q sub d t of s t. So, given the state at time t and given the Markovian decision rule you get a probability distribution here which is of on the actions. So, q of a sub d t of s t this was a Markovian Markovian randomized policy or Markovian randomized decision rule. So, this gave you a probability distribution on the set of actions. So, it had to be greater than equal to 0 and the sum over all the feasible actions that the actions available at that state had to be equal to 1. So, naturally all other actions that are not available at in that state are given probability 0. One also had a randomized history dependent decision rule a randomized history dependent decision rule is one where the you map at any time at any time t the history up until time t to a probability distribution on the set of actions available at in the state at time t. So, q of q of a sub d t of h t this told you the probability distribution probability of choosing an action a when you have chosen a randomized decision rule d t in and you and the history realize up until time t is h t. This then gave you a probability distribution on the set of actions and it had the earlier the constraints as before. Now, we also saw another set of policies which was which is called the set of stationary policies stationary policy a policy pi was said to be stationary if it is Markov and if the decision rule applied at every time t is actually independent of t. So, pi is said to be stationary if d 0 is equal to d d 1 is equal to d 2 dot dot dot till d n minus 1 then the policy is said to be a stationary policy in general one uses the word stationary to mean time independent. So, one talks of stationary problems stationary transition probabilities stationary costs all of these or stationary rewards by these by this one means really that the these particular terms the rewards of the transition probabilities or the overall characteristics of the problem are actually invariant with invariant with time. That means the they do not depend on the time instant at which these are evaluated. Then similarly when we also had a stationary randomized policy a stationary randomized policy is one in which we apply a randomized decision rule and when applies the same decision rule at each time. So, you had the same randomized decision rule at each time this does not mean that you take the same action at each time it just means that you take the actions with the same probability at each time for the same state. So, the once you fix a state and you fix and you tell me what the decision rule is and you tell then the probability distribution on the actions does not depend on does not actually depend on time. So, this quantity here q sub d t of s of a this is actually independent of time t this is this was called a stationary this was called a stationary randomized policy. Now, we finally took note that anything that any of these policies that are deterministic are special cases of their randomized counterparts. So, a Markov policy is a special case of Markov randomized policies. Therefore, the set of Markov policies is included in the set of Markov randomized policies the state set of stationary policies is included in the set of stationary randomized policies. The a set of history dependent policies is included in in the set of history-dependent randomized policies. Now today what we will do is we will get some clarity, more clarity about all of these different types of policies by looking at a specific example. Now in order to calculate, in order to work out this example we need to understand how is it that a randomized policy actually works. So what exactly when we choose an action at random, what exactly is the cost or the reward that we are going to get and what is going to be the transition that is going to occur with what probabilities are the states going to transition to the future states. Let us do this for simplicity for a first for a Markov randomized policy the same can be done also for other history-dependent policies as well. So let Dt of S, let Dt be a Markov randomized decision rule. Now let Q sub Dt of S of A equals the probability of choosing action A under decision rule Dt in state S. So what happens then to the reward that one gets. So when we are the way this works is that when we are in state S and we have chosen a randomized decision rule Dt we would end up choosing an action A with this probability, probability given by Q of A sub Dt of S with this probability an action is chosen. So then question becomes what is the reward that we would get because we are not choosing any particular action we are choosing actually an action with a certain every action with a certain probability. So what we do is we say that what we are looking for is the expected reward. This is again in the same spirit as expected utility theory anytime there is an uncertainty involved one takes the an expectation over the rewards or the utility or the cost that is involved over the uncertainty. So what one looks at is then the expected reward. So the expected reward is reward at time t in state S due to this policy Dt. Now Dt remember does not specify any specific action but we will just simply write this as the expected reward that you get when you apply policy Dt in state S. So this is called this is the expected reward from Dt in state S this is simply taken as the so this what happens is we have in a randomized Markov decision rule we take action with a certain probability. So what we say is that we are actually getting the deterministic reward that we would get from that particular action with a certain probability. So with the probability being Q Dt of S of A where now the summation A is over all the actions are so the expected reward when you up when you choose a Markov randomized decision rule Dt in state S is the expectation of RT of S, A where the expectation is taken over A and the probability distribution is the probability distribution specified by the Markov decision rule. What then we can ask a similar question about what then happens to the transition probability or the transition the probability of transitioning from one state to the other. So the state transition probability equal to once again we write this as probability of transitioning to state J in state S when we choose a decision rule when we choose a decision rule Dt under Dt it should be S under Dt in state S and what is this probability well this is again one takes the expectation of the of the state transition probabilities that we have for each state and action. So we choose take this we have already this state transition probability given to us the Pt of J given S, A we assume that A now is a random variable chosen with a probability distribution Q Dt of S Q Dt of S of A. So we just multiply this by its probability Q Dt of S of A and we take the summation of over A in capital A this then is the state transition probability. So under Markov randomized decision rule the expected reward and the transition probabilities are given by these expressions. This leads us to a fundamental question in all of Markov decision theory which is when does there exist a Markov deterministic policy that is optimal the set of all policies all you know by all history dependent policies history dependent randomized policies. So the taking the most general set of policies which is the history dependent randomized policies and one if one optimizes your once cost over by choosing allowing the action to be chosen in a randomized manner with over taking into account all the information in the history when do we still get a Markov policy that is optimal. This is a fundamental question we are not yet in the stage where we can answer this question we will answer this in a couple of lectures but they remember that this is a question that we are building towards. So what I want to do next is to actually show you an example in which all of these various types of policies are computed and are written out so that you get clarity on how exactly are these policies defined. So in the example that we will consider there is a system that exists in two states the system we have a system that can exist in two states or these states are denoted by S1 and S2 let me write this here S1 and S2 these are the states that it can that it can exist in. So note that these are not the states at time 1 and time 2 these are actually the two states of that the system can actually exist in. Now when the system is in state 1 when state is S1 when the system is in state S1 the decision maker has two choices for actions two choices for actions let us denote these actions as A11 and A12. So the first index here 1 denotes these denotes that we are actually in state S1 and the second index denotes the index of the action. So this is the first action in state 1 this is the second action in state 1 when in when state is S2 there is only one action and there is only one action and let us say that action is that action is called A21. Now if one if the if the decision maker if you choose if A11 is chosen in in state S1 if A11 is chosen in S1 then the then the reward the reward that you get is actually 5 units. So one gets a reward the decision maker gets a reward of 5 units and the state transitions to S2 with probability half with probability half and remains in S1 with probability half. So when one chooses action A11 in in state S1 you get a reward of 5 units and you transition to state S2 with probability half or you remain in state S1 with probability half. Now if A12 is chosen in S1 so if A12 is chosen in S1 then what happens? Well then he gets a reward of 10 units then he receives a reward of 10 units and the state transitions the state transitions to S2 with probability 1. So with certainty the state transitions to to state S2. Now if action A21 is chosen in S2 then then he receives a reward of of minus 1 units. So if A21 if A21 is chosen in state S2 then the decision maker receives a reward of minus 1 units in other words he incurs a cost of 1 unit and the state transitions to S2 with probability 1. Actually I should not really say transitions or what I really mean is that it remains in S2 with probability 1. So this action is chosen when you are already in state S2 so if you choose an action if you choose action A21 in state S2 you get a reward of minus 1 and the state remains in state S2 with probability 1. Now this is the way we have formulated this problem. I need to also tell you what is the what the time horizon of the problem is I should I need to also tell you what the what the terminal costs in the problem are all of these things are yet to be specified I will I will specify them in a moment but but I want you to notice before before we move forward to those details I want you to notice one important feature of this problem which is which is that if you look at these these rewards this reward here this reward here this reward here each of these rewards is actually independent of time. So the reward that I get when I am in state S1 and I choose an action A11 is equal to 5 regardless of what at regardless of the time at which this this action was taken and this and the time at which you were in this state. So regardless of the of the time instant or the decision epoch at which this is happening the reward is fixed the reward is always 5 units the same is the case here as well the reward here in of choosing choosing action A A12 in state S1 is still is still 10 units regardless of the time instant at which this choice was being made this and finally the same is true here as well the reward of choosing out of choosing action A21 in state S2 is independent of the time at which this this choice is being made. The same is also true about the state transitions the state transitions here use transition from S1 to S2 with probability half regardless of the time at which this this these choices are being made this transition is also independent of time and finally this one also is independent of time. When the when this is the case when the when the rewards and the transition probabilities are actually time independent it is possible for us to represent this problem in a much cleaner and much more compact form. So instead of writing out these all of these details you know verbally in such a such a long and tedious manner one can actually just do a quick pictorial representation that captures the all of these elements of the problem. So note that this works only when the when the rewards and transition probabilities are stationary in other words they are where so the problem is stationary when so the transition the the the the rewards are independent of time the transition probabilities are also independent of time. In this case we there is a cleaner representation possible this representation we will cover in the next in the next lecture.