All right, so I think this is approximately where we left off. Here's a comparison of our k-means results with complete linkage results. So this is an error slide. This is actually the single-linked results, as you can easily write it. And as we said, some aspects of the data after the classification into these regions are actually quite well-represented. Many of the data points that sort of look bi-modal here fall into the same cluster. But there's a bit of a problem that Rafael pointed out in his last slide. If you look at this cluster here, there seem to be very disparate data points. This bi-modal distribution and round-flat distribution that seem to all fall into the same cluster. Now, why aren't the flat ones over there and over here? And the problem here is that since k-means uses Euclidean distances to update its centroids and to calculate the distance from the centroids to the data points, the whole thing kind of depends on how the variance in your data points is represented. So you can imagine if the variance in one dimension is very large and very small in another dimension, the only information available for clustering really is in that one dimension with the large variance. And to treat all your dimensions equally, or at least not to inadvertently downvalue their importance, you should really be rescaling this to the variance or standardizing your data before you do k-means cluster. And once you do that, the clusters turn out rather differently. So all the data are now standardized on the variances in the different dimensions. And the clusters actually look a lot more similar in the homogenous in the more, for instance, in this cluster here, you really get clustering together very nicely of this bi-modal distribution and you really distinguish it well from the unimodal peak here. So standardizing the data before you do the clustering allows you to make sure that all dimensions contribute equally to your clustering problem. So kind of converting the expression levels to z-scores? Yeah. Yeah. OK. So these are useful techniques fast and easy to implement. You have to beware a little bit of memory requirements for hierarchical clustering. Why? What is the memory requirement dependent on for hierarchical clustering? What's the fundamental comparison that you use for hierarchical clustering? Remember? Distance matrix. So there's a memory requirement rising from the fact that you need a matrix of all pairwise distances. And it's a square matrix. So basically, for n data points, you need n squared sized distance matrix contracts. It's a bit ad hoc in defining the number of clusters and in defining exactly which distance metric to use, what constitutes a good clustering, and so on. And so it is very useful as an exploratory data analysis method. The results of cluster analysis are not necessarily the be all and end all of your analysis. Biological insight is still required. Now I'm going to go rather briefly through the more technical aspects of model-based clustering. Basically, as one example of more advanced, more insightful statistical methods that can be brought to the game. So model-based clustering is based on explicit probability models. So whereas in the other clustering methods, you don't actually go in with any notion about what your classes and clusters should look like. Model-based clustering is based on so-called normal mixture models. Basically, a normal mixture model is simply different normal distributions associated with each class in your cluster and then all added together. So if I take two normal distributions and I plot them on the same window, I have a normal mixture model because I've mixed these two normal distributions in one data. So now if you have these normal mixture models, as you can clearly see, you can apply sophisticated mathematics to represent the probability that the clusters can actually be well represented in such a normal mixture model and then analyze what the properties of each contribution, each component in these normal mixtures would be. And mathematically, this comes down to an eigenvalue decomposition where your components in the mixture are simply a sum over parameters that determine volume orientation and shape. And don't borrow slides from people who do that.  It's the bane of the audience. See, this is what happens there. Hard to turn off, too, when you borrow a slide from some words, bells and whistles that interact with the audience. Anyway, so the parameters can be set so that the normal mixtures should all have equal volume and the spherical, essentially, or they could have unequal volume and spherical. Or they could have equal volume, shape, and orientation. Or they could be completely unconstrained regarding volume, shape, orientation, and so on. So basically, as you're adding different larger numbers of parameters here, like with Orkham's razor, you're basically getting a better fit at the end at the cost of including more parameters to describe what you're doing. And there's a trade-off here, because with enough parameters, you will get the perfect fit. But it may just be dependent on your parameters and not on the actual data. With too few parameters, your analysis will be too coarse. But, and again, will not represent the data well. And you can solve that in the m-clust-r package. And we'll go through an example of that. The interesting thing, though, is that in this instance, you can actually compare the different models using the so-called Bayesian information criteria. And in the Bayesian paradigm of things, basically, what this does gives you a probability that a given model is correct after accounting for the data that you've seen and the prior probability that the model would have been correct in the first place. So this is totally weird. OK. Important aspect about the so-called Bayesian information criteria is that two different values contribute. One here is calculated as the measure of fit. How well does your model actually fit, predict, and represent the data? And from that, you subtract a penalty term, which is essentially the number of clusters that you need to represent your data in the first place. So the less clusters you need, the better. But the better each of the clusters fit, also the better. If your clusters aren't good, you're going to get poor measure of fit. But if you have too many clusters but they're all perfect, you're also going to get a low information score. So this is where this trade-off between cluster size or cluster parameters and the number of clusters and the quality of fit is handled and calculated in a mathematically rigorous way with sound statistical theory behind it. And thus, it can be used to choose the number of clusters and covariance here. So I'm going to skip over this here. It basically works in pretty much the same way. In the different degrees of freedom that you can apply to the normal mixtures here, and the one that induces spherical normal mixtures here, you would get this kind of clustering here, which is, if you just visually compare it back to the table, at least it's as obvious as the means of cluster. But it turns out that you can even get a better information score under a different parameter where you have a little more degrees of freedom, because then everything falls apart into just three clusters within a day. And now, looking at this visually, I think you note that three clusters now in each of these genes fall together very, very nicely. So it's a very, it seems to me, the best method of all that we've seen so far under the criteria of the best being the most visually obvious, similar within these fields of the ones that we've looked at so far. So once again, the bottom line is try different clustering methods. Model-based clustering is an attractive alternative. It's easy to use with that M-clust package, which is not installed by default. You simply have to type install packages M-clust, and that's on the course Viki. The best clustering method, thus, really depends on what you want to achieve. But one thing that I wanted to speak about a little is that clustering in such circumstances may not even be the best approach to begin with. And this brings us to this whole idea of density estimation. Now, clustering is a partition method. Clustering forces the data set to be partitioned into different sets. So if we simulate a data set, consider the following piece of our code. We set the C to a particular value. We create a vector, which is an arc array. We create a matrix, x1, which is an array of 70 uniform distributed values in the range between 0 and 10. That's what is 35 rows and two columns. And we create a second matrix, we call that x2, which is 30 normally distributed values between centered on just 5, with a standard deviation of 4.7. 15 rows and two columns. Now, we calculate the ranges, the x ranges for these values. Because what I'm doing here is I'm superimposing two different distributions in two dimensions. One is a uniform distribution, one is a normal distribution, and there might be outliers. I want everything on my plot, so I use the command range to make sure I know what the smallest and the largest value is. And I put that on the variable x, right? And then y range, the same thing for here for the first column and here for the second column. So I can make sure that my plot actually covers all of these data sets. And then I plot them independently in a plot. This is a normal scatter plot of the first ones, and I call them black. And then I tell this r new equals t. And then I do a second plot, and this r new equals t causes that new plot not to be updated, but it's overlaid over the first plot. So this is how I get the second plot overlaid on the first. And in order to do that in the same coordinate frame, of course, I have to specify my limits on the x and y axis. So I can't let r, what it normally does very well, automatically determine the ranges, because then they would not get essentially into the same. This is now colored red. I will suppress display of axes. The axes have already been displayed here. I will suppress the display of labels. The labels have already been generated here. I just add pure points as they are, and the result looks like that. So this is my normal distribution, and this is my uniform. This is the way my data came around. I just know, because I've been generating them in the synthetic data set from two different distributions, that I should be coloring one red and one black. But that's not a luxury I normally have when I look at data. It would be all black or all red, and I would need to figure out what's going on in that data. Is there anything of interest? Now it's obvious if I start clustering this into, say, two or three clusters, I would probably get the cluster partition along here or right along here, and so on. So there's certainly going to be many possible ways to cluster this, but it's probably equally obvious that I will not be able to consistently cluster data from noise or destroy my data as well, because there's overlap between the noise and the uniform distribution by data distribution. This is one. Synthetic example that illustrates a situation where it's probably not good to try to cluster, because the notion of partitioning is not what the data is made up of. The data is not purely able to be partitioned into two things. We have data that we'd like to recover and we'd like noise. Now density estimation is an approach that might be much more suitable in this way. So what's density estimation? Essentially, density estimation takes a discrete set of data points, and it tries to generate a probability distribution that's the most likely distribution where the discrete data points might have come from. So if I take a normal deviate, like R norm 100 in R, where I get a set of points that are distributed according to a Gaussian bell curve, and I plot a histogram. That histogram is going to be jagged and having corners that sort of look like a Gaussian curve, but that are not themselves the Gaussian curve. Density estimation is the problem. How do I reconstruct the smooth curve back if I only have observed single individual data points? The example data here is taken, I think, from Peter Dalgarde's book, Introductory Statistics with R. Raphael has mentioned that book. It's also on the course Viki. I find this a really, really excellent introduction into working with R. It explains a lot of the code that you use, even things that look trivial about the code at first. So if you're serious about this, I think this would be one of the first books to look at. At U of T, I don't know about the other institutions, but at U of T, we have it online in the library. So if you go to the U of T library and you look for the title of the book, you can connect to it online and just read it on your computer everywhere you are. You don't even need to buy it. OK. So this is data. It's a bimodal distribution with geyser eruptions in Yellowstone National Park. And the question is, there's a distribution behind that, a probability distribution, but we only have sampled these probability data. So how can we reconstruct that? The way this is done is to run a density estimation. And what the density estimation does is it takes a number of so-called kernel functions. So basically, just in the Gaussian case, bell curves. And it adds bell curves together until it can fit and reproduce the observed data well enough. Now there's a term in there which is called bandwidth. And that's the question of how broad these bell curves are going to be. If they're very narrow and very pointed, every one of these bell curves is going to sit exactly on top of an observation and they're not going to contribute to each other. If they're too broad, they are going to basically smear over the two peaks that we have in the histogram. So this is a little bit dependent on the resolution. I'll show you another example on that. On the so-called bandwidth of the bell curve, but R has bandwidth, SJ tells it to use an automatic parameter to optimally adjust the bell curve. So that's essentially all you need is to say, well, we want the density estimate for these eruptions with this bandwidth parameter. And we can put that into a plot command. Again, overlay it over the histogram in this case. So press the titles, press the labels, press the accents, and so on. Cold-read red and define line width equals 3 to make it very distinct and thick line. And that's what that looks like. So the density estimation here has now given us this primodal distribution probability density function that would give rise to these histograms that we actually observed in the data. Now, why is that useful for clustering? Is everybody approximately clear with what this density estimation does? So you have discrete data, usually never enough of it. And they're discrete anyway. And you're trying to establish what's the underlying distribution that could have generated these discrete samples that we're observing? With the setting of the two more brother or. Bandwidth. Here, this bandwidth parameter says use one of the automatic algorithms. You can also set it specifically. Actually, it's interesting to play around with it and show how the curves then get more rugged and sharper or broader and more similar. So if you could actually try several and use the best one? Yes, yes. There's an algorithm how to choose that optimally. So this is a little bit of an example from my own work. I'm interested in structure sets and proteins. We're interested to find recurring patterns in protein structures. And we're interested to take small patterns and then ask, do we have clusters, essentially, of similar patterns in the very large, multi-dimensional structure spaces of, say, all amino acid fragments that have five amino acids and that are folded in some way. So this is one representation of structure space that we've ever worked with structural biology. You know what a Chonron plot is. It essentially finds backbone and torsional angles along the backbone of a protein structure. And if you plot these aggregates here, you see they are dense regions here, here, and here. This corresponds to the Nalapelix. This corresponds to the E-thrand. And they are also dense pool regions here. And the problem that we're setting ourselves is to find out how many or where are there interesting clusters in all structure space that might tell us something about how proteins fold and why proteins fold. And the way we've been approaching this is similar to kernel density estimation. It's a problem that's not very amenable to pure clustering procedures because they're all defined. And there's a lot in that that's just noise and that influence our clusters. So what we'd really like to get at is just the best representations of each of these, what we call structural motifs. So this is the way we go about it. So we just have one dimensional structure space and discrete observations. So this is one particular piece of protein structure. This is another little fragment from protein structure and so on. Small discrete observations. And then we use a Gaussian kernel to estimate the density in that structure space. So basically what we do is we associate a Gaussian kernel with each and every single one of these observations here. And then we sum over all the Gaussian functions and we get this density estimate as a potential probability density function that will rise to all observations. And now in order to find the representatives of these clusters, of these structural motifs, we don't try to cut them apart and pry them apart into different subsets. But we look for the local maxima in that density space. Yes? Did you present a significant deviation in some again? The deviation of each particular deviation. Right now, all of these are just standard observations in these axes. But the standard deviation of the Gaussian kernels is basically the same thing as the bandwidth. And that's, I think, next slide after that. I get to that in a moment. So there's no good and reasonable way to choose the standard deviation except in some way. Well, no, I get to that when I have the slide for that. That's exaggeration. What's the best choice of this parameter, the standard deviation? If I use very broad curves, I basically smooth over all the differences in my cluster space. There's something immediate value that's quite reasonable. But if I use very narrow curves, then I can get sub-minimum that I might not even want to divide it. So you can think of this like resolution, looking at something at a very coarse-grained resolution or a very fine-grained resolution. And depending on what resolution you look at, you will get different results. But that's OK, because that's the way our data is structured. If we look at it in a very fine-grained way, we should want to see sub-structures and things falling apart again into small details. But if we look at the whole thing at once, we might only be interested in the very large pictures. And if these are maximally broad, the entire space will just have a single peak that you can find at no other local maximum. So basically, tuning the bandwidth of these kernel functions for the density estimation gives you a way to look at your data in a more smooth, or in a more fine-brained way. We then go along and we just look for maximum, local maximum. Of course, if we would be looking for a global maximum, we would just find a single point. But if we were looking to well-medate local maximum in some neighborhood of the point, we can then pick out points that are represented again. This is not a clustering constraint. But we're looking for local maximum after order of kernel density estimation. And then we can say, well, these local maximum have a certain neighborhood of all things that are within some distance within the cluster standard. And once we have the neighborhood, we can superimpose the fragments. This is one of the examples. So this is ranked 399 in our little database of what you've found. It's a piece that has seven amino acids in length. This is the backbone superposition here. And we were only using the backbone for calculating the coordinate distances. But what turns out is, even though this was not a search criteria, there are very significantly non-random sequence propensities. This position here is always either phenylalanine or kerosene. This position here has a high likelihood of being a spartan acid or a botanic acid. This position here is almost always serine and so on. So why did nature choose these particular sidechains and these particular sequences? Well, it's because these specific folds and shapes with that particular sequence give low energy confirmations. Low energy confirmations have high probability. Since they have high probability, they get peaks. When evolution has explored its possibilities in structure space, and we can reproduce these peaks with the kernel density estimation, find such things where sequence and structure come up to something like a representation of the protein folding code. So this is not result of a clustering method, but result of a very different approach looking for local maximum after density estimation. And some of the problems that we look at are much more suited to that. OK. So let's go back to the digital example. What does kernel density estimation in this data set? So I said we can cluster it. The code is there. It's easy for you to try to see whether you can come up with meaningful clusters here. But if we run the kernel density estimation, combine the two x and y, the two black points and the red points together into one joint file because the density estimation has to run over all of them. We run the density estimation, but we only run it on the x coordinates. It has to be density estimation. Here it's one dimensional, so we just project everything down on the x coordinate. We would get a similar result for the y coordinate, so we could do other things to do it in higher two or three dimensional spaces. And plot this as a fact, loop curve over the same plot, and then it looks like this. So this now shows us that we're interested in this, that there is something interesting going on in our data very high in that data distribution. And it would be relatively to the coordinates of that peak here, and just rank everything according to distance in these coordinates and start analyzing what it is. It won't completely separate signals from noise right from that, because that's just not in the data. But it will give us a good starting point and start analyzing them and grouping them. Yeah, very much scratching the surface regarding cluster and algorithms. There are many, many others. If there are many others, it probably means none of them are good for all situations. So it's a useful tool. You can shoot yourself in the foot with it and come up with the impression that there's something in your data that really isn't there. Once again, use it with moderation. Use it for exploratory data analysis. If you have to derive quantitative results, make sure that you use different approaches to clustering and see how robust your clusterings are and see if you can get independent validation. That's always the most useful thing, for instance. If you cluster according to expression profiles and then you can independently see you have an enrichment of genontology categories. That's a good validation that your clusters are actually meaningful and not just artifacts of your data. OK.


People generally would revolt at the idea of being an instrument. It's like, am I a robot?  And so it's great to be a robot of the truth. So the thinking will slowly peter out. It will take some time, but after a while in most people, it sort of settles. And I'm not saying it's an easy path. It's not. It's a sadhana. It's a spiritual practice for the very tough and those who are, you know, ready to hold through.


Hi, everyone. Thanks so much for joining us today. My name is Justine Calma. I am a science reporter at The Verge. We cover science and technology and my reporting looks specifically at climate change and the environment. I also have a focus on environmental justice and equity. So I'm really excited for our discussion today on equitable recovery for public spaces and city planning. We have some fantastic panelists. And so we can start with some introductions so that you all can get to know each of our panelists a little bit and the great work that they're doing. So I'll start off by introducing Catherine Lusk, who is the executive director of the Initiative on Cities at Boston University, where she spearheads university-wide programs and urban research. And I will hand it over to Catherine to tell us a little bit more about herself and her work. Hi, thank you so much Justine. It's a pleasure to be with all of you today. As Justine mentioned, I serve as the co-director and founding executive director of the Boston University Initiative on Cities. We are a university-wide research center at BU. So I have the pleasure and privilege of working with faculty in the social, natural and computational sciences as well as in public health. And we seek to catalyze research in on and with cities in pursuit of sustainable, just and inclusive urban transformation. We also create experiential learning opportunities for students and connect them to internships and course-based work in cities. And I'll end by saying we were actually co-founded now many years ago by the late mayor of Austin, Tom Menino. So one of the major projects that we spearhead is the Menino Survey of Mayors. And we have the pleasure of interviewing over 100 mayors across the country every summer. I'll talk about some of those findings today, even though we're in the field right now and about halfway through our interviews, but we've learned a lot about how mayors approach equity and equity in the public realm and a lot of the challenges that they're facing as we not emerge from the pandemic, but as we try to look down the road beyond it. Thank you so much. Thank you, Catherine. And then next we have Christian Bannister who is a digital experience design director at Gensler. Christian has been conceptualizing, designing and developing award-winning interactive experiences for more than 20 years and I will throw it over to Christian. Great, thanks. Yeah, so I think you started, still a little my thunder there, but I'll pick it up. Yeah, so I work with Gensler and I work with the DXD team. And really what we do is we work at the intersection of people, place and technology. So it's really about integrating technology and really kind of finding opportunities to integrate content and data into the built environment. We're currently, I should mention, we're currently matching a grant with the Knight Foundation for a project called Urban Periscope where really the goal is to increase awareness of data for civic engagement. Really it's to sort of like, kind of really onboard citizens to the potential of data by allowing them to get their hands dirty in a playful augmented reality experience that we're sort of integrating into the urban experience. And that's been really interesting and exciting for us to kind of push new technology and these new opportunities to connect with people and really get them excited about data and what's possible. So this is really what I'm bringing today. Hopefully we can talk a little more. Thank you, Christian. We will talk about it a lot more, I'm sure during our discussion. And then we have Melissa Gaston, who is the co-founder and executive director of the North End Community Coalition in Charlotte, North Carolina. Melissa is a community organizer and also a realtor with her own firm. And I'm really excited to introduce Melissa Gaston. Hello, everyone, glad to be here. Thank you very much, Justine. I am the co-founder and the executive director for the North End Community Coalition. And here in the North End, we're just about a mile and a half outside of the city of Charlotte, but we are trying to bridge the digital divide and connect our residents with the resources that are out here and available just to make their other lives a little easier, definitely on the ground working with the community, connecting them to all kinds of resources and people and just being in the trenches, as I like to say. Thank you. Thank you. I love that you're in the trenches. Hopefully you can bring that view into our conversation and we can kind of join you there as we dive into discussing equitable recovery. The first things first, I wanted to ask how you all have seen the pandemic exacerbate existing inequities or create new ones and how can we then address those inequities? So I would be glad to go first and answer that question for us because of our location, where we are and the demographics and the economic status of a majority of our residents, that the pandemic had a dramatic effect on our residents. Things that some of us took for granted like internet access, just being able to have a safe place to be and stay away from the COVID and things like that. It just, things we took for granted. So as an example, residents who didn't have access for internet, they couldn't do a telemed visit. They couldn't help their children connect to school. So if your children are already struggling with school for whatever reason and then you don't have access to get to the technology that we just really showed how much that had an impact on it. Things you don't think about and it created even greater isolation for our residents as well. If you have technology, you can, you know, zoom or FaceTime with somebody but if you don't have that access that also creates additional isolation.  So it really had a demonstrative impact on us. Thank you so much for sharing Melissa. It really hits home for Gensler. I mean, we do a lot of design for workplace and it really has been sort of an inequitable sort of kind of experience for people in different demographics sort of adjusting to this hybrid reality of the new workplace. They were sort of promoting and for certain clients, like we have a lot of clients that are tech companies. We work with Facebook and Twitter and it's sort of a different situation for some of those people who have a lot of space, a lot of income, their connectivity is good. They have all the technology set up but then there's like other people who are living in crowded homes that just really don't have the space to be able to sort of realize that dream of the hybrid workplace. And that's been really interesting for us because we design the spaces but we haven't always like been designing for people's homes. So it's like an extension of an architecture that we've always been considering for years which is fascinating and sort of a new, I guess a company for our team. Yeah. I think I would say there was going into and then emerging from the pandemic, right? So there were inequities in place and elements of the sort of infrastructure of cities that was seriously eroded at the beginning of this sort of time period we're living in. So in all the years of surveying mayors we've sort of found that they talk about kind of the three legs of the stool of what makes a vibrant city, right? And so you want sort of healthy physical infrastructure transit, housing, bridges, schools. You want healthy and thriving fiscal infrastructure meaning is this, the city's finances healthy? Do they have a good bond rating? Can they get access to capital? Is the economy diverse? Can businesses, new businesses sort of open and operate and thrive? And then they talk about social infrastructure, right? So they talk about our people, the people that make a place sort of as wonderful as it can be. Are people actually thriving? Are they healthy? Can they live up to their full potential? Is there some degree of social cohesion? And if you look at all three of those some were broken before, infrastructure, physical infrastructure was broken before, but I think social infrastructure and sort of fiscal infrastructure in cities has really eroded or corroded both because of the pandemic and because of systemic racial injustice that has been just part and parcel of the urban experience and the sort of American experience for decades. And so, you know, when we talk about when we talk about cities, you sort of look at all three and because you both made such great points I want to like tease with a close by sort of teasing of the little point that's come out of the Menino survey so far with our interviews with Mayors this summer. We're asking them what are the long-term consequences of the pandemic that most would bring them? And we've only done about half of our interviews but two things are rising to the top. One is learning loss among kids, right? That was inequitable going into the pandemic and it's gotten worse for communities of color and lower income communities. And the second is mental health and trauma. It was inequitable going into the pandemic and it's worse as this endures and there's racial injustice endures. And the overlap with the points that, you know that you guys have been talking about are both relate to broadband and the digital divide because telehealth is one way you can serve and support communities in addressing mental health. And it's another way that's really critical to closing educational gaps. And then the other one is, you know because we have researchers that do work on educational divide and do work on telehealth and mental health and the physical environment people are in adversely impacts sort of outcomes in both of those areas. So can you get access to it? And can you be in a space that's private? That's quiet, you know that's actually health promoting and that's conducive to learning while we're all having to endure sort of being at home and having to endure the sort of exclusion from the normal settings where we would work and we would get access to healthcare and we would learn. Thanks so much. I'm curious to what if any opportunity, you know I quote unquote new normal might bring, you know from here what do we do to address these all these inequities? It's a tough question I know everyone's working on it. There's a lot to unpack there. Yeah, I mean I think it but it's for, I mean I'm gonna go back to the project we're working on tonight because I wanna talk about that. And I mean, so we're kind of going back to civic engagement and we wanna get people to really lean in on shaping their world. And I think that's something that, you know that we've been, you know we feel really strongly about it. We understand there is a digital divide and there's a lot of inequity that's just kind of systemic and part of our society but at the same time we think there's like value in getting people to really sort of hold their leadership accountable really add some transparency to data the way that funds are allocated to really get people thinking about that but not in just a way where they're sort of consuming graphs and diagrams and insights that have been sort of distributed in more of an editorial way or a part of like, and here's my insight it supports my research this is something I wanna tell you you wanna get people slicing the dice in the data including, you know developing their own insights and validating it on their own and then really amplifying their own ideas and I know this requires social media but we think that that is like a way to sort of kind of facilitate these conversations that are more open so I guess in a way it's really optimistic because we believe that the social media or the people being connected to the internet can ultimately benefit everyone but obviously there's been a lot of issues with when we look at what's happened in last couple of election cycles and just the way that things the way that sort of social media has been misfiring and misinformation, fake news a lot of things that are happening right now so those are negative outcomes but I guess we're still working towards some positive ideas I guess we're still very optimistic about connectivity, the internet and empowering people with usage of data so that's something that we're doing right now I mean it's, you know and I don't know that's a few thoughts there so I'll jump in right here for us and our community and our coalition we've been kind of going back to the basics I agree, you know social media and having the data is very important but we just need to get to the basics so people can at least connect to the internet so they can have the to be able to share their information or either to be able to share their data if you don't have that connection and you're not going to people's houses or you're not having these big community meetings where people are able to engage you're really isolating people and they're not able to provide the data that's needed and the data that they have is very important to change what's happened and to improve what's happened in the past and keep developing and going on so one of the things that, you know really struck me during the whole pandemic thing so some kids were able to connect to the internet because they had a device the school gave them a device, whatever but they weren't necessarily familiar with it because it wasn't like they had computers in their home all the time or they just used them at school but the person who was at their house trying to help them may have been an elderly person you know, a grandparent or an older parent who did not know how to use that computer or better yet, their parents were the healthcare workers or you know, the people that had to go to work they didn't have a choice to stay home so their kids are home by themselves trying to connect to the internet they can't connect, they're having difficulties or they're doing what kids do I don't want to do school work but and so that's a whole missing thing so there's definitely that aspect as well so I just don't know how we can come together and collect the data and do those things so we can improve upon it without connecting to the people we have to figure out a way to connect to the people and really be able to institute change that's gonna be effective for them That's a great point there are some, you know the numbers are improving as far as like connectivity and people who are sort of have access to smartphones and mobile devices over the last 10 or 20 years you see like huge really positive trends and movement in that direction and I think an interesting dimension of this is the way that people use connectivity some people that will not even they'll be more consumers than producer you know what I mean? So it's more about like people will be connected but they'll be streaming video and things like that and maybe not even know that they're connected in a way where they can actually be leaning in as opposed to leaning back because I think it's something we think about a lot is like, what is the posture of that sort of digital experience? I mean, are you leaning back and watching something and you know, playing a video game maybe it's even a little closer to leaning in but this idea that, you know there is the potential for some people to just shift their posture and become more involved and realize totally that there are still inequities but I think that's an interesting dimension that's one of the point out of the problem I mean, I'll stay on the topic while we're on it I'll stay on the topic of community engagement you know, I think change change certainly changes the local level mayors will say that it's the most personal job in politics and I think constituents will say, you know it requires a certain degree of relentlessness and vocal advocacy and activism to really affect change, meaningful change change with dollars, changes sort of allocating resources and one elected official told me you know, it's not enough to be on the list of priorities for us you wanna be one of the top three items and that for the advocacy and activism comes in it's like how do you actually get an elected official to say that thing is one of my top three and I will get it done and I will get it done now and you know, I look at community engagement and I think about the inequities again, that we had going into the pandemic and that has endured during the pandemic and community engagement so my colleagues at BU, Katie Einstein, Max Palmer and David Glick have studied this issue at length looking at inequity in the zoning process and looking at inequity in how new housing gets constructed in cities, right and what do they find? They find that the people that show up the zoning meetings entity older white male homeowners often with a legal background and they're protecting their asset and so they're precluding new housing and particularly new affordable housing from getting built and what they've actually been looking at is has the shift to remote engagement sort of lessened inequities during the pandemic I won't speak for them but the initial lessons are no it hasn't reduced inequity that the shift to digital engagement still in the current processes we have still create more space for the most vocal and the most privileged to turn up and speak in opposition to things that are really serving the greater good for the community and so I think there are things about the process of community engagement that continue unfortunately to be broken. For sure and you all have touched on this already but are there ways to use the power of data to highlight where we need more targeted investments? Certainly, I think that it starts with transparency but it also it is really about that engagement because the data is there it's just that people aren't using it and I think there's no real kind of gateway or real invitation into sort of using it in a meaningful way and I think also put in a face on it so to me I find it really interesting that someone has these assets that are available for them to craft insights and amplify new ideas across social media it's there and it can happen but I think you just need to see someone doing that and get inspired I mean if it's a little bit more punk and DIY and then we can put a face on it and we can get inspired by that person who has a message and can use data to support it and I also think that the idea of being able to validate the data because people are questioning everything nowadays so it's like drilling down into individual data points and just saying like yes that data is real it's not just a graph on a newspaper article editorial this is like something I can see I can see the voices and the people and I could sort of because there is a fabric of experience that supports this data so I mean kind of connecting those things I think is a good way to get at some of that. I think I'll point to two examples from cities where I think data has been deployed in a way that's really driving meaningful change one is here in Boston so Boston had something called it has something called the streetcaster program and streetcaster is actually an effort by the city to remedy inequities in the built environment specifically in relation to maybe the most underappreciated element of infrastructure which is the sidewalk. We don't think about sidewalks we're fortunate to have sidewalks in the Northeast not every city in the country has sidewalks certainly not every city has sidewalks that are actually created and maintained by the city itself but we do here in Boston. So with streetcaster the city went out and they manually inspected every single block of sidewalk and they created a score to understand where sidewalk quality was inferior. But that created a massive list like millions and millions of dollars farmer money than they could ever put into fixing all those sidewalks in a given year with a given capital budget. And so they overlaid that with socio-economic demographics of the neighborhoods and they overlaid it with transit access to try to understand where their lower rates of car ownership meaning more people are walking more people are walking to transit more people are taking the bus and they use that to pinpoint and allocate those dollars and prioritize investing in neighborhoods with the worst quality sidewalks that really use them that are high users. At the same time they tried to actually fix the 3-1-1 system because what was happening was wealthier neighborhoods this is data wealthier neighborhoods were calling to have the same darn pothole fixed over and over and over again. And they were getting there with a squeaky wheel and so they were actually getting all of the attention and all the dollars of those sort of patch remedies. And so they were the city said no we actually have to fix the system it can't be response time and we're going back to the same neighborhoods again we actually have to be reallocating dollars to permanent upgrades in neighborhoods that feel less investment and be more responsive to the neighborhoods that are even less likely to call to fix the problem. So that's one and the second one I'll say quickly is the city of Milwaukee also the city of Minneapolis actually created parks master plans that took into account both the quality of parks and the demographics of the neighborhoods surrounding those parks. And so they were again scarce dollars scarce resources how do you allocate them they actually were prioritizing both places that had sort of inferior quality that were due for upgrades and neighborhoods where they have for the higher number of users lower income users and sort of a density of users so that they were allocating those dollars in a way that they were having the best then most effective impact and remedying historic inequities. Those are two things that I think tie the data to the dollars. Yeah, those are amazing. Those are two very good points I see both of those happening in the city of Charlotte the sidewalks and not having the sidewalks well we don't have sidewalks all over Charlotte so some places do most older places don't have them and the people who use the sidewalks are not the ones who have the best sidewalks but the people who call and complain are the people who live in the south end of town or what they call the wedge and those are the individuals who make all the noise so they get all the improvements and they get all the, because they know how to and it's just, it's amazing. And then the other point that you made again about Milwaukee and I mean it's true here and it's just, it's amazing just like zoning here with zoning if you don't, you see the yellow sign but by the time you see the yellow sign go up or they put a sign announcing it is probably too late where other places the developers are meeting with the residents to find out what they want and what type of changes and the effectuating those changes based on what the citizens are saying but here they're not doing that and so you're right they get on and complain or they send a letter to city council or to whoever the zoning and they send all those requests electronically through the internet using that social media to do that and then the other people aren't here when they have meetings they're in the middle of the day when they're at work because they have to be at work so the inequities are still there, the data I just, I'm at a loss really I know that's what you do Christian for the data and if we can just translate that and somehow and bring it down to the people who are less who are more impacted by the lack of data would be a great we have to figure out how to do that and how to make it correlate. That all resonates with me I feel like my work is a lot of turning data into stories and you know highlighting how that actually affects people's lives and when it comes to things like sidewalks and green space that's something I've reported on too how important sidewalks are for mobility is hopefully for folks with disabilities and in the wintertime when there's inclement weather which we're seeing a lot more of recently and places like parks being important refuges when we have really bad heat waves like we've seen this summer although those places aren't always accessible and equitable and I think on top of all of those issues the pandemic has also made us rethink how we use public spaces and how we're allocating all of the public realm so I'd love to talk about how how can we ensure more equity and access to things like parks and green spaces and sidewalks even. Well, there's an aspect of this that I think that we're really interested when we talk about digital experience we want to reduce friction if we want to make this inclusive I mean those are drivers that we always go after and it is really part and parcel of the way we design things we go from a human-centered design approach so you want to understand the need and how to connect with people in the most effective way and that's really in one of the things that we're getting at with some of our new design thinking is like really embedding these things into the environment itself so when we want to talk about sidewalks can we look at a sidewalk and get an insight related to that site so the data isn't so much of an abstraction it's not the planner view it's the view from inside that I can start getting these insights and I really think that distributing them it maybe doesn't even require connectivity but we talk about ways of integrating these data dashboards and views and insights into the environment itself I think there's a lot of options there and we're just trying to be creative about that and just really kind of opening it up transparency because I think it is a lot of these things are transparency we've seen some alarming trends and in San Francisco gentrification is unbelievable I mean the median house price is completely unaffordable it's broken, housing is broken in San Francisco you know what I mean and it's so interesting that there's data that just is a very simple story a couple layers of data and you can see these trends that are alarming and anyone could see them and also anyone could see the way that zoning has been sort of disproportionately it has an agenda that doesn't support all people and that's something that's also a very clear trend that goes back for 50 more years that you see the way that zoning's been and not everyone knows that story not everyone has that at their fingertips well we all have it at our fingertips we just don't know how to access it and we don't know that we need to or maybe we should be looking at it so those are some things that we've been thinking I know there's a lot there but I'm excited about the topic Yeah I mean I think I because I was talking about sidewalks and parks I'll move on to trees I use that as an example because it's one area where there's been a lot of media coverage on the relationship between things like tree canopy and historic redlining right and then we know scientifically we know that there's a relationship between things like tree canopy and air quality and air quality is one of those issues that we know there's tremendous inequity it's becoming a more prominent issue as I was telling you guys before this all started we were hit with the smoke from the forest fires even here in Boston even though we're miles and miles away from them and trees can't solve that problem but they are an important part of improving air quality in cities and so when you look at a map and you look at neighborhoods historically neighborhoods of color and lower income neighborhoods you see are less tree canopy but mapping that is really expensive and I think one of the things that has been exciting is seeing new resources that are coming out that are really national and so I'm pointing to one and I just looked it up to share with everybody the American forests if you haven't looked at it has a new tool out that actually maps tree canopy for cities all across the country and you can drill down into your own neighborhood if you actually wanna see and then it looks at not just where are the trees but whatever the actual benefits or the benefits are missing out on if you don't have that tree canopy and I think things like that where there's a third party they make that investment but it's a national resource that also has local relevance for residents who wanna advocate for improvements to things like tree canopy then there's actually these tools and resources at their disposal that they don't have to be this sort of data whizzes or work with the academic experts they can go to that I think the second thing is even the trust for public land has new measures of park they've always had measures of park access now they have measures of park equity to try to understand park acreage and park quality and investment by different neighborhoods and within different cities and so again, those are the kinds of things someone's done the work for you but if you're a local resident working at a local level in an individual city you can tap into resources like that and that helps you make the case for why your city or your neighborhood needs greater investment and greater resources. Yeah, definitely. You pointed out a couple of different things the tree canopy, if you notice in areas where there's predominant people of color there's generally no tree canopy or very little and yet if you go into a neighborhood that has a tree canopy if you're transitioning from one to the other you can noticeably feel how much cooler it is where there's somewhere that has a tree canopy and somewhere that doesn't just like parks and things like that so in Charlotte they have this we wanna make sure we have a park I think within that two miles or a mile of everyone's home there's a park that you can go to and so what they call a park for some people maybe a park with a couple of benches on it and maybe a basketball hoop, a court and that'll be it for the park and then you go to another park in another part of town and they have all kinds of amenities in that park and it's just so amazing those are the kind of things that you can say well, we have a park here and they have a park within a mile of their house that they can go to but the inequities in the park they just did a study here where they went around to all the parks and the parks that were in the worst conditions were all in areas of people of color but then they put tons of work into other parks but the people who are complaining about the parks are again, the people who are in the wealthier areas because they utilize the parks more but do you think they utilize the parks more because they're nicer parks? Do you think other people wouldn't wanna utilize the park close to their area but it's just not something that's appealing to go to so it's just all those kind of things that are out there that you just don't realize unless you know in a lot of areas where people of color and especially because of redlining the type of zoning for the houses or the areas like and there's a neighborhood where we are here and basically the whole neighborhood is considered like I-1 land which is industrial land but it's residential so there's houses built on I-1 land so it used to be industrial or they built industrial next to houses because they didn't care and nobody complained and said anything so the data is there but how do you get that data out to say well this is here because one of the things that the city don't get me wrong city I love you city of Charlotte well one of the things is they don't wanna change some of those industrial zones because they're thinking oh then the businesses are gonna have to leave so we wanna keep these still as some industrial zone although there's residents right next to it so how do you make those adjustments? How do you take the data and make that say well no we need to change this and make it actually happen? Yeah we actually just published a video on the urban heat island effect and how and why some neighborhoods are significantly hotter than others it's on the Verge Science YouTube channel but we went out with thermal cameras in East Harlem and the Upper East Side to see a pretty dramatic difference in temperatures from neighborhood to neighborhood so there's definitely a lot to dig into there and so much that can be done around changing physical infrastructure to prevent heat related illness and prevent heat related death and make it so that you don't have one neighborhood that is suffering much worse temperatures and health effects as a result. With that said I did wanna talk about whether we are spending enough time looking through different lenses knowing that the same strategies don't necessarily work for all communities how do we do that better and be more inclusive of community voices that differ from community to community? I think this touches on a lot of the civic engagement that we've talked on before but yeah I mean the strategies are there's no one size fits all. Yeah I mean I think one example like that comes to mind for me I think everyone is trying to solve this right now I don't know of a lot of great evidence at least or even really great examples from cities I think one sort of older example that comes to mind for me was a number of years ago the city of Toronto, the head planner for the city of Toronto recognized inequity in engagement around zoning decisions and she said okay we're gonna completely change how we engage citizen engagement like completely change how we change we're gonna completely change citizen engagement in relation to zoning decisions. And they did essentially a lottery and broke down the demographics of the city the age, different racial and ethnic groups different religious groups in Toronto and just by lottery sent invitations to different residents saying would you like to be part of a community advisory group around zoning decisions in the city of Toronto? And people responded and they applied but it was a random process right it was trying to get people who would never otherwise be engaged in the city and so they applied and then they went through essentially like a citizen boot camp to figure out what is zoning, how are decisions what are the things that we have to worry about what agency does what? And then through that process they actually came to they became the sort of expert advisory council that was guiding the city rather than just the resident groups with the most vocal people with the most time to kind of show up and harangue elected officials. So that's one example and I guess the second example would be participatory budgeting. Which is not a panacea but a lot of cities across the country have been using New York, just in New York city counselors in New York often use it to say here's a pot of money and let me actually turn this pot of money back over to my constituents and they come up with the ideas they scope those ideas and they vote on those ideas and those are the investments. And in Boston we have about a million dollars in capital budget. It's a program called youth lead the change and it's youth driven participatory budgeting. The young people come up with the ideas scope the ideas, come up with the ballot and vote on the initiatives and it's a million dollars of the city's capital budget that they vote on every year. That's amazing. We're doing a lot to engage kids because we think it starts by like reframing attitudes towards these things and like understanding what's possible and starting with a simple story and kind of deepening that understanding. But I think just the hooks from the very beginning we're interested in there's a museum exhibit that we're designing which is sustainable cities which is really gives kids the opportunity to adopt a section of city and it's a projection map, it's high tech city it's mechanized projected map but the kids can make decisions. Like one of the decisions is the building mix. So it's a very simple idea of zoning. So what is the mix? How do you combine your building mix? And as you do it, you watch your city transform and change and you see the impacts of some of those decisions because everything's a trade-off in a way and you can also adjust your green space mix. So if you turn up your green space some buildings drop and turn into parks. I mean, it's all fantasy but the idea is that kids realize that, yeah, I can shape my city, you know what I mean? This is something that's totally possible and all these decisions are being made by somebody and in the future that could be me just even just that realization that somebody's making these decisions that all of this is possible, you can sort of have some agency in your own community and maybe get involved in these ways. It's just, it's really one-on-one stuff. I mean, but I think it's also valuable just to kind of get that reframe some of this stuff. So it's not just like a world that I've inherited but a world that I can impact. I think there's something there. That's great. I also, I really wanna hear Melissa's take on this as well but wanna quickly open it up to any questions that folks wanna submit. We'll continue the conversation with, you know, any questions you want answered from our panelists, you can drop them in the chat. But Melissa, did you wanna add on? I have a lot to say. I don't know. That's great. Yeah. Let me give my thoughts together. Yeah, let me give my thoughts together. I just think that what's, I think that's important. What Christian said was very important. I love the ideas. I love the ideas that everybody has. I just don't know how do you turn those ideas? So as an example, we had a project, the smart cities engagement, where we worked with the city and our community and we said, okay, here's some money. We have these, we call them kick starts, not projects. We call them kick starts. And they said, okay, here's the kick starts. You determine from the community what you need and what you want in the community. And I think there were four overall that were suggested by the community, maybe five, but there were four suggested by the community. But then there was one that wasn't suggested by the community, but they just said, okay, we're gonna do this one too. We're gonna throw it in there. And that was the one that was least successful. So to me, what that says is, if you're inviting the community and you get the community behind and they're the ones giving the ideas and how the impacts and what they're looking for that you'll be successful in that. So somehow we have to turn this in to say, yes, if you have the community behind it and the community is able to provide their input. And I love that idea where they were and people were invited to come and join this committee and they did and then they became the advisory group to the rest of the community or to the other groups. That's so important because people are thinking, oh, this is important for me. I had an impact in this and people wanna feel valued. So they wanna feel that and because they wanna feel value, then they're willing to be engaged in it and they're willing to share what they learned. So it's not just certain groups of people that are able to impact change that everybody can impact change because you have the opportunity because you were asked and you were included in the process. So I think those are all things that the cities that we all are in and the areas that we are working we all just need to go back and I said it again, I'm trenches on the ground. So I'm not at the high level with all the other stuff. I'm like right there in the ground where it's happening talking to the people, talking to the residents, being in their face, but not in a negative way but being in their face and talking to them just to get their ideas and get it out and to be able to share that. And that's just so important because I think no matter what happens to be really effective in any type of change you're gonna have to engage with the community. Don't just come to a group and say, hey, this is what we think you want. No, what do you want? Tell us what you want and let us determine how we can do that for you. You know, like how do you want that done? What does this look like to you? And really ask and don't just ask and not do it but ask and help them give them the resources to do what they need to do to get done what needs to be done for their communities. I love that. It's such a great melding of data and being in the trenches that we've gotten to have today. Thank you so much for all of your insights. And I'm excited to see all of this work move forward before closing out. I did wanna introduce Lillian Corral, the Director of National Strategy and Technology Innovation at Knight Foundation with some closing thoughts for us. Grace, thanks, Justine. And thank you to the panel. I think a lot of the comments share really bring back the point around using data to really understand inequity. And I've heard Catherine really talk about this. You know, I think we can take for granted that it's really clear what inequity looks like in our community. And at the same time, if we really wanna try and solve a lot of these inequities, we really do need more of this data at our fingertips. So there's a lot more that seems like we need to be able to do. And it sounds like the engagement piece is still a nut to crack. And so thank you to Melissa and the team in Charlotte and then Christian at Gensler for really pushing on the bounds of how we make data more accessible. How do we really engage the public in this discussion? To close out, I just wanted to give some last minute instructions. If you're still with us, thank you so much for finishing up day one of the Smart Cities Lab 2021. As a team member here at Knight Foundation, we're so grateful to be able to be in conversation with you all. If you are interested in learning more about the grantees that are on the Hoppin platform and via Hunt, we now invite you to visit the Expo Hall. You can find the icon on the left side of your browser or if you're on your mobile device, you'll see it at the bottom. And if you click on the Expo Hall, you'll be able to actually go into various sessions, see grantees. They are virtual booths, if you will, that are being hosted by individuals who are running programs and initiatives across the Knight Smart Cities portfolio. And when you click on the booths, you'll see a video or Google slide presentations and you'll be able to interact with them and ask questions, hopefully exchange information and perhaps pick up some ideas for your community along the way. Please join us tomorrow for day two, just a sneak preview of what's happening. In the morning, we'll be kicked off by Kelly Jin, who is our Vice President of Community and National Initiatives, who's gonna kick off a day that's really a bit more forward-looking. We're gonna tackle the issue of mobility and equity and we're also gonna talk a little bit more about autonomy. We're gonna delve deeper into one of Knight's initiatives around autonomous vehicles and autonomous devices and really look at how the public can get engaged with such a sort of forward-looking technology, but yet that has a lot of regulatory implications today. And so how do we actually do the hard work as Melissa just alluded to, of working on the ground in the trenches and actually letting community have a voice in all of that. And then we'll also, for grantees, have a session from Panthea Lee around strategy and impact and how do we keep our eye on impact when there is so much variability and so much ambiguity in the world around us. So please join us tomorrow and right now I invite you to go through our expo hall and meet others. Thanks so much for joining us today.


It has been a standard operating procedure for a very long time in the NFL that after a touchdown has scored the ball will be placed at the two yard line and the offense has the choice whether to kick an extra point which is worth one point or to line up as if they were on offense and try to move the ball into the end zone again which would be worth two points. Kickers have gotten so good at kicking extra points though that in 2014 just eight out of 1,251 were missed. Things will change in 2015 though. So let's look at what the difference will be and we'll try to figure out what this will really mean for teams. So after a touchdown teams will now have the same two choices either attempt the extra point or go for two. The difference is that if they want to attempt the extra point the ball will be placed at the 15 yard line instead of the two. If they decide to go for two they can tell the ref that and the ball will be placed still at the two yard line. Moving the ball back to the 15 yard line that's where they'll snap it from will result in about a 33 yard kick and you generally add about 17 or 18 yards to figure out how long a field goal is, 10 yards for the end zone and then 7 or 8 for the snap back to the holder. The other change in this rule is that the defensive team can block the extra point or if the team's going for two they can get the ball on a turnover and they are allowed to run it back to the other end zone and doing this would be worth two points for the defense. Traditionally if the extra point was blocked neither team would have gotten any points. So will this rule change really change much at all? Well kickers are still pretty good from 33 yards they hit 32 of 33 attempted kicks from 33 yards in 2014 that's 97% rather than 99% so don't expect too much of a drop off. Then again with so many extra points being kicked just a 2% drop off would mean an extra 30 missed extra points over a whole season. It's probably not too much to worry about though because that's only about one point per team for the whole season. So will this mean that teams should take advantage of keeping the ball on the two yard line and go for two now? Teams over the past five years have rarely gone for two only about twice per year for each team on average. This is somewhat surprising considering the success rate is right around 50% technically it's 48% but this means that in the long run you would score just as many points converting two point conversions half the time as you would kicking an extra point every time. Obviously we can't see into the future but if that rate of 48% holds true it means that whether you go for two or kick the extra point you'll likely end up scoring about the same number of points in the long run and if that is the case then teams will probably just keep kicking the extra point. What could happen though is that teams will start to go for two more often. This could result in a shift of how often two point attempts are successful. If that success rate goes up maybe offenses will practice it more or teams will attempt them more against weaker goal line defenses it would mean that a team would be better off going for two after every touchdown. However the opposite could also happen if the two point conversions are being converted too far under that 50% mark it would probably be best just to take the extra point. I don't expect to see much change in 2015 as far as the number of points that are scored because whether your team is lining up for two or just kicking extra point even if it is from 33 yards it probably won't make too much difference at all in the long run.


Today, we have Bundy from Venekia. Your news release this week was outstanding, Bundy. Thank you. It was talking about providing real substantial data that the crypto market is not only here, but it continues to grow. Can you talk to us about this? Sure. So our news release, by the way, thanks for having me. Always a pleasure to be here. The crypto markets are often seen to be something that's unregulated and harder to access. And the exchange-traded products is a vehicle to provide institutional investors access to crypto assets. Effectively, there are listed securities like exchange-traded funds. Exchange-traded products are basically exchange-traded funds or exchange-traded notes, and they have underlying crypto assets. So for financial institutions looking to get exposure to either crypto assets or the yields generated from crypto assets, ETPs are a great instrument. And what we saw last year in 2022 was despite the market downturn, despite some pretty big Lehman-like moments in the crypto industry, the number of ETPs increased. They doubled. And in the last month, this January 2023, we saw the AUM's increase and the AUM's increased at a greater proportion than the overall crypto markets. So crypto markets, as you know, again came above $1 trillion in market cap. That was about a 33% rise. The AUM sets the assets and the management for ETPs actually rose 39%. So that reflected about an 18% premium on the actual crypto assets. So what does that mean? Demand for institutional quality products that can be purchased by traditional finance, financial institutions is on the increase. And that was a big takeaway. And that was a bit of a surprise. All right. So let's talk to our audience. We have a lot of members in our audience that are very conservative. They like asset-backed investments. They are watching this sector with interest, but they do prefer having a bar of gold in their back room. But they don't want to discount this particular sector. Can you talk to us a little bit about how those investors can move into your space and feel a higher level of confidence as they're making investment decisions to round out their portfolio? This is a long-winded question because as you pointed out, institutions are continuing to make this investment in this space. Do I have it correct? Absolutely. So gold is a good proxy for a lot of people. But just like in the early days of the internet, people saw the internet as being a replacement for film or television or radio or magazines. It was all of the above and none of the above. So it created a brand-new paradigm, a brand-new paradigm for communication and ultimately for transactions. Similarly, Bitcoin is not a replacement for gold. It is similar to gold in so far as it is an asset and that's fungible. But it's also like a currency that you can use for buying and selling. But it's also like a stock because it gives you a yield. In this case, it's in the form of staking fees and yet it's got a stored value element to it which is down to the basics of gold. So it reflects multiple financial assets and that's the beauty and charm and perhaps also some of the confusion around that. Is it a stored value product? Is it a unit of measure? Is it a transaction mechanism? Well, it's all of the above. But the one thing that I can say is there's no other asset class by far that has given the returns that Bitcoin and other cryptocurrencies have given in the past 15 years of its history. And that for someone looking at where the next 15 years would be is a bit of a rear view mirror look and seeing what the future might possibly hold for you. And earlier in an investor talk, which you hosted, you were discussing how the international market is a lot more is a lot more aggressive about appreciating what's happening with the crypto assets at this time. And you were referencing an event you went to in Dubai. Would you like to provide our audience kind of an update about not only the audience that was there, but the interest? The approach towards cryptocurrencies is fairly binary. So you have some governments taking very harsh views on it and other ones taking a very liberal view to it. The UAE has become a place where innovation is encouraged in the digital asset economy. And therefore it's becoming a magnet for large industry participants to converge. One such event took place last week. It's called the Satoshi Roundtable. It was a ninth year of its existence. And as the name suggests, it was very crypto oriented. People who were there included the heads of Binance, Tim Draper was a noted Silicon Valley and cryptocurrency investor, among several others. Coinbase, you named the top companies out that they were there. And aside from the large investors, he also had a number of young startups. And that was a real surprise for me. Pleasant surprise, of course. So you saw the fidelities of this world who were showing how they're making crypto asset purchases more possible for US retail investors. At the same time, you saw you had a whole bunch of young companies showing the utility of new products and new protocols coming out. Now, the framework for participating in the digital economy is being created in certain countries. Some countries in Europe, the UAE, Dubai in particular, is also very positive towards that. Japan has some good frameworks for that. And that contrast with some of the trends we're seeing in China and the US, which is much more restrictive. It's on the other end of that binary one. And I think that's to their detriment because countries that are encouraging it are seeing capital inflows, they're seeing startups come through, and they're actually seeing some cool new products that no doubt will be seeing. I'll give you one anecdote here. I was involved with advising Skype in the very early days. And one reason why Skype could not start in the United States and Silicon Valley is because the two founders had set up a company called Kaza. Kaza was a file sharing site, and the Record Industry Association of America had basically taken them to court for a line for illegal music downloads. The consequence of that was that Skype could not start in Silicon Valley because the minute the founders touched US soil, there were subpoenas against them. So almost by kind of default, they set themselves up outside the US, and London became the center for where they started. And that kicked off a huge ecosystem of startups got there, transfer-wise, monetized. There's a whole bunch of companies that started out of London, which became the FinTech hub for many years because the US took a more restrictive approach, and other countries benefited. I'm afraid that something similar might be happening, afraid for the US, not otherwise, because other countries are taking a more liberal approach. In Germany, for example, 20% of a fund manager's allocation can be two words. Securities that basically have crypto assets denominated against them, they are aligned for that exposure. And I think that's healthy and encouraging. In so far as it's done in a transparent and regulated way, many people get afraid of shy of regulation. We actually welcome it because that's going to take care of some of the maverick behavior or even the potentially illegal behavior you've been seeing there. So countries like Dubai or rather the United Emirates and some parts of Europe, some parts of the Far East are far more progressive in their approach towards which I think is going to show long-term benefits for them. Bundy, it's so nice to have a digital asset expert to join us today and provide a global overview. Thank you, Bundy. Thanks for having me. Always a pleasure.


Good morning everyone. My name is Danielle Scheffler and I am presenting today on the difference between automated and manual accessibility testing. Before I go ahead and get started wanted to do a very quick intro. I am based out of the Washington DC area in the US and I'm a deputy program manager at Booz Allen Hamilton. We're a large global consulting firm with expertise in digital data science and cyber security, and we focus on business government and military organizational transformation. I am also an accessibility consultant at Salsa Digital. I've been in the Drupal community for well over 10 years in the government space for over 15 and have expertise in UX and accessibility. I'm excited to be here with you today. Unfortunately, I know it's virtual but very much looking forward to connecting with everyone throughout the day. So I want to go ahead and dive right in. The first thing I wanted to do before we even really get into the meat of the presentation is talk through accessibility just making sure that we're on the same page of what it is and why it's important. And then talk about the definitions of what we actually mean when we're saying manual accessibility testing or automated accessibility testing. Talk about some of the differences and then just do a very, very quick overview to see how many standards are covered by each of these testing types and that includes both WCAG 2.0 and WCAG 2.1. If you're not familiar with WCAG we will quickly cover that as well as we go through. So first of our accessibility basics, I wanted to share a definition that I really love for accessibility. And I apologize because I'm not usually somebody that reads the exact text that's on the screen but really wanted to dissect this a little bit so I'm just going to go ahead and do it. The degree to which a product, device, service or environment is available to as many people as possible on as many devices, visual browser, screen reader, mobile device as possible. Now notice that we don't have anything here specifically about disabilities. We're not calling out auditory, we're not calling out visual, motor, dexterity, cognitive. It's just focusing on the people and making sure that their experience, no matter if they have a disability or no disability at all, that their experience is going to be equivalent. We can't say the same, but it will be equivalent no matter who they are. And so I really love that this talks about the inclusivity of making your, you know, in this case we'll say website or your platform accessible. All of that being said, we know that there's still a very big focus on disabilities when it comes to accessibility. I did want to go ahead and give you some statistics and show you that one billion people live with a disability. That is 15% of the world's population. You know, it's the end of 2020, and I still hear people say, well, you know, I don't think there's anybody with a visual disability or anybody that has a hearing issue who comes to my website. The first thing is, how do you really know, right? There's really no way to measure that. But the other thing is that someone who's living with a disability is just like the rest of us, right? They just want to, you know, find information. It's an e-commerce site, right? Like they still want to buy something just like the rest of us. And so we really need to keep that perspective in mind that it isn't about the disability. It's about the person. So as we think about why accessibility is important, right? Let's just say that we're going to continue on this path right of thinking from the disability angle. We need to think about the fact that disabilities or maybe even we should say diseases are growing globally. And if we take Australia out of the equation or just US out of the equation and we think globally, we do have an aging population. And while none of us like to think about it, with age come health conditions. Not always, right? But a lot of times. And so you have things like diabetes and diabetes can cause glaucoma. And so you have, you know, issues with eyesight, for example. If you think about someone that maybe has Parkinson's or has MS and that might affect their movements, right? And how they're using technology. Think about the had an accident, you know, from car accident or something with sports injuries, et cetera, that can't use their hands because they broke it. And that's their dominant hands. They're used to using a mouse, they're used to using a trackpad. And all of a sudden they have to use their non-dominant hand to get around a website. They need to use their keyboard. We need to make sure that the sites and the platforms that we're creating are keyboard accessible, right? And of course, accessible in general. And so it really plays into this, you know, second and third bullet that if we're making our sites, our platforms accessible, it's not just for the disability community, right? It's really thinking about that inclusivity. We go a little bit further into some of the regulations, right? I promise that I would say, or give the definition of WCAG or the acronym, it's Web Content Accessibility Guidelines. And they are put together by a group or even committee, I should say, as part of the WCAG. I'm sure a lot of you have probably heard 2.0, 2.1, right? I mentioned that earlier on. 2.0 is, it's been around for quite a while. And that is the government standard pretty much internationally. And 2.1 builds on top of that. So it is backwards compatible. So if you're looking at 2.1, you're actually looking at all of the guidelines for 2.0 plus some. You're looking a little bit more on responsive and a little bit more on some of those motor dexterity and cognitive disabilities. And in addition to that, there's actually a draft out for 2.2. So it's showing that the committee that works on these guidelines is really focused on always thinking about the new technologies that are out there. And what people need to be able to access, making sure, again, that we're giving everyone the same type of information, you know, even if it is in a different format. So they are getting that same sort of experience, I should say that everyone gets. So in addition to those guidelines, they come in what I always say three sizes, there's A, AA and AAA. I'm not going to do a deep dive into that A for the interest of time, but B if you do a very quick Google search, you'll be able to get a lot of information on this. The one thing that I did want to point out we talked about how 2.0 is the international standard 2.0 level AA is going to be the standard for regulations across the world just in case if you're curious as to what the government guidelines are. So with all of that, I know that was very brief, but I wanted to get into really the meat of our presentation, which is talking about what automated and manual accessibility testing even are. So if we look at automated testing, it's checking that a website's components and theme are usable for all of the disabilities writer types that I had mentioned before, by using automated accessibility tools. It doesn't mean that you don't have to do anything. It just means that a lot of the work is kind of cut out for you already. Notice that I don't list content in here doesn't mean that you can't check your content but obviously it is a very very big lift if you are trying to do testing on every single piece of content that you have so highly recommend that you follow the guidelines to make sure that you're testing your content to be accessible right before it's even tested, but if you wanted to you could put content in here as part of the definition. So if we go over to manual testing, right notice that it's the same piece right we're checking your website's components and your theme, but in this case, we're not doing automated testing right we're not just using a tool. We're turning off sound to check, you know, audio descriptions to make sure you know there are captions right we're checking all these different things. We're making sure that everything is keyboard accessible. And we're using assistive technology so that could be a screen reader. It could be something that speech to text like dragon. It's really making sure that you're running a lot more testing on your own and not just relying on something else to help you. So let's go ahead and talk about some of the differences between these two. So for automated testing. I like to say it's fairly easy. I mean you can click a button and get all the results I mean all you need to do is put a URL and noting I wanted to actually step back for a second and say that these tools you're not just going to be putting your website name, and then everything's going to be done for every single page right because there are different templates because the content is different etc. You are running page by page so wanted to make sure that was clear. So when we say it doesn't take a lot of time. Of course it depends on how many pages you're looking at but at the same time, there isn't a whole lot of okay let me go through each one of say the 38 standards for WCag 2.0 AA. You're just, you know, clicking the button, most of the time and then the test is run for you and the results come up in a matter of seconds. You need to have a lot of prior knowledge right because the errors that might come up or the violations are going to give you some of the information that you have right it says okay this is a violation, and here are some ideas of how to fix it. So, it's good that you can do this without really having a lot of that accessibility knowledge already there. Software is generally free, not all of it but most of these tools do have you know free plugins or you know you can go to a site and just put in your URL and we will talk about some of this tooling. The audit is either free or inexpensive so of course this depends on if you are a developer running this against your own code. If you're a client that is asking for an audit for automated testing, you know it's not going to be as expensive. And then the other piece though this is right one of the cons because I know I've been painting probably this rosy picture of automated testing is that it only covers about 20% of all issues. And that's actually being generous. It's also inconsistent, so I wanted to really touch on that, because of the fact that I was just doing an audit about a month or so ago maybe on this also site that just launched they just relaunch their site. And I use the testing tool that I love. I've been around 12 to 14 errors on one page and I was just curious what would happen if I used another testing tool is doing an experiment. One of them said that there were no violations. And the other one said that there were about eight. So we get 08 and 14. If I didn't know anything about accessibility. I would honestly find that very confusing and wouldn't really know where to start so there's that piece. The other piece while it is fantastic that these different tools give you examples sometimes of code that you can use. Remember that that's very generic it's not something that is, you know, looking at your build. And so that might not always give you all the information that you need as well. So if we go over to manual testing, manual testing is very, very thorough. I had mentioned not too long ago that there are 38 WCAG standards. This is looking at every single one of those. It can be time intensive, which right can be a little frustrating sometimes but you know as we talked about we want this inclusive environment and so we're going to be able to do this. We're looking at all those standards or if we're looking at 2.1 and we're looking across, you know, all 50 of those. And so it is thorough that being said you do need someone that has expertise so you can't just, you know, run a tool or just say somebody that's like I've heard of accessibility I kind of know what it is. You need an accessibility consultant analyst expert, you know whatever title you want to give this person, but they do need to have expertise. So the software may not always be free that comes around screen readers and we will talk about that in a minute. The audit may be more expensive right because it's not just, you know, a tool that you're running and that's kind of giving you all the answers. But again it covers most of the issues is your accessibility analyst or your consultant going to uncover all 100% of the issues. No, because there might be things related to content you know there might just be something that gets missed, but you are going to be able to find right a lot more than 25%. The other thing is that when it comes to remediation, you're also going to get something that's a lot more tailored to your own build, as opposed to something that's more generic and so it will help.  So with all of that, if you tell me Daniel I can only do automated right now that's all we have the budget for you know I don't know what do I do. I'm going to tell you just because you might only find 25% of the issues to not do it like no no no no no. Please please please do your automated testing. You know that's that's always been my goal is to get as much accessibility testing as you can, but I really want to make sure that people are aware of the difference. So somebody's not running an automated test and saying, Okay, I'm accessible right you need to do your manual testing in order for that to be true. So to wrap up really quickly for tools, automated testing I use Andy it is a little bit more manual. It's an open source tool that came out of the Social Security Administration in the US definitely recommend trying it out. You've probably also heard of wave acts which is through DQ site improve and then Pali, and then screen readers are voiceover for Apple. It's also on iPhone as well, not just on computer jaws which is a paid screen reader for PC. A lot of people love it though which is why they don't always use narrator, which is the windows screen reader. So the top one is for automated testing, and then the bottom one is for your manual. And then just really quickly for standards covered. I'm not going to go through every standard I really just wanted to do an example. So for WCAG 2.0 for automated testing. It looks like a lot of things are covered. To go to manual testing. These are all of the additional standards that are covered on top of this so you can really see why that 25% number comes out so we have, you know, this list, and then a much bigger list of what is covered for manual testing. Same thing for WCAG 2.1. Remembering that also covers all the standards for 2.0, but you have two standards covered here for automated testing. And then you have all of these covered for 2.1. Keeping in mind that when I'm looking at all of these, this just covers A and AA and is not looking at AAA. So thank you so much everyone for attending. If you have any questions would love to open it up. Feel free to email me as well. And looking forward to talking with you later so thank you. Hi everyone. If you have any questions, please feel free to put them in the live Q&A. We do have about two or two and a half minutes left for me to be able to answer a few questions. There were a couple that was one for any AI based automated testing tools for better coverage. Not that I am aware of. I know that you might have some tools around like Site Improve and Padmapper that will do a little bit more for you. But again, that's all around automation. Pali you can use with BLT for your Drupal builds and again that provides some coverage. That can be very messy honestly though when you're trying to present to clients just because everything is spit out generally in what looks like a terminal. And so that's why I recommend as much of a slog as it can be sometimes just using some more of the front end tools like Wave or Axe or Andy and then coupling that with manual testing. Cool. And the next one is what's your favorite automated testing tool from Grant? Definitely Andy. So that's ANDI. They also have a little bit of a tutorial as well. More than happy to talk about that offline as well. I know we only have a little bit of time but happy to answer more questions on that later. And then I do actually see one in the discussion forum about how important is reading level becoming. It sits as a WCAG AAA requirement. It is somewhat important but AAA is very, very difficult to follow. So if you can focus on that great but it isn't something that's required and certainly not something I want to say that you should spend a lot of time concentrating on after. Even WCAG themselves says that they realize that not all content can meet AAA just because the standards are so complex. Another question from Jeff Free on, do you know of any automated testing tools that can be integrated with CI, CD processes such as Jenkins? As far as I know the only integration is Pali. Acquia actually has a pretty good write up. It was something that I had been working on at some point with them on Pali and BLT but as far as I know that's the only thing that's available at this point. DQ which is DEQUE may have a tool but those are the only ones at this point that I'm aware of. And then last one, any experience with or thoughts on Dynomapper from Ben? Yeah, so Dynomapper is a good one. Some of it is hit or miss I would say in terms of the results that I've seen. Certainly you don't have anything negative to say but it is like a lot of the other testing tools in terms of just not always giving you the full results. And then I believe there's a UX component as well which is great except it's built off of usability.gov which is actually something related to the agency that I work with now in the U.S. And that's actually being decommissioned so you might actually see a few things unfortunately in Dynomapper that aren't working. Okay. Are there any other questions? Did you answer the one about reading level in the discussion? Yes, I did. I got to that one. All right. Thank you everyone so much. Please feel free to shoot me an email or find me in Drupal Slack and really appreciate your time today. So thank you. Thanks, Danielle.