Now I will give the floor to Bunny and I hope that you'll give him a warm welcome of applause before he starts his talk Thank you Thank you. So Mike on great. I thought they're a little optimistic thinking this would fill salt one for a talk like this But thanks everyone here for coming in preparing for this talk it was It was interesting I realized as I was interviewing people that I lack a certain authority to speak to For example gender issues in computer science And I run the risk of alienating the people I hope to empower if I get the message wrong And I also realized it's a touchy subject for a lot of people and the risk raising the ire of a bunch of trolls, but I think social policy should not be dictated by Fear of trolls. So let's get started So let's start talking about how much society is included in technology. This is a study that was done on 200,000 people very recently and The interesting part you want to look at is the purple and above bars are the people who kind of know how to use You know spreadsheets and word processors and stuff and the people, you know, the orange and the red can barely do a search query Like they could Google something and say, you know, what is something and that's about all that can get out And so it's although this was done over a fairly wide a drink 16 to 65 It's you know, it's easy to forget in a place like CCC that there's actually a large portion of the world that's not included in technology And why is this inclusivity important? It becomes important when you talk about putting up to a vote So if you want to explain situations like surveillance or net neutrality or even like the selection of emojis to people and You want to have a vote on it.  Sometimes the works for me attitude isn't good enough particularly when it comes to a vote there's a couple of very prominent examples of where Think where we had surprising outcomes and Brexit and the American elections because of You know things And so a more specific example is like open source I am an open source activists and I think that open source and the leeway movement is about empowering everyone But if you give people source code and you don't tell them how to use it Are you really empowering them? So there's this this example that happened earlier this year X screensaver popped up a time bomb on my novena Saying like this version is very old Please upgrade so I start googling like what the heck is going on like is this like what's going on? And then there's this great thread where the maintainer JwZ You know sort of get some popcorn and read it It's it's interesting and he says like when they even bothered tell me a version of running I say that version is three years old and then I say but this is the latest version of my distro ships He says I then your distro sucks and they say but I don't know how to compile code from source herp derp I paste and everyone goes away unhappy. So in a situation like this It's like both the developer and the users are disempowered actually by the situation It doesn't necessarily help to just say like I just put the code out there And I don't necessarily tell people how to use it and expect them to know how to use it And so openness is powerless without inclusiveness sharing source code empowers a 1% or even less than 1% of society And if you think that's good enough We need the government's help to enforce the licenses that we rely upon and we need society's help to sponsor and patronize our efforts And if you think that open source is a moral or social imperative We should try to empower more people to preserve and sustain the practice, but it is not practical to teach people one by one How to code it's not JWZ's responsibility to have told people how to compile a source one at a time What you want to do I mean as an engineer you want to optimize your efforts You want to look for obvious issues high leverage events that can get you a lot of gain And so for example if you look around on the left-hand side as a picture from Oskar open source conference You know you'll be challenged to find the two women in that photo. I think actually CCC is quite good on inclusivity So the right the gender ratio is quite better here, but When you are looking to increase participation, it feels like this could be an You know an area of good gains It's like someone coming to me and saying hey my battery life sucks And you find they're using a 7805 regulator to you know to you know go off their power supply or I'm out of memory and your memory allocators only allocating on the odd pages or even pages like you've just thrown away half your resources Even before you got started and oftentimes when you fix bugs like this the paybacks can be more than 2x You know just fixing that one problem often will help you find other problems and get even more people included into technology How bad is the bias I mean here's some recent numbers in 2015 Undergraduate degrees in the United States only 15.7 percent were word to women. That's 2,000 women compared to 12,000 men In that lower graph there is a comparison of Computer science versus other professions like mathematics and physics computer science has half the number of women in it Then there are in mathematics or physics even right so it's particularly bad in this in this field And it's not just a US problem like this is here's a here's a graph showing the breakdown In Europe and you can see the numbers in Germany for example is 10.5 percent women getting degrees in 2000 computer science And then you know there's some people who ask like you know maybe there's actually genetic advantage You know in computing or something like this, which is a ridiculous statement to that I find the only argument is just to respond With no right it's grumpy cat. No, I actually had a series of slides I was trying to reason through these people I was like you just can't reason with people who make statements that Inaccurate the fact of the matter is that just women can excel in computers and hardware I work with many women who are very gifted in this in this field some of them you recognize like LeMore Friede on the left who does I ate a fruit. There's I abadir who does little bits Nadia peak who's very influential in the fab app community There's gchi on the right hand side who I'll talk about later on who I'm working with on this project Naomi Wu who's been working on the a maker in Shenzhen who's on the confluence of design and technology and fashion and There's another lady here Non-way Gong who's also doing fat technology and fashion and I almost sure you've seen products She's done announced on a big stage, but every single time it's awesome guy taking credit for her work and Then there's star Simpson who is also talented engineer and I really enjoy her recent circuit classics project And so clearly woman can be involved maybe the biases cultural In my experience China has a much better balance Much better gender balance in technology fields when I work with the factories there I see lots of women in the in the offices and I asked for a picture actually of a contract matter for Facture I work with quite often called a qs I said can you send me a staff photo recently and I just went through and counted and it's 62% female and this I'm not talking about like assembly labor This is not like you know the people the foes you see of like Fox kind of people in blue smocks and stuff who are just You know doing rote labor this is these are people who are doing staff They're skilled in component selection sourcing manufacturing project management logistics and design and so forth so perhaps You know if you can see evidence that it's possible and other cultures are more inclusive maybe There is a cultural issue and there are a couple experiments and culture change that offer to Two of them I looked up that were well document studies involve involving cultural change That indicate that culture is a significant factor affecting inclusivity. So for example Carnegie Mellon Did an experiment where they tried to change the culture around Their curriculum and they managed to go from less than 10% enrollment to over 40% Roman in five years Females and in Harvey Mudd they had a similar result from 12% to 45% in five years And I kind of read through some of these and the key insight I got is that computer science is unique in assuming that you Already know it before you study it It's kind of like a medical program expecting that you've already done surgery before you enroll or a law school I expect you've already been to trial But a lot of computer science programs already assume you can write Reasonably complex programs he like language before you even get to college and so the intro classes are literally called weeder classes They're there to get rid of people who weren't exposed to computers before they got to college It's like walking into a Chinese class It's introductory and seeing a bunch of Chinese people there and then you know You have to get the best grade in that class for you to get on and you just don't you don't even want to participate in that situation And it turns out that this bias starts at a young age Here's a on the right-hand side There's a ratio of the number of children who are being given engineering construction kits by gender As analyzed by name. This is a recent study that was done and Because Because the bias starts a young age by the time you get to college if you're like, hey I want to go to computer science You just don't have the chance to really catch up at that point time you're excluded from the experience now Harvey Mudd and and and CMU were able to address the issues by redesigning their intro Computer science courses by offering courses as soon that you know nothing to in the beginning and to help you catch up but we can't count on Changing every undergraduate computer science program. So we have to start earlier We need to find a way to change the culture and so I was thinking about this a little bit more and I happened to be reading a biology textbook and I came across an interesting thing about sauerkraut I love sauerkraut and It's made with nothing more than cabbage salt and water and every time you put those three ingredients together You always get sauerkraut. You don't get something else Right if you don't have to start the culture with anything It just naturally turns into sauerkraut every single time and I read a little more about it and turns out that at the beginning When you have cabbage you have a neutral pH and you have all kinds of different Like you know bacteria and yeast on there and there's one particular Species of bacteria that eats the cabbage and releases lactic acid and makes it more and more sour creates more and more acid Eventually killing off everything around it until there's only one species left and every single time you put this recipe together You always end up with sauerkraut So if you want something other than sauerkraut, you can't fix the problem by just adding more Cabbage right you have to do something to change the culture or the conditions To get a change in the outcome and this sort of costing environment actually affects both genders I have a good friend in my collaborator Sean Cross He's the developer for novena firm veil. I need to be chubby and Linux Like he's the reason that the stuff I build isn't just shiny door stops that do nothing like you know The software on it actually is very important and he's a brilliant developer He's supportive and modest and he really values a work life balance But this is not kind of a personally type that Sort of thrives in a caustic environment neither him or I really lean in types. Some of you will recognize that word You know he got his undergraduate degree in psychology for various reasons and as a result his opportunities were limited And initially he was forced into like QA and support roles They wouldn't want to give him a development role until I saw some demos He did that were absolutely amazing is like I you know I said I want that guy in particular working on my stuff and then we were able to start doing our adventures together But that that sort of environment That he was originally put into didn't allow him to really stretch his legs and become everything he would be who he could become so To change the culture you have to change the green it's now like sauerkraut But also like beer and if you didn't add hops to your you know to your to your beer You would end up with kind of sour tasting beer half the time and that's not necessarily Something that does that's desirable and so I've been doing an experiment and change in in paper electronics and The question I guess is that by changing the ingredients of electronics can it change the culture around it so instead of You know going with wires and breadboards and so forth. What happens if you start with like paper and Copper tape and so forth and this is an idea that was thought of by Gichi And we ended up starting a company together We actually have a company called chibi Tronex around this and it's You know interesting to note that the company itself is actually primarily female I think the last thing you want in something looking to create change is a group of men making recommendations about what we should do for diversity and so If you look the initial results of Our experiment has been positive so far. This is from G's doctoral thesis where she went and analyzed The sales numbers of chibi Tronex from the beginning to today in the beginning. We were 70% male the green section is male and the red section is female And the blue is we couldn't tell from the gender from the name And then over time we organically grew into a more stronger female audience right now we're doing about 70% female which is pretty interesting for an engineering construction kit and Something that I've been trying to figure out is how do we go beyond just simple circuits and into something where people can learn or even love to code And so we've come up with a learning system that's aimed at increasing diversity particularly in computer science at young ages and so analyzing our experience in sort of chibi Tronex so far and try to think about what were the things that Were important factors in and having it become more inclusive I find that balance was very important. So we want to treat engineering as equal to design Familiarity is very important having materials our universe that people feel familiar Getting into and also simplicity is also very important. So I can go through each of these three Principles in a little more detail. So balance engineering is equal to design. So There are many good examples. I well familiar with of how design can revolutionize technology You know the mp3 player was sort of a boxy an interesting thing until the iPod came out and then you know The iPhone clearly revolutionized smartphones And there's a thing where you know blueprints alone are kind of inscrutable if you just you know look at The thing on the left the thing on the right. You don't really know what it's for It doesn't have a meaning to it in and of itself But these are this is the code in the in the prototype circuit for something that was much more immediately relatable This is a the dandelion painting that G did where she took Electronic circuits and put them on a backdrop and then overlaid on top of it a picture of dandelions And you can actually walk up to a blow on it and the white dandelions would like let go of their fluff and and drift across Across the page. It's a magical experience overall And so she's able to go ahead and take you know, what is inscrutable and turn it into something it's immediate really able by combining engineering and and design and So the curriculum that we've created treats engineering design as equal partners You start with an exercise for example here to create a pressure sensor And then you flip over the page and then you're challenging me to tell a story Why why did you build this pressure sensor? You can see the hint of it here There's a little heart and as you press it the red light starts to grow glow and then and then your challenge to go ahead and draw a story over the the blank sheet and You know diverse thinking is more inclusive. I think so When you measure Achievement only on one axis There's only one direction where you can sort of get to and it's a sort of a thinning out of the ranks as you get towards The ends of the axes But if you go ahead and you say, you know, let's look at both engineering and artistic ability and try to combine them together You have a much larger space Where you can create interesting things and it's important not to just you know write it off as like I say Okay, well, we're gonna put boys on one axis and girls on the other axis like in both axes You can have people who achieve very well in either one But when you go ahead and you look at it in a multi-dimensional space We have orthogonal parameters you create a bigger space for achievement and I think that can help get more people involved in technology familiarity is the second principle and using materials area are universal so Familiar things are naturally comfortable. So a lot of us started electronics looking at breadboards on the left hand side And there's a little magic that happens either that you have to sort of take them apart and show the people the underside that shows The wires connect underneath for you to get them It's not something that you know a lot of people are familiar with initially Whereas papercraft and in the style that we use it uses materials like tape and paper You know you lay down the copper tape and that's where the electrons go. It's very clear. It's very familiar Paper is universally familiar. It's an everyday material. It's you know books. We print on it We you know, we use it for bags. We make origami out of it Paper is also an incredibly expressive material. So you can do a lot of very interesting interesting things from the artistic side with paper this is a sample of of Artwork that was created as part of a exhibition that My collaborator gc did you can go to paper curiosity is that media at the MIT to you to check out more of them But there's a lot of really fantastic things that people were able to do with paper and just light And we want to extend that to even more with with programming But not only is paper and expressive material. It's also an engineering material This is a picture of actually a quick prototype that Zabs came up with of a bias network for split-core transformer Just means need to come up with like a you know circuit board like instead of taping a whole circuit board Let's go ahead and just use paper copper tape and make it all happen And it's really interesting that you can see it's self-documenting right You know you don't have to guess what the values were where things went You can write on the paper and it just all comes together. It's it's working with axel and through-hole components and so Paper versus breadboard is an interesting sort of comparison. I have the two side-by-side here Paper is Compatible with both surface mountain through-hole types whereas breadboards can only do through-hole Paper natively supports comments Whereas breadboards you can't really write on them Paper you can you get it's thin and flat and you can fold in 3d shapes So when you're done if you want to go and fold it over you can just go ahead and do that You can't do that with a with a breadboard The substrate cost is practically free Breadboards are relatively expensive, but on the downside, you know the substrates are not reusable You may require some soldering and the components are typically not reusable Which has been the barrier for us to try and get into more expensive stuff like microcontrollers and so G came up with this idea of Putting a microcontroller on essentially a paper clip a note a notebook clip This is a picture from a thesis on the top and we've been refining it into the thing that you see below Which is a rigid flex circuit board circuit board Which has a rigid portion of the PCB and a flexible portion of the PCB and we can use it to wrap around the edge of a clip board So you can go ahead and wrap it around the edge of a clip you can craft circuits on the pieces of paper you can Let it down and then the circuits themselves are in contact with the the microcontroller And so this is a very familiar format You don't have to explain to people a lot about how to you know get started with this what to do about it You just show us someone how to use it and it's very natural and it's very easy to use So we're working now on a sort of a general notebook clip type prototype where you just go ahead and you put your You know your microcontroller on a clip you go ahead and clip on the edge of a piece of paper a notebook and in your off encoding But the final thing that's very important is simplicity How do you actually get code on to the device in the first place great?  It's hard to get code into a microcontroller, right? It's like you have to have a cross compiler and you have to have some drivers and blah blah blah There's sort of stuff and and the thing is configuration isn't science or logic, right?   You should know these things already and so we really want to avoid Pushing people into that sort of cult of RT FM and the beating others You just start creating more of that sauerkraut culture again at the end of the day And so a challenge we were faced was finding an interface as truly universal to program a microcontroller You know there's things like USB and lightning, but all the required drivers and so forth But then there are things like eyes and ears we all these are how we interface with their computers the analog holes So to speak right ears are pretty universal eyes are pretty universal So I looked at those interfaces as options and sound is You know simple it's ubiquitous has good browser support, you know headphones are on everything almost everything I don't you know want to have to use your proprietary fucking ugly microwave transmitters in my ear to listen to sound but the more importantly Sound is also data capable So if you use sound we finally have an ability to Use the cloud and sound to upload compiled code and so I'll talk a little bit more about how we make that happen This is where the talk starts to get a bit more geekier It turns out that making things simple is really really hard We've been trying actually for over three years to kind of make a version of this product The first two generations were completely stymied by configuration problems We tried doing generation one generation to where you can see in the on the bottom here are at tiny based microcontrollers, which were really difficult for people to use Gen 3 was sort of like the prototype that G came up with and gen 4 is where we're kind of finally getting to something that we might be able to to share and We designed it from the bottom up for simplicity. So go through the infrastructure like how do we get? You know code turn it into sound and get into my controller So we'll start with what I call the file air kind of loosely a file air We use AFSK modulation audio frequency shift King. We use two tones to represent a binary zero one It's exactly like those old modems from like, you know, the 70s to 80s. I don't know if any of you all enough to remember it But we use 8.6 kilohertz and 12.5 kilohertz roughly and We picked the tones to be mp3 survivable We wanted these songs to be able to be encoded into mp3s and played back so you could share songs and not just Source code if you want to later on There was a bit of a challenge implementing this because we wanted to pick an MCU That had a price of under one dollars because that rigid flex circuit board You saw where we had part of it rigid part of flexible is not cheap That actually runs up the cost quite a bit So we had to find other ways to to keep the the bomb cost reasonable When we also in addition to be under one dollar had to be able to do about 20 megamax per second of performance Had him integrate ADC Open source friendly was very important to us And we also wanted to be backward compatible with a lot of the community stuff in Arduino Here's a kind of a big chart of you know Some microcontrollers that were considered from the at mega to that tiny general plus and blah blah blah and so forth There are definitely some cheap microcontrollers you can get off China, but they're not very open source friendly So we kind of threw those out There's some very popular ones like the at mega, but they don't have the performance and they're also very too way too Expensive and then you get down to some of the other ones like the psoc 4 the xmc 1000 LPC series And basically we found is anything under one dollars that range didn't have the performance that we needed Which left us with basically one of two options either SCM 32 the f0 L0 series or the kid at a scale of two We settled on a kinetics. That's currently one of my favorite series Probably mostly just personal bias, I guess It's a 40 megahertz Cortex M zero plus which means it does a 32-bit single cycle multiply very important for the DSP Has 32k of flash 12-bit Sardak In volume, it's about 90 cents to 97 cents, which is good There's no NDA required for the documentation and unlike the STM 32 it has fast GPIO bit bang capability one of the weird things we found about the STM 32 is that we want a bit bang There's this I guess a feature where they try to do synchronization of rights through some sort of gateway into the GPIO But it would like means that your maximum bit rate bit bang rate was limited. It would cause trouble for things like Driving WS 2012 LEDs are doing USB low speed This is the the data sheet for it. You can go online and read it and look it up all you want When it comes to connecting audio, there are three issues that we have to solve You know, you can't just put it directly into the microcontroller. You have to fix the DC offset We had to think about the connector cost and height and the cabling issue DC offset is important because audio itself Isn't necessarily, you know, it's basically an AC coupled signal Which means half of the the the waveform is going to be below ground if you feed that into the microcontroller directly You only get half your waveform. So we have to recenter the the the audio Second is the connector costs and high auto connectors aren't free. They're also thick So they're five millimeters tall the tallest component our board. Otherwise is 1.7 millimeters If you want to be papercraft friendly, we really want to make it thin so you can fold Fold paper over it without having a big lump in the way And also cabling quality cables are expensive, you know And cables also easy use especially in the classroom if I have to have the teacher with multiple cables to plug in their computers They're gonna lose some and it's gonna be frustrating experience for the teachers So we came up with this kind of hybrid solution Where we designed a cable that uses the fifth pin on USB OTG If you look at USB, it's commonly known as a four pin standard But the OTG connector has this fifth pin which is which identifies whether you're a host or a device We just sort of say we don't need that we're always going to be a device We don't have to have the AB capability and we just wire the left audio Feed into that pin into our device and then we created a Hydra a Y-shaped cable That gets us roughly two cables for the cost of one and a half So it helps us save costs it solves the thickness issue and it the solution ends up looking a little bit something like this where you have One thing plugging into the USB micro connector, and then you have ahead of both an audio connector and a USB connector for power electronic solution We AC couple With a an R2R network to an RR network to center up the audio The one volt p2p of audio centers up nicely in the 3.3 volt range of the ADC FSK waveforms are pure sinusoids, so we don't have to worry about DC balancing That's cheap a couple cents. It turns out that There's this bootstrapping problem if you ever touch like a Mac book or something and you feel that sort of like buzzing sensation Off of it. That's because that case is like floating up to like 50 volts or something like that, right? And so when you go ahead and you plug a headphone jack in there's this awful pop that comes out And that pop is actually enough to go ahead and put the microcontroller that we have into latch up And so we have to put these like diodes to absorb all of that voltage difference And when you first plug it in and that's actually the most expensive part of the design It's seven cents and then and then we wired up the USB D plus D minus lines to the MCU Why not and it turns out well if you went to Sean's excellent talk yesterday He implemented a low-speed USB stack that allows us to essentially turn LTC into something like a makey-makey if you want at the end of the day or you can just use the non-compliant Plug port you get your RX and TX out of the thing So in the software side, we just do non-coherent FSK demodulation It means we convolve the incoming samples by the sine cosine of each frequency So we have to do four convolutions each convolutions a series of multiply accumulates That's 250 multiply accumulates per sample at our sampling rate That's 20 million multiply accumulations per second hence the performance requirement of the MCU We sort of started from Lin modem and sort of Refined it until we have what we have now you can see the two Repo links for it at the O3 GCC setting it uses about 65% of the CPU so There's not a lot of CPU left over for doing anything else if we miss a packet There's not enough RAM for me to buffer anything this all has to happen online So if we have to do CRC or hash checking or 4D error correction It all has to happen in parallel with it with the demodulation of the audio and so this gets me into what I call The Mac layer it's not really a Mac layer, but this is sort of like how we deal with sort of errors and so forth The audio Starts with a herald a control packet If you actually listen to the stream you'll hear a little Versus static and then about a one-second pause and then the other stuff coming through the reason why the herald packet comes first Is that you want enough time for the microcontroller to erase its flash block if it if it's going to be reprogrammed Otherwise you're sending data while it's erasing flash and that takes a long time Every single packet is protected with a murmur three hash we picked that because it's 32-bit hash and it's fast and small And we have to compute this in line with FSK We couldn't do anything like MD5 or anything more computationally expensive because we just couldn't fit it into the inner loop We also had to do some bod striping so in order in order to recover the bod rate You have to have enough transitions So we do a simple thing we XOR every seventh byte with 5.5 and every 15th byte with AA we get enough transitions With you know typical computer code. I mean, of course, you can create something pathological. It doesn't work for this but usually compiled code doesn't have a pattern that matches up exactly to that and And then you know the data packets themselves as sync header version some sequence number And the whole thing is again protecting the hash and then our fact our forward error correction is we transmit the data three times so Pure redundancy. It's not really like a ECC or anything like that And the reason why we had to do it this is there's no back channel like microphone pinouts aren't standard You know if you ever tried to take any Android headset plug it into an iPhone or vice versa You you know the frustration and we want to avoid people having unexpected bad surprises So we had to build a protocol that was feedforward only Errors do happen in transmission. So for example, if you're you know programming from a smartphone and someone sends you a notification in the middle They go ping and that will cause an error Which will make your packet go away and so we have to do a retransmission And we find that three times is off is enough to basically get through almost every Condition and also because 60% of CPU and almost all the RAM is used for demodulation There's no CPU RAM left for any error correction code like hamming or sec that we just rely on and redundancy and pray So a little more about the next layer up once you get the code in what happens next We have an audio updater bootloader if the code is good. We jump into Chibi OS, which is a full multi-threaded microkernel OS And it has a bunch of application threads beneath it out of the 32k in flash We use 22k for the bootloader syscalls in the OS a bit for some signature and interrupt table management And it leaves nine and a half K for user application code Now Chibi OS even though it has named Chibi it has nothing to do with Chibi Tronix I want to be very clear in that it's made by this guy Giovanni de Sirio He's it's very actively developed maintained. I I'm particularly fond of the OS And the Chibi is just a borrow word from Japanese, which means cute or tiny or small So it's appropriate in both cases The Chibi OS itself has a HAL abstraction preemptive multi-threading semaphore is cues Synchronous asynchronous IO thread safe memory pools and allocators So it's a very you know, it's a feature full feature rich OS that fits in a few kilobytes Zobbs added to it a syscalls extension, which allows us to essentially do dynamic linking He uses the rmsvc call mechanism to create 250 256 thunks That allow us to share OS space code with the application code so we can implement the Arduino emulation layer EEPRON hybrid emulation and all that sort of stuff and the reason why we did that is we had to fit within the patience of a child constraint so 8,000 bits per second means about one second per kilobyte of code uploaded in a child will lose interest in about five seconds Okay, now the children aren't listening anymore Basically the baseline Arduino zero lot binary was about eight kilobytes when we first started looking at it And that would have been about eight seconds to just to get the baseline And so we wanted the average binary size to be less than a kilobyte So we did this the syscall extension to try and preload most the bloated commonly used libraries in our dream land like floating point operators and string operations and Also try to streamline the general loading process as a whole so typically, you know the average getting you know Kind of hello world getting started binary is just a few hundred bytes And even the bigger programs are only a few kilobytes And so when you want to compare the available code space and say an Arduino Uno versus LTC Consider that we've already preloaded about 12 kilobytes of libraries and Significantly, I wanted to make sure I wanted to make a product that was not just for kids so The LTC itself Extends from novice to startup. We want to create an on ramp that can eventually lead to professional level development Right. I didn't want to create something where if you got into it and you just sort of ran off a cliff And you couldn't go any further. So for example of the threading APIs are exposed in the syscalls The we want to make direct OS a bootloader development possible via open OCD if you saw Zob's talk yesterday You would know how to do that We made sure that the hardware components were China ready But also Compatible with small-scale prototyping so if you were to get into this and say hey I've built something and I want to make a few more of them You have the option of both going to dig a key and if you want to build even a few more of those You have the option of going to China and both of them are very low friction in that case and of course everything is all open source you can go to github and Either to mine in his rebote you find all of the the stuff there that little Section down there is just a display of the multi-threading interface inside gbOS And so finally talking a little bit about the the experience layer the user experience It's important that we had a very sort of simple user experience at the end day that just always worked So it's a web-based interface where where you can go ahead and code and you can pull up some examples and Then just hit upload and it starts playing out your compiled code as as sound There's a little bit of stuff behind it that had to be implemented to make that happen So we have a web IDE and examples that uses code mirror as its base And you can store your sketches locally using web storage, but if you want to go ahead and You know Share your code. We would like to put point people towards github because that's that's where we should point people I think The final see files that you want to upload are then sent to a server it's a it's a Docker image that's Kubernetes ready Turns into a binary which is sent back to the browser and then there's a web an audio modulator that attempts to Modulate it using web audio or wave via JavaScript And if an error happens we actually have a mp3 fall back Magnetism where we would then modulate on the server side and mp3 sent it over and then have them do an mp3 player in bed The compilation server requirements was that we want to have like very minimal bandwidth requirements So we want to be able to run the server like forever and not have to be burdened by its its cost And so the average sketch is about a kilobyte and the average binary is about a kilobyte When you want to buy this thing visiting our store pushes over a megabyte of content so we figure That was fine, right? You know if we were willing to to serve you the images to to to buy this thing We can certainly serve you the bandwidth to compile for it for quite a long time The overall design itself is stateless So it's legally difficult to obtain consent from people who are under 13 years old to store their data And so it's a very important feature to us that we have no user accounts We don't want your stinking data And if you want to go ahead and share code of people you go to get hub and you and you and you share it via there That's a plan feature, but we should have that in This greatly simplifies scalability maintenance So as we need to go ahead and scale up we just spin up more instances if one of them falls over We don't care. We know there's no databases to restore. There's no Concerns about losing user data and whatnot and so you can go ahead We have some Docker images if you want to spin them up and run them yourself and you know reduce our load You can do that as well So audio modulation we needed a just works method that doesn't involve uploading 100 kilobyte audio files web audio would have been ideal except that as of like late 2013 It wasn't supported in OS X And if you look at sort of the school environment or kids a lot of times kids are getting hand-me-down phones from parents and so You know 2013 is kind of the range you start bumping into when we did early tests and field trials or finding kids are like Oh, my browser is too old. I can't you know modulate the cat modulate So you go into wave which can be generated browser side by JavaScript And now you have 89% of the browser supporting this and you have to go all the way back to 2009 To really old versions of Safari before you lose support for that that seems all good except there's like This red column on the left, which is Internet Explorer And you're like, oh, why do we care about Internet Explorer? Turns out that 27% of desktop installs are Internet Explorer That's as of August 2016 Wikipedia if you can believe it but what we're finding in particular is that it's significantly highly used in schools and Institutions with inflexible IT policies they get windows they install and they don't want you install anything on these things And so you're stuck with Internet Explorer in a lot of school environments And so in order to handle that situation We offer we would fall back to the server site codec send an mp3 file which As of IE 9 you can go ahead and play that so hopefully this gets us pretty good coverage It's about 93% of the browsers out there So the net result at the end of day oops All of this is that we have broad compalability And simplicity and ease of use we have a device that can be programmed with everything from a smartphone To a laptop to even if you had a high enough fidelity record player or a cassette tape You could go ahead and pre-record Your songs onto those and part of the idea that this is that we wanted to try and Draw on people who would even be afraid to code in the first place But had some desire to customize our technology So if you're a crafter and you want to create a certain pattern of lights and you don't think coding is for you We wanted those people be like help You know I want to have something that does the sort of thing and someone could share you their song and they Could then program it and be like oh, hey look I can control I can modify my technology This is something I can do but then hopefully they would be like I want to do a little more And then they then it's like well, maybe I should just look at the code and they find it just a bunch of numbers They change like all of a sudden they're sort of tricked into getting to the whole coding exercise by By sort of being pulled into the idea that technology is something you can control and not something that they're sort of Be holding to at the end of the day So LTC fosters diversity through a combination of balance familiarity and simplicity You know we look at making sure that the curriculum itself balances you know design and engineering and We try to use familiar Interfaces materials Such as paper and copper tape and so forth to try and bring more people in and we Shoot for a very very simply experienced getting started Through the web base interface with the audio upload So at the end of day, you know open source is built on inclusiveness. It's not just about sort of committing and forgetting It's all about pulling and merging and fork You don't have to always take the pull requests, but it's really important to accept new ideas and to improve the code base And so if we care about you know open source We need to empower more of society to understand its value or society may end up determining its value for us In a vote and if they don't understand what they're voting on it could it could it doesn't necessarily go in the most enlightened fashioned and so that's My talk about the left to code platform We'll have a beta in 2017 you can follow chibi tronics or visit chibi tronics com for updates if you go to the failoverflow assembly This is a picture of the sign there The little lights on the bottom of the of the failoverflow light are being driven by an LTC microcontroller right now So you can walk over there with your whatever device you want and go to ltc.zobbs.io Which is the current staging server for the latest stuff plug in the headphones write some code press and hold reset on the microcontroller And hit upload and see what happens. I think it'd be I mean we are looking for more evidence to see if our stuff works Truly cross-platform so if you come by and it doesn't work for you for some reason we'd like to know what actually has gone wrong and That's it. Thank you Thank you very much bunny and Now for questions. Do we have any questions from the internet? And while we're getting the questions from the internet and the mic working for the signal angel You can stand up towards the microphones if you have any questions in here and while we wait We can have another round of applause for bunny. Could we have question from microphone number one, please? Hi bunny first of all. Thanks for the talk. I've noticed that for the Development environment you are using Are you you know flavored C++? Are you sure are you sure that's a good thing for our beginners? Yeah, that's that is actually an interesting debate so There are block-based programming languages like scratch that are very good for beginners and we have Active efforts to also develop that as well It's kind of my feeling that I wanted though to create An on-ramp to get people like you know I'm targeting sort of like people who want to go undergraduate level college courses And so if you enter one of those situations and you've only been doing like block-based programming And you haven't seen the syntax and dealt with semicolons and stuff then then it can be a little overwhelming And so I also don't necessarily think that C++ are there Do we know libraries are necessarily the best? introduction But it's it's very well community supported at this point in time I Think there's like sort of like a daydream that maybe we would do like we're talking today like a rust based system that could let you Compile an upload code to it or something that that is kind of neat neat and cool But the problem is there's so few people use these languages that people had questions like we wouldn't have anyone to answer them So I see thanks Have question for me internet Hello, there's a question from IRC and it's a bit long How can we tackle stereotypes when the image of the sweaty male teen in a darkened bedroom hacking the planet is the mainstream? I see this as a lack of understanding of a whole number of factors Notably lack of understanding of the capabilities of computers and how to use them the wider lack of diversity in the sciences As well as cultural ideas of what an office run by mail management is like How do you think that we can tackle this stereotype in culture? All right, I think I That was a that was a lot of words which I which I think I'm trying to try to even parse in my head I think I had a buffer overflow about halfway through that I think So I caught part of it sound like there's an issue with the stereotype that exists in society But then then the question was how do we tackle that stereotype? I guess I Mean, I guess the guy's not here, so you can't yes, that's what's the question. Yeah, okay I mean, I think I think by just in general like Like what we're proposing here is like it starts from the beginning just becoming more inclusive at a base level and changing the culture In getting more people involved that stereotype will naturally sort of melt away as it becomes more inclusive and more people like become involved I think I Don't think it's something that Yeah, it's not something that we that you How do you say you can't? I'm having struggled to find the words for that for what I'm trying to say here But but I think that basically at the end of the day the theory that I have is that if if we aim to be more open-minded as a Community and if we aim to be more inclusive particularly at the beginning The culture should naturally grow into something that has I think less of that stereotype around that people become more aware as more people Can become aware of how we are and how we operate they will they will they will you know say hey That's not the stereotype anymore. These people aren't all like that at the end of the day. I think so I have a question from microphone number four Hi, hi. Yeah, so my question is a little bit related to the last one when I was younger I was very curious. I actually wanted to learn program and more technical stuff, but I just couldn't I didn't know How to get the information our school didn't really have a program. So with your product You're offering a great Great opportunity to learn but the problem is to get it to the people who want to get involved So do you happen to have a program? Are you? Involved with school with schools or people? Yeah, so that's a good point Availability and getting These things into the hands of people who want to do it is very important And so absolutely. Yes. I mean one of the the core focuses of Chibitronics is schools. We we We do a lot of outreach and we try to engage with them directly It's kind of My marketing manager gonna kill me But it's like a secret that any school ask to get 20% off right just just for asking senior school, right? And then you know in so-and-so for so so like so but the thing is is like we want to you know Give discounts and get it into schools because that's where it makes the most difference at the end of the day The another thing is like we you know when we were trying to put together the supply chain and you know There's a lot of effort We went to actually getting the stuff to fit in this really low power low cost microcontroller at the end And a lot of that boils down to like the cheaper We can make it the more accessible the technology becomes and so we wanted to be able to create something that Comes at a price point that schools could afford and get into Situation now that they're unfortunately like schools that are so poorly literally can't buy dirt like we were contacted by school in Pasadena Who was interested in it and they're like you know to give me an idea of a situation We had to cancel the gardening program because we couldn't buy dirt for the gardening program like Okay, like I really want to help you but but you know There's some other major social economic issues happening when your school is underfunded to that point and we have to address Those problems maybe separately, but ideally you know, we can we can get this into as many hands as possible by focusing on the schools Brilliant, we have at least three more questions. So please remember questions and not comments microphone number three, please Yeah, first of all great work. I really like the idea of uploading code using audio So I have a question to the technical aspect of this So why didn't you put a microphone on the PCB so you don't need the wire? That's a good question We had thought about that and I thought that I've been really cute except that when you consider that you have a classroom full of students And they're all trying to program And then you have students programming other people stuff it ends up a it's noisy and B it becomes there's a lot of crosstalk it's Certainly possible to do it microphone based, but we decided that it would be more You know you get less crosstalk if you use it headphones directly Yeah, good point. Thanks. Yep microphone number five Questions gone microphone number one then yeah, my question is Did you look for I mean they are ICs who? Demulate AFS K directly. So you don't need the processing power on the Microcontroller itself. So did you look for those? A separate I see that could do the AFS K modulation directly so I didn't have to do it on wider range Microcontrollers right right so some sort of hardware assisted situation for doing this I mean it turns out that when we're in this sub one dollar constraint like Like even adding an op amp on the outside for example I talked about the DC level adjustment like we thought about using op amp initially I've added like 10 cents that's very 10% of the microcontrollers cost and so we're getting on to optimizing pennies adding a whole Never another packaged integrated circuit outside to handle the FSK as like a DSP or a prepackage solution It would have to be like under 10 cents for it to be worth it And those like that type of technology isn't really available today in that low cost because I mean partially Because no one does no one does AFS K anymore like the modem chipsets are all dead and buried and like you know One micron CMOS anyways, so So yeah, I mean it would have been great if we could find something that could offload that from the CPU But we couldn't we couldn't find anything like that Thank we have one more question from the internet and then we might try and squeeze two more questions in Okay, a question from Twitter. Do you know about the calliope mini and the micro bit edu? And are these comparable to your attempt? The calliope mini and the micro bit. Sorry what? Microbit audio you said That was not included in the question. Oh, let's try and see you get that answer I do I do know about the micro bit and that one's a very interesting one I was I was trying to hear said the micro bit audio was like, oh, did they are they doing audio programming too or not? But I think the micro bit is more a conventionally programmed Microcontroller I haven't heard about the other one But as far as I know, I think we're the only one that does AFS K based programming audio based programming Which allows you to sort of open up to a much broader range of devices and part of the reason it turns out that it was so important for us to get into audio programming is that The weird thing is happening in the world and that kids don't have notebooks today They don't get laptops the first computer kid gets is a tablet or a smartphone And so when they were particularly younger age if they're introduced to coding and they're giving a homework exercise And they come home and say mom dad I have to use your like your work computer to do my you know homework they say like sorry like you know use your use your smartphone, right and so and so We really wanted to create something that would enable you know kids who are you know the dominant thing to get You say a smartphones and tablets to be able to Try to learn coding. It's admittedly very difficult to like code on a touchscreen because touching keyboards are crap for that But I'm really hoping that I think that's a problem that's solvable if someone actually cared to write a better Keyboard interface for coding. It's like right now. They're all just optimized for texting microphone number two I'm kind of surprised the pay the taping down copper and soldering on paper works as well as you like did you have problems that you had to solve to get that to work and Is soldering or is there not a better option than soldering for that? But so so yeah, maybe I didn't make this very clear But so so copper tape and paper works great First of all, I mean the copper tape itself has a nice property It's very easy to cut into finer pieces. So if you need to go finer pitch You just slice off a piece and you can do like SO 16s and SO 8s and stuff The the soldering is only necessary if you don't have a solos method. So the typical way we do Teach with it for example the LEDs is you have stickers.  That's that's actually one of the constraints of of our technology is to get the dangerous soldering irons You know apparently soldering is forbidden in the hack center. We were told that earlier like it's a dangerous activity And so we had to get that out of the classroom And so and so so yes actually once once you get into the you know once you have the conduct conductive adhesives it's pretty easy to use and Surprisingly like paper itself is doesn't burn at all when you solder it That's that's something that originally when when G showed me the technique. I was like can you really do that? It's gonna like you know catch fire and lay now It's actually it works pretty well the copper tape itself is a really good heat sink So pulls out almost all the heat away from that from the paper I mean if you just stick the solder tip on the paper you'll burn a hole through it But if you're soldering the copper tape itself so much heat gets whipped a way that it doesn't end up affecting the paper Good excellent. I think we don't have any more questions. Could you please give bunny a brown? One world round of applause


Hi and welcome to the session. Let us discuss the following question. Question says, complete the following statements. First part is probability of an event E plus probability of the event not E is equal to dash. Let us now start with the solution. First of all let us understand that for any event E, P E that is probability of an event E plus P E bar that is probability of the event not E is equal to 1. Here E bar stands for not E, E and E bar are called complementary events. So from this expression we get probability of an event E plus probability of the event not E is equal to 1. So we can fill in this blank 1. This is our required answer. This completes the session. Hope you understood the solution. Take care and have a nice day.


Again, using that good setup, double thigh ride, get him going forward, here, pull the ankle, put his leg in here, here, here, all right, put his leg in, he goes flat, our back point, now he goes flat again, now let's start building up our back point, go straight five more, okay, let's try that, he goes flat, now go to the bar.


Sunscreens are intended to prevent further facial photoaging, but can't dramatically reverse damage presently visible, whereas topical retinoids are the most important drug class to reverse the size of aging. Also known as all-trans retinoic acid, tritinoin, sold under a variety of brands including Retin-A, is a prescriptionally topical form of I-M-N-A that can visibly improve mild to moderate photo damage, including fine and coarse wrinkles, freckles, other pigmentation, improve overall skin texture after months of daily use. This is accompanied by a restoration of collagen formation, skin biopsies taken before and after 10 to 12 months of tritinoin versus placebo found that while skin collagen formation dropped 14% in the control group, it rose 80% in the retinoic acid group. The downsides are the side effects. So common they have their own name, retinoid reaction, plaguing a high proportion of patients including redness, stinging, burning, itching, and peeling. Ironically, the constant irritation may be a causal mechanism, constantly forcing the skin to renew. The adverse effects do tend to improve over time, after a peak after about two weeks of daily use, and you can minimize the irritation by starting at a lower concentration, for example 0.025% instead of 0.1%, and slowly ramping up from applying at first twice a week for a few weeks then to every other night for a few more weeks before finally advancing to nightly application as tolerated. Given that a small amount is systemically absorbed out of an abundance of caution, women of child-bearing capacity may want to steer clear, given the severe birth defects caused by a similar drug, isotritinoin, sold formally as the Acne Drug Accutane. There are gentler, less potent over-the-counter topical retinoids, retinol, retinol, and retinol. There's retinol acetate, retinol palmitate, propionate, retinol, retinaldehyde. Once absorbed into your skin, your body can convert them into small amounts of retinoic acid. However, data is limited on these non-prescription retinoids. The results of clinical trials on the retinol esters have been routinely disappointing. Even 48 weeks of daily use was found to be ineffective for reducing wrinkles compared to placebo. Retinoldehyde and retinol, however, hold some promise. There was a 48-week head-to-head trial of retinoldehyde versus retinoic acid versus placebo. Now retinoids can be so irritating that it's hard to blind participants as to whether or not they're in the control group. So objective measures that may be less susceptible to placebo effects are especially important. So what they did is make silicone molds of the crow's feet wrinkles around the eyes of the participants before and after being randomized to one of those three groups, and then analyze scans of the molds using high-resolution digital image processing. The retinoldehyde group showed a similar reduction fine lines and wrinkles as the retinoic acid group with less irritation. Only 23% of the retinoldehyde group experienced skin irritation compared to 71% in the retinoic acid group versus 4% in the placebo cream group. Of all the non-professional retinoid options, retinol may be the preferred choice, though. It causes even less irritation than retinoldehyde, in significantly different than placebo, and a randomized double-blown placebo-controlled trial found that 52 weeks of use improved crow's feet fine lines by 44% and modelled pigmentation by 84%. But tritinoldehyde retinoic acid has by far the most robust track record of efficacy, so why not just ask your doctor for a prescription? Because long-term topical tritinoldehyde use may increase your risk of an even more stinging side effect? Death. Topical tritinoldehyde therapy in all-cause mortality. The Veterans Affairs Topical Tritinoldehyde Chemoprevention Trial was to be a six-year randomized controlled trial to see if it could help prevent skin cancer. But the trial had to be stopped early because significantly more people were dying in the retinoic acid group than the placebo cream group. At the time the study was halted, 19% of the subjects in the tritinoldehyde group had died, compared to 14% in the placebo group. Between 1% to 8% of topically applied retinoic acid is absorbed into the bloodstream. Could it be killing people? We don't know if the increase in deaths was a statistical fluke that happened by chance or a real biological effect. The probability that we'd see such a discrepancy just by chance is about 1 in 100. Tritinoin continues to be banned in Europe for cosmetic purposes.


What's up everybody, Jason here for jaspyskatesprac.com 2023 hip parade baseball autograph limited edition series one just sold out. This is a dual case break random teams number one. And again guys, 20 total boxes, remember 30 spots. Everybody that's around the team now will be only one hit per box guys. So there's a good likely chance that you might go hit list but hopefully you don't. There's a couple of the sell sheet hits there. You can potentially get out of this. Now let's do the randomizer guys first. So here's the dinosaur, there's a customer name. So these customers from Jason down at Nestor got in straight up Dennis, Dennis Raymond got those from Chronicle 7 packed filler her box break. And you got diamondbacks to the nationals. Roll it, full 12, six and a six. Here we go, one, two, three, four, five, six, seven, eight, nine, 10, 11, 12. 12 times, Dan at the top, and then a Ryan at the bottom. So there's 30, here we go, 12 times on the teams now. One, two, three, four, five, six, seven, eight, nine, 10, 11, 12. Orioles done in the Royals. So, Dan, you have the Orioles.  So, if anybody wants to trade, give you guys a quick trade window. If no trades, then we'll close it and we'll start the break. So I'm gonna pause the video guys. Alrighty guys, just one team went for trade, nobody chatted back, so we just closed it. Here we go guys, so let's rip open the first case and then we'll go on to the second case. First one we have here is a little A-Rod, and it looks like Texas Rangers, signed legendary game jerseys. Very nice, Texas Rangers, Alex Rodriguez, and that is David Dolly. Then we got a Alex Bregman, Bowman Inception, 1099, autograph, Houston Astros, Houston, what a J. Nice Vladdy, Vladimir Garot Jr., that looks like it's a sell sheet, I believe, right? Yeah, for the Blue Jays, that's a rookie autograph from Transcended Baseball, seven out of 15. Nice one there for the Blue Jays, Nester, with that one. All right, let's see what else we can find in here. We have Paul Goldschmidt, Maculade Marks, to 25. St. Louis Cardinals, that is Nester again, that's last spot mojo this time though. We got a Spencer Strider, look at that, Midnight RPA to 25, for the Atlanta Braves, that's Jordan Cohen. We got a Jose Ramirez to 10 for Cleveland, Cleveland Guardians, Aaron Roach, tried to trade, couldn't trade mojo, man, there you go, on the board. We have a Salvador Pettis for the Royals to 25, Triple Relic from Museum Collection Baseball, Kenzie Rose Ryan with that one. All right, three more to go guys, then we got another case. Nolan Aronato to 30, Carvalho Rockies, Nolan Aronato, Rockies that is Tyler, and a little Frank Thomas, the Big Hurt, on card from Finest Baseball. Chicago White Sox, that's Raymond. One out of 100, box one. And a little JD Martinez from Gilded, little Rose Gold, right? One out of five, that's pretty sweet. Nice one there for the Boston Red Sox, that's going to Devon. Alrighty, first box in, or first case in the books. There you go, just the way they came out like that. A lot of different teams hitting there. Let's see if we can still mix it up a little bit more in the next case. There could be repeats. Here we go guys, 10 more hits. We got a Chase Petty out of 75, the Blackboarder and Bowman is so nice. It's a first round pick right there, Minnesota Twins, Nestor with that one. And a Joey Vado from Dynasty to 10. Cincinnati Reds, one of Brandon. Ozzie Smith, you're a wizard Ozzie, that's from the St. Louis Cardinals. St. Louis Cardinals, that is Nestor, last spot mojo. So a couple of repeat hits already. Jared Kalanek rookie autograph for the Mariners to 75. Gold label. Mariners, that is going to add. We got Glaibre Torres from Definitive, nice little patch autograph to 10. Yankees, Nicholas, that's with that one. Alright, five more hits guys, halfway through. And we got a Jeff Bagwell from Five Star. Houston Astros, going to J. Four more to go. We got David Ortiz, big poppy. Decades, Otto, Boston Red Sox, that is Devon. Alright, three more hits guys. A dual autograph, wow, we're going to have to randomize this bad boy, how about this? That is also a sell sheet hit. Ronald O'Cunha Jr. and Juan Soto from 2022 Flawless. So that's Atlanta Braves, Jordan. Washington National Steve Burch. So that's going to be a little randomizer there. One happy person, one sad person. But that is the case hit for this case. That's another sell sheet hit. We got Ronnie Mauricio first for the Metz. It's a 499 refractor. Metz, that's Dennis Jenkins. Alrighty guys, and the last box here, last hit, box 10 out of 100. Here we go guys. Alright, if you guys like this hit for it, we do have a football version of it as well. And it's Ozzy Alves for the Atlanta Braves to 15. So Braves, you're for sure getting that one Jordan. And there you go guys. Nice case right there again. Let's do the randomizer there for Atlanta Braves and Washington Nationals. Switch it, Braves Nationals. Click dice roll, two and a one, three times. Good luck. One, two, three. Go to the Nationals. Two and a one, three times. Nationals, that is Steve Burch. There you go buddy.  That was the break. Hip Parade Series one in the books. Jaspy's casebricks.com.


Hi everybody. Also, can I just say that Jay's presentation was fantastic, and I just have another round of applause for that. Awesome. And I'm just getting back into playing D&D as well, so all of you should too. All right, so now we're going to talk about this marketing thing that we're actually here for. Let's start. With, unfortunately, the reporting struggle is real, right?  I hate it.  We spend so much time doing it. It doesn't feel billable. It feels like it sucks, and then we give it to the client, and they don't read it anyway. So why do we even bother? It's very depressing. And we've been talking about this for years, right? Like, from the very beginnings of our industry, reporting has just been bad all the time. But yet, we keep doing the same things all the time. Why? I don't know.  Well, our reports were bad. You know, we wonder why people don't take our recommendations. Our reports were bad. So our communication, especially via reporting, just really stinks. And it doesn't help us as an industry, because if we're giving people bad stuff all the time, then how are people supposed to tell the good marketers and the bad marketers? Well, they can't because we're all terrible. So now that I've got that depressing stuff out of the way, let's share our struggles. We're going to name our problems, and then we're going to solve them. And then I promise by the end you will feel way more optimistic about reporting than you do at the current moment. This is the trick of presenting. It's like, I'm going to make you feel really sad, and then I'm going to pump you up, and it'll be awesome. There's three major problems in reporting. The first one is that we often confuse clients with the things that we give them because we're confusing the idea of monitoring with reporting. The second thing is we just vomit up stats, hoping someone will trust us because look at all this data. Something in here must be meaningful, right? That's does not work. And then the third thing is that we're reporting on things in the end that don't actually mean anything to your customers or your client's customers. The people you're actually marketing to, those people, doesn't mean anything at all. So how can we fix this? We're going to start first off with confusing, monitoring with reporting. And you may be thinking to yourself, well, what do you mean monitoring and reporting? Well, what is reporting? This is a very deep question of the day. I know it's quite late to be asking a deep question, but if a tree fell in the forest and a GA event wasn't recorded to account, right? Like, what is reporting at the end of things? So monitoring is data. Reporting is insight. Clients do not pay us for data. Anybody can get data. Clients pay us for insight. For example, your organic is decreasing. Monitoring the organic, that's the symptom that is decreasing. Reporting gives you the result on why it's decreasing and how you're going to fix it. Often, we report just on monitoring stuff. We don't report on reporting stuff. So here's a great example. One of the tools that we really enjoy using at Kickpoint is called Deep Crawl. I hope there's lots of happy Deep Crawl customers in the audience. It's a fantastic tool. So one of my favorite reports in Deep Crawl, and how much of a nerd am I? I'm like, one of my favorite reports in our crawling tool. Yeah, I've just, I've been on it. I've been doing this a long time. I'm a huge nerd. The report that tells you how many pages existed on your site, but yet did not receive a single organic visit. This is a sad report, because it means that all the content that you did was just, right? Like, Google's doing nothing with that.  But the problem is that often when we're doing reporting, we don't actually look at that kind of thing until the end of the 30-day reporting cycle, and then we give the clients a report that looks like this. And then we're like, yeah, that's really bad at the end there. Something happened. That's where the box is. Something happened. We'll get back to you. That is a terrible thing to have. Like, nobody wants that kind of meeting. You ruin trust between you and your client or your boss, right? Like, everything is sad. Instead, if you were monitoring this and actually keeping an eye on it, then you would know ahead of time. So, for example, in Deep Crawl, again, we have this report. When a crawl happens, have it push a thing to Slack that says, hello, I have finished a crawl now. Please come look at me. And then you can see if things are good or bad. And then you can proactively say to your client, hey, look, we did this crawl. We found that this is increasing. There hasn't been a major holiday. We're not sure why. You know, World Cup is not on or whatever it might be. You know, all the other things that screw with data and the amount of people coming to your website. We're not sure what the problem is and we're going to look into this. That's proactive and that's because of monitoring. The client doesn't actually care about this kind of report. They don't want this kind of report. They want to know that you're on it. That is the difference between monitoring and reporting. And monitoring is still incredibly important. You have to monitor the health of your work. These things just don't need to make it into that final report because they aren't something that tells us where we're going, where we need to go. It's a difference between strategy and tactics. Social strategy is not post four times a day on Facebook. That's not a strategy. That's a tactic. Monitoring is not, oh, hey, the number of pages that Google visited is going up or down. That's not a report. That is a monitoring point that you can turn into some insight and do something with. And the other part about reports is that they should tell you if you're actually meeting your goals, which brings us to the next part. We deliver reports with every possible metric under the sun. And so I have some reports that we have received from clients who've worked with us and given us the reports that we got from the previous agencies. And if your report is here, I am very sorry. Okay, so this is one. Look at all the green. Green means good, right? They got these wavy lines, you know? What does this mean? That's great. Year over year increase of 14% of organic traffic to the site and 17% of new visits.  Like, you could have used two different phones and gotten that. Secondly, like, what... They don't care about the difference between day one and day two. And actually, my favorite part of this report, and you may not be able to see it here when you download the slides, you can. Over on the right-hand side, it says other options. And then it says mobile versus desktop. And this is show SEO start, 1101, 2017. I assume it's an internal metric. So you just, like, turn that on and your stuff looks better. This is a real report a client got. Pretty meaningless. This one is even better. This is for car dealership. So this is all they got and it was accompanied by the same data in tabular form. There's no context for this. People filled out forms. Congratulations. Here are the different forms presented in the worst possible way visually that they could possibly be presented. It's kind of amazing. And this one... This is not a small spend for this campaign. This is a month $6,200 on AdWords. And they didn't hook up conversions.  This is, I swear, this is a real report a client got. And then you wonder why you get fired, right? It is this right here. So if these are any of yours, like, we can talk later. I'm not mad at you. I'm mad at the report. And maybe I hear all sorts of horror stories about the stuff that people have to put up with and other agencies, and so, like, we can talk about it.  We can get there. And we've all made bad reports, right? I spoke about reporting here at 2014, and I showed a really spectacularly horrible report that I gave to a client who thankfully has not fired us.  They should have fired us. So if you want to go watch My Shame on YouTube, feel free, because that report's there. We're just going to let it go now. So why do we do this? Why do we include every possible metric in reports? Why do we just throw things at the wall and hope that something sticks? And our theory is that it's insecurity. Maybe the campaign isn't working. Maybe we don't know if the campaign is working. Maybe we don't have access to the stats that we need to prove that this is actually working.  And then you keep paying your bill a good day. Right? Like, this is what happens with these kinds of reports. But what actually happens is that the client looks at this and says, your report is built into thrown of lies. You know, if you can't trust the data, then we can't trust you.   It just makes that rock bigger and harder to push up that hill. And often being in digital marketing, we can feel like we're pushing a rock up a hill and endless giant hill, and there's, you know, bears. So, right? Sometimes it feels like this. But that trust, if we have that trust, it makes that rock smaller, it makes that rock easier. Maybe even if you have enough trust, the client will come and help you push up that rock. That's nice, right? So, let's start with the end. Let's start with where we're going to go. Before, at Kickpoint, before we start with any project, we want to think about the end of the project. What's going to happen when we've gotten to our destination? There's going to be balloons and streamers and donuts and puppies, right? It's going to be great. So, the way we get there with the puppies is we introduce a goal charter. And again, going back to 2014, I talked about how you can take goals and turn them into metrics and KPIs and reports. Well, this, since then, in the past four years since that presentation, we've used that technique, but this has actually turned into what we now call the goal charter. And we didn't just name it that, so it sounded fancier. It's actually a different document now. So, the goal charter is where you take every single goal that your client gives you, and then we break it down into clear objectives and measurable results. It is a document. It's not a spreadsheet. It's actually like a narrative. We're telling a story in the goal charter. It answers three key questions for the client. What is the goal? It's harder to get than you may think. Why did these goals exist? This is the objectives. This is what we call them. And how will we know we've met the goal? What are your results? My goodness. That sounds so easy, right? These are actually extremely difficult documents to write. But it's a really great way to start off your engagement with the client because you're really diving into where do you want to go? I just want you to buy me AdWords ads. Okay, but before we spend a single dollar here, let's talk about how we're going to know when we get there. And that's the important thing about these goal charters. Plus, it brings you and the client. You get to know each other a little bit better. It's a really great way to start on the right foot for a project. So, recently, I tweeted, I'll be talking about reporting. Please tell me your reporting challenges. Give me some real goals that you have been asked to meet. And what was really interesting about the responses to this tweet is I didn't actually get a lot of goals. I got a lot of measurement challenges, which I think is actually something that's for another talk, which is a symptom of how we approach goals in this industry. We approach them as measurement challenges. Clients just want to know if their phone is ringing, right?  They just want to know if it's right. And then they can make their own judgment call and if it's good. So, that's the important thing is make sure you watch your language when you decide how you're going to approach these goals. And the client, they just, what do they want to know, right? That's the goal. So, I chose one of the goals. It was a pretty clear client request. So, Matt, thank you for asking this. So, he said, I recently had a client ask to report the percentage of people who clicked driving directions from Google My Business Listing that actually made it to the store for 2,000 locations. Easy. Okay, I said, please tell me more, Matt. So, their goal is driving foot traffic through local pages and Google My Business Listings.  They wanted a better idea of driving directions. The assumption is people clicking driving directions are way more likely to visit than just a local searcher. The idea is to determine an ROI for local listings management. Great. This is what I need. So, the goal is how much budget should we put towards local listings management, which is a little bit different than, say, 100 sales, right? But it's still a really worthwhile goal because this can help us figure out other goals. Once we know this, then we know a sense of budget for this thing, and then we can free up budget for other stuff, like paying us more to do some cooler SEO stuff. So, the objectives, we want to know that we're spending the right budget on local listings management, and part of that is we need to understand the ROI of local listings management. So, how do we measure that? Well, there's a few different ways you can measure that. You could... The Google Beacon Project, which some people may be familiar with, has real potential with this. It's where Google sends you a beacon that then reports back to the mothership on people and their phones, and it doesn't sound creepy at all when you explain it to regular people, right? Just do not tell regular people how we do this because they will turn off location tracking, and that would be bad. So, that might work. You could also, and this is how we would recommend measuring it, you could compare driving direction request numbers with people counters at your individual locations. So, you can't say this person had driving directions requests and this person didn't, but let's say, for example, I know that 2,000 people walked in the door and there were 200 driving direction requests. Next month, 3,000 people walked through the door. There were 300 driving direction requests. Okay, well, it's 10% each month. That's pretty consistent. Then, over months and months and months of measuring this, if that percentage stays consistent, then you know that this is a metric you can trust. There's no correlation that you can trust between these two numbers. And we've done this for several clients. One of them that worked really well for was a home builder because people just wander into show homes and we're running radio ads, so how do we tell the percentage method? And really, if there's no correlation, then you know that's a metric you can't trust and that's okay. You can say, you know what? We tried this, it didn't work. We're going to try something else. And maybe it's more ad words and you can absolutely tell driving direction requests from there and Google ads or whatever they're going to call it next week. So you'll get some metrics in there and you can mash it all up in Google Data Studio. So that's the way you can think through this kind of goal and be able to tell the client, okay, we're working on this. And then they can see the results at the end of this. So for every single goal, go through this process. We started working with a client that has 14 goals. That's a lot of goals. So we decided that four of them were actually like right now goals and 10 of them were later goals. And from those four goals, we turned out seven specific metrics that will tell us if we're actually meeting those goals. So just seven. This report has seven things in it. I'm going to show you a sample report. This is not for this particular client, but this client, they're a moving company and they want to know, you know, did people look us up? How many people looked us up? Are pages getting more organic visits? Well, it's like, okay, well, more organic visits in itself is not a goal, but are you getting more business from organic visits, for example? So really thinking through these goals quite deeply. And then you end up with a report like this, which I think if you work in local at all, please feel free to steal this. This is a report in Google Data Studio for a real client. This is real data. I took their logo off, because obviously. One of the things is they want to know, this client loves knowing if their phone's ringing. That's what they like. This is the phone ringing. Are they getting emails or visitor appointments being booked? They're psychologists. They want to know if people are coming in and getting psychology. So this is for last quarter is what this report is for. So you'll notice that in the report, it doesn't say phone calls. It says, is your phone ringing? Micro copy in reports is so important. I know micro copies is like huge buzzword, but these are micro copies by their very definition. And it's very important that you write it out so the client understands exactly what you're talking about. Is your phone ringing? Our visitor's calling you once they're on your site. This is the number of people who've called via call rail because they have it implemented on their site. Are you getting emails? This is our visitor's clicking, tapping, your email or filling out your contact form. And in Data Studio, we add up the number of people who are clicking or tapping their email as well as the people who are filling out their form as one metric and presented here. And then our visitor's booking appointments, people who fill out the appointment form. Not all necessarily come in and then they report and eventually we're going to be able to take this because Data Studio just like last night mentioned that you can blend data sources now. Woo, right? Nothing has made me more excited because I was going to get mad about this in this presentation. Now I can't. I'll have to get mad about something else. Anyway, but now we'll be able to mash together No Shows and be able to deduct it from there and anyway, it's going to be great. But the other part of this is which pages your visitor's view because once they start blogging, this is going to be important for them. And we write, once you start blogging, this will help you find out what content is most useful for clients, et cetera, et cetera, et cetera. And you may notice at the bottom we've got a what's next and how to use this. What's next is like stuff we need you to do, client, start blogging, for example. And for us, we train clients. They do a lot of the implementation themselves so this is really important to remind them. So when they go and look at their report, they remember, oh right, I was supposed to do that thing. Maybe that's why I don't have as many visits as I was supposed to because I didn't do the thing that I was supposed to do. So it really helps with that open communication. It helps with your clients. And really, if they just look at the top line and that's all they look at, great. They can make the time period. They can look at this whenever they want. They can look at it every single day if the mood strikes them. It's great. Go for it. Don't have to call us to do it. It doesn't cost us any more money. Once we set it up, it's set up the end. And then you're going to go through all your goals like this. And then once you hit a goal, you're going to start another. Back to that client with 14 goals. Once we hit one of those four that we've decided to really help if you do regular monthly work or you're in-house, sometimes you have project inertia.  This is boring. I'm sick of it. Start on a new goal, right? Once you hit a goal, you start on another. And this will also help for agencies who do monthly because it will make you feel like your client. You're doing stuff every month for your client, which is awesome. New things happening and new goals are being spent.  But we still have a problem when it comes to goals. You're like, Dana, you just taught me how to do goals and they're telling me it's bad. The problem is that, and this is an organizational problem, is that we measure what's important to the company, not to the actual customer, which brings us to reporting on things that in the end don't actually matter to your customers, which the people you're actually marketing to or selling to. I know we just went through metrics, but let's pick better metrics to work on. And this is not something that you're going to necessarily be able to do tomorrow. This is something that's a longer-term shift that I would like you to start working on in your organizations. Because the problem is that you can end up in a situation where you have a report and entirely reports on metrics that matter from an organizational point of view and not the customer's point of view. So I read a great example of someone who had a Quick Links menu on their website and they reported that they were like, wow, this Quick Links area is really popular. What's the people clicking on the Quick Links? Well, they clicked on it because their site search sucked. So if you just reported on the wrong things, you said you should report on whether or not people found the thing they wanted, which the answer was no, and they ended up getting rid of Quick Links and then fixing the search instead. That's a better metric to report on. So you end up with a metrics like, how many people contacted us? Did we solve their problem? What was the bounce rate? Did people find what they needed and left because our site was helpful and that's okay? Or how long did they spend on the page? Did they actually read the content and find it useful that was there on that page? And one of our major problems is that we aren't focused on helping our customers solve their problems as fast as possible. We focus on things like, oh, we don't want any pogo sticking happening where they click on a result and then go back again. Maybe your result was good. We get mad at Google for scraping our data and I mean like, okay, fine. It's not great, but it's Google and they can basically do what they want, unfortunately.  But I mean, the reason why Google is doing this stuff, the reason why Google is doing snippets is because our websites are terrible and therefore Google decided they can do it better. So we did this to ourselves. And that is why Google said, well, how can we make it easier for people? How can we make it so that we're solving problems as fast as possible? Snippets. So have you asked your customers what their problems are? In general, organizations put themselves first. But that's the old way of thinking. I would like all of us to start becoming customer centric. So for example, Twitter, hey, can we have an edit button? You know, at Twitter, how about a heartwarming video? The world asks Twitter, hey, can you get rid of the Nazis? And Twitter says, do you want to turn any characters now? This is not solving a customer problem. Twitter clearly is not listening when we tell them things, right? So this shift is not just desirable. It's important if you're going to stay relevant in business. We have to keep up with the changing ways our customers want to shop and engage with us and interact with us. We can't do this if we don't make them the center of all of our decisions and all of our goals that we report on. Because success is not measured in campaign cycles. The six-month campaign, the quarterly campaign, the however long you're going to talk about a campaign, is an entirely artificial construct that we've placed on marketing.  And if we've spent enough budget for Q1, right? They do not. It is not of interest to them. But in our case, we really need to keep asking why. Why did the customer want this? Why are they using the site this way? Can we solve their problems faster? For example, it's about the experience the customer wants to have, not the experience that you want them to have. Google Maps does not decide, oh, I want directions to the airport. No way I'm going to take you past the space needle because the space needle paid us for it, right? That would be a bad experience. But imagine if they did take that money and Google, please don't do that. That's not the kind of thing where people would say, oh, but we want you to see this and this and this before I let you go. Just let them, this is what the customer wants. Take yourself out of this. You don't know better. The customer actually knows what they want. So if we're thinking about how we're going to measure this in terms of conversions, the old way is measuring conversion rate on a form filler or purchase. The new way is measuring how fast an order arrives. And you can do this right now. This is an easy thing to measure. You can see perfectly post-lasic. How do you measure this? Well, you monitor social because they'll probably talk about it unless people have their stuff public when they shouldn't. The new way, whoops, is when their father likes and respects the new home care nurse that you sent out. How do you measure this? Reviews, right? All of these things can be measured. And this is how you make a company think about their customers first instead of metrics first. So let's call these customer conversions. And I'm not a fan of the word conversions. It's such an inside baseball kind of word. But we need a bridge from the old to the new. So I'll call them customer conversions for now. The old way are organizational conversions. The new way are customer conversions. So how are you going to measure this stuff, right? I talked about it a little bit, but, for example, you can measure things like the number of featured snippets. And that could answer the, can someone find their answer quickly? How many snippets does the organization have? And can we provide a resource for someone's question? Could someone answer all the questions they have about our product that are actually going to our website? And you can use stats or analytics to measure this. They're just out there. You can go talk to them. They did not pay me to say that. Did we, was a post, did a post spark a conversation on social media? Are people telling their stories? Are they building a community? You can hook this up to rival IQ, which is my social reporting tool of choice. And then you can look at engagement rate. And rival IQ also has a nice data studio connector now. And then you can take all the likes and hearts and ha ha's and all that other stuff from Facebook and figure out math wise if there's a sentiment behind it. Like this post made people sad. This post made people happy, for example. That can all be done with existing tools that we have. And this one is a problem that we've been working with for quite some time, which is what your content consumed. And this is a new metric I really want to introduce you to today. The old way of measuring content would be all the pages, three minutes, we're going to fire an event at the end. Well, some content is really short of a good metric. And people hoard their tabs. You might leave a tab open for weeks, right? Yeah, I hear you. I know who you are.  You probably just leave your inbox all in red too, don't you? Thousands. Disgusting, right? What does your house look like? All right. All right, what do you need? So how do you combine these metrics to get an actual read on whether or not somebody read your content? So how long would it take to read this content? It's the first question we need to ask. So it's, you know, X number of words. That would take five minutes to read. This other post is a lot more words. It would take ten minutes to read, right? Reading time. We can calculate this. This is the thing you see in all sorts of websites now that warn you that you're about to get into something serious when it's a 20-minute read. That's there. And the second question is, when does the content end? When is the end point of this particular piece? So did both these things happen? If they did, the content was consumed. If they didn't, you either had people who abandoned ship, they skimmed, or they hoarded the tab for later, because they're bad people. So I don't have time, obviously, to go into all the nuances of this, but we have released a Google Tag Manager recipe that you can put on your website, and it will measure through custom metrics if people spent long enough on the page to read it. And the way we do this is we actually look at how long it's going to take to read the piece of content you define, and then send this as a metric. And then if they scroll to the bottom of the area you define, we send this to analytics as a custom metric. And then if both those things happen, we send it to analytics as a custom metric. We've been testing it out on our own site. It works. We've been testing it for, I think, about six months now. So please go download it. Let me know what you think. I hope you really enjoy it. So thank you. I'm not done yet, but thanks. So now what? Okay, so the first thing you're going to do is you're going to share your gold charter. Remember back when we were talking about the gold charter where you got all excited about measuring content? Okay, so you're going to take that, and you're going to share it throughout your entire organization. Because marketing is a profession that's very public, right? Everyone has an opinion on marketing. Oh, my roofer said you should do this, and my son, he did websites, and he should do, right? Like, we all have this thing happening. Can I get this in cornflower blue? Like, these are all things that people ask us all the time. So by sharing the gold charter, you're able to sort of head this stuff off at the pass because it's a really clear path on how you're going to get from the current state of suckage to awesomeness.  This is obviously bad. And then it just... Now we have the review strategy communicated to everybody. But this is an important part of the gold charter. Because the difference between a successful company and a mediocre company is that in a successful company, everyone is rowing in the same direction. Everyone has the same goals in mind every single day. And if you don't communicate that, then how is everybody going to know that you're all rowing in the same direction? This is a very Canadian slide, I understand. So let's remember why we're doing this in the first place before we all leave here all inspired. People do not switch agencies because they're happy. I mean, this sounds obvious, but I think a lot of people forget. And this is something when we work with a new client and see the team is looking at the website, we're like, God, this is terrible. It's like, if it was good, they would not have hired us. Let's remember that.  These results are great, or these results are the best results we've ever had. It's totally fire this agency. That never happens. So obviously people leave us because we're not doing as good a job as we can. And what happens is that people make changes because they don't understand. They don't understand that what you're doing is meant to get them to where they need to go because we haven't communicated that to them. And what's worse, the organization might not even have a unified understanding of where they want to go in the first place.    Then you end up in the blame game. This is a terrible game. Nobody likes playing this game. You know, you can point fingers all day, but when it comes down to it, no one agreed to report on when the relationship was first established. No one knew where you were going. So how are we supposed to know when we got there? No one... This is a terrible game. Do not play the blame game anymore. In the end, we're all trying to help people do better. So when you tie goals back to real results, you show the monitoring that you do as part of this full picture. You communicate exactly why you're reporting to your entire organization. Everyone's on board. Everyone knows which way the boat's going. They're excited to see the results come in. They can commiserate with you if something did not go the way it was planned as opposed to blaming you for sucking. Sometimes it's just not your fault. Sometimes it was just bad creative and you didn't see that coming. That happens. And we're all trying to help people do better and be better and do better marketing. So when you get back home, I really want you to figure out what better means for you, how you're going to get there and what that means for your reports. Thank you.