The Holocene peat model, HPM, is a dynamic simulation model designed to study the interactions between peatland carbon and water cycling over decades to millennia. It accounts for the effects of peat depth, water table depth, and vegetation community composition on peat accumulation, which is determined by the balance between above and below ground productivity in litter slash peat decomposition. The model also takes into consideration the effects of peat bulk density on the water balance, which influences the growth rate of the peat column and the capacity to shed water. The model's outputs include time series of annual carbon and water fluxes, peat height, and water table depth, as well as a final peat profile that can be cored and compared to field observations of peat age and macrofossil composition. This article was authored by S. Froking and T. Rulet, E. Tuidola, and others.


I have to make pictures, yeah. Are we done? Monique, we need you. You can touch it. Behind something. More and more developers, good. Okay, so in this session we keep everything very informal. So this rule is simply this. You have a question, I'll give you the microphone, and then one of those people will answer the question. But before we do that, I want to go very quickly past everyone that they can say who they are, the name, what they did for Blender, or what they're working on. Oh, hi. Yeah, I'm Lukas. I've been working on Blender like two years ago, I think. Yeah.  Hi, I'm Monique. I'm working on the sample-based compositing. Does anyone know what I wish? No. Where is it? Is it public, finished? No, not finished. Where is it finished? Oh boy, it's going to take a while. In the future, definitely in the future. Hi, I'm Mike Erwin. I work on OpenGL and system compatibility. At least working now for Epic. And you are working for? And I work for Epic Games. I am Bastien Montagne. I'm working on Blender development mostly on the asset management, idea of a rise, idea management in general these days. And I'm working for the Blender foundation. And we fix this like half of the bugs in the bug tracker. It keeps Blender stable. I work for Blender Animation Studio. And what is Blender Animation Studio doing nowadays? I don't know, I think that's a question to you, but I do everything. We have no idea what they're doing. Hi, I'm Brecht van Lommel and I mostly work on cycles nowadays and some other parts of Blender. I'm Stefan Werner and I'm working for Tangent Animation, mostly making cycles faster, use less memory and not crash. Yeah, I'm Pascal Schoen. I'm working for Aridas and I implemented the principle shader in cycles. Hi, I'm Lukas Dockner. I'm doing cycles development stuff. I'm working for theory studios now and some stuff that probably some people may know as the denoiser and I'm also doing UDEM network rendering, stuff like that. When do we get UDEM? You can get it, it's public, but... The patch has been submitted for review? Not yet. I'll do it once I'm home. You do it when? When I'm home. I'm home. So that's like tomorrow. Hello, Tuesday. Okay. You have to keep them sharp. Hi, I'm Sibren and I'm currently working on Alembic. Ah, keep working. Keep working. Hi, I'm Joey. I work on VR stuff in Blender at the moment. Probably going to do some GL stuff as well for 2.8, maybe some Vulcan, but I'm not going to talk about that. Hi, my name is Jeroen and together with Monique, I'm working on the sample-based compositor. Hi, I'm Kier. I work on the motion tracker for Blender. Hi, my name is Sebastian and I work on fluid simulations. Oh, come on, come on. You have been working on Blender. Yeah, I have been. I'm Andrea and I've been working for Blender 2.5, the last time where I did the big things. A lot of work, a lot of work. The file browser, file system, a little stuff. And now I'm looking for an area that's a little bit abandoned, maybe, and come back. The sequencer. The Blender sequencer. That's really abandoned. Nobody does it. Maybe. We'll see. Okay. I do too. Hello, I'm Inesh and I was working on Blender a bit of time ago. Now not so much anymore as I got employed somewhere else. But I hope to get a bit of free time again and go back more into the game engine simulation parts and their activity. I see you're an expert now in game logic and how to design game logic. That's what I heard. So pick the brain and make sure that all the design comes out. No, I know how difficult it is. Okay. Thank you. Okay, I think I had all the developers. You can manage this. So now, guys, girls, what's the first question? What do you want to know? You know everything already. Come on. Feature requests. Everything is fine. You were first. Logging. I'd like there to be more logging and I'm curious to know whether it appears in anybody's plans. In particular, I get very frustrated that Blender's messages from script just pop up this thing that say, go look at a console if you have one. And I end up having to go back and restart Blender this time from the command line so I can start seeing output better. And I'd like someone to talk about logging. Logging. Logging. Come on. It's already exist. Well, it does exist, but it could be made better, but we don't currently have any resources to work on that. It's a nice project for a beginner interface designer, blah, blah, blah. Anyone just around to implement? What do you mean, not for rendering, but in general? For all the interaction you do in the software? Or you mean rendering, logging? It's like Python trunk errors and they say, hey, please check the console, but the issue is that on Windows you can toggle a console, but on Linux on OS 6 you cannot toggle console unless you've started Blender from the console. So it's kind of... So we can kind of hack something with overriding the STD out and STDR and piping them to the space in four things and print errors there, but there are interfaces possible here. I don't know, I hope someone from this audience, they see it's not a rocket science and they can totally help here and get someone else involved in development. Is that not something for Campbell? What is the Python department? It's not directly Python, it's like some C and some interface. It's mainly a question of how to present this in an interface, I think. It's on the radar now. It has been locked on video, it has been streamed, so we will keep track of it, okay? What are the very handshakes? What's the chances of getting Natron and Blender to share code or concepts in terms on the compositing side of things? Because both have benefits. I don't know, I know we talked to some Natron devs, but June... What do you think? Yes, we talked to the Natron devs, and it's also open source, so we looked at the code. They use a totally different system and based on that, it's not easy to integrate inside Blender as being sharing code. It's more on the technical level that they make and we make different decisions about how to do compositing. It's also a difference between a standalone compositor versus an integrated compositor we have. So it's choices. We can now make sure that Blender can export the files, a motor layer that we have to share in format for motor layer files. Is that working for anyone else? Anyone use Natron? Natron? Sure. Does it work with Blender if you save our motor layer? Can you read the motor layer? No. Some can, I didn't. I saw, maybe, no. He used Natron, that's right. But I mean the biggest problem with Natron are the two problems. Of course, one is the clone. It's meant to be a unique copy, right? That's good. It's really awesome if somebody does it. But that's what the design concept is. They should keep doing that. But that's a goal in itself. And the second is they support a plugin as an open VFX. Open FX plugins. That's a library of plugins which is allowed to be commercial and closed. For example, the first Natron demo I had from the maker, I said, okay, let me click around. So I'm going to add a picture and I want to blur it. And then he said, oh, sorry, I don't have blur yet. So you have to install a commercial blur plugin. And I applied it. What happened, I got a watermark on top of my picture. And then if you want to get rid of the watermark, you have to pay the code of the blur plugin, the license fee to get your blur working. And I can understand why, but that's really not my thing. That's not how I like to work. So Natron has been evolving now better, but they still use open FX plugins. And that's an infrastructure that you cannot combine with Blender. Because in Blender, you cannot have that kind of plugin. You want to talk? Yeah, yeah, yeah. You're wrong, but maybe their design decisions are more correct than ours.  Used to be. Well, it depends on what you think that is better. Because also the open FX format is really in very old format. And this is also not really up to speed with modern hardware. So although the compositing code might not be shared, the motion tracking code is shared. They are using the Blender code. And they work with them to get it integrated. So Natron is using the same tracker. Ah, our problem. Hello. Is it possible to render Alpha Channel behind Glass Shader material? Is it possible to render the Alpha Channel behind Glass Shader material? I did look into that a while back. There were some requests like being able to composite refraction on top of something. The problem is, theoretically it's doable, but really because you have the problem if you have a refracting object in front of a background, you would want the background to be refracted. And for that you would need a separate render pass that stores the outgoing coordinates and everything. And it's not as easy as it sounds. It could possibly be added, but it would not be as useful as you might think. Well, I think, I mean, it's indeed true that you cannot then refract the background properly, but I think in a lot of cases it's okay to ignore it and just assume it. It's basically to treat it as if it's transparent. Indeed, if you have a window, the refraction is very little because you have just too late, you know, the refraction tool window is very little. So it's certainly something I think we can implement. But is it now using some kind of fake Alpha or is it not at all? Well, I mean, you can kind of do it manually, right? Right now you can have a refraction or transparent, and if you replace your refraction with a transparent BSDF, you can manually make like a glass shader that kind of does this. Indeed, they're not... I mean, in the internal, we had Alpha for glass, lots of quotes. But it was all fake, right?  You look at, okay, this is, I think, the amount of transparency and we call that an Alpha number. A little bit. Cycles is very more complicated because cycles is about light transport and stuff. So the transparency is not a value you can take out of the picture and then composite it in another picture. Simple. Well, I mean, I know how to do it in cycles to make it work. I mean, I've implemented in Arnold, so I mean... How would you do that then? How? No?  If you hit the background, then you'd make Alpha and if you had another object, you don't... I mean, it's... Okay, you would never look as if it was rendered in one pass. No, of course not because there will be no refraction, but if it's glass, then there's not a lot of refraction anyway. I mean, so it's...  I don't know when, but... In the bottom, what?  The more you talk about it, the higher it will go. So what is on the top of your list? I don't know. That's... I mean, it's not a public list, you know. It's... You will surprise us. Anyone from this side? I guess my question has to do with rigging. Here you can answer that. But is there a way in the future to have more control over the deformations produced by bendy bones, you know? Maybe like weight-painted deformations to get just the result you want? Bendy bones. Yeah, bendy bones. I mean... I think animation right now is more in just a kind of area of work. So, but yes, it shouldn't be that hard to implement. It's always the same question. You need developer power to do it, so... It goes to the bottom of the list, I guess. It goes to the bottom of the list, I guess. It goes to the bottom of the list, I guess. That isn't a rider, yes. I think Joshua is totally up to look into bending bones improvements because he recently had quite some fixes and little improvements in there, so... Maybe Joshua is looking... Joshua is currently a developer from New Zealand. He's also working on a development grant from the Blender Foundation. And we would try to keep him on board for the coming period or sort of work on animation features like the bones and other things. But this is in ongoing, so I don't know what the result will be. You? Hi, I'm a bit of a cat background. I was wondering if there is IGS, Step3DM import possibility, maybe with a plugin? .3DM Rhino. Is that the 3DM or not? Yeah, yeah, that's the 3DM. I know only... I know only Rhino. So, yeah, I'm also using Rhino, so I know this file format because at RDS we use this extensively and I haven't really thought about doing some importer for this, but maybe this can be done. I have to have a look into how Rhino is storing the data, how 3DM looks like, and maybe this can be done. Maybe at first there's a plugin. Is it a text format 3DM? Yeah, I think it's a text format. But I think, yeah, I have to look into this because it's normally for NURBS curves and not for meshes. And yeah, they added meshes to it, but yeah, have to have a look into this. What I understand is Rhino is more or less like parametric surfaces and everything which we cannot represent in blender natively, so it's going to be some limitations anyway. No matter how much useful it is. Everybody's using subdivision surfaces now, right? What do you need NURBS for? No? And who needs NURBS? Can I see people who use NURBS? Wow, what? You? Oh. Or the people who use Rhino for modeling. Rhino will use this. But there's no way you can export from Rhino to blender. Do Rhino has OBJ or something? Yes, always converting first to meshes, yes. But then you lose all the potentials of having NURBS. So, yeah. You can export in OBJ or FBX, whatever. But, yeah. An implementation could be nice. I mean, the NURBS code from blender is at least 25 years old. At least. 30, 35. So, when I was a little kid I wrote the NURBS code. I thought, well, it was very modern then in 92 and stuff and everybody was doing NURBS. But nobody really picked it up. We had a couple of people who tried. But NURBS is very technical. It's mathematical surfaces and stuff. But NURBS should also be presented in a way that you can use it as a modeler. It's fun. Rhino is doing a good job to make it accessible for artists. And that's not a trivial thing. There's a lot of really cool tools you have for trimming and cutting and melding and blending and those kind of things. That makes NURBS editing cool. But we don't have all those things in blender. If you could export the raw NURBS blender can't do anything with it because we don't have all the support. So you have to stick to converting it to messes. Or we call someone from that side of the camera to help us improve NURBS. What we could do is the taste plan one day. Taste plan, yes. Is there any taste plan that we have here? Who knows what the taste plan is? No? Also plug-in for Rhino. It's about passing seamlessly through NURBS model to a mesh model modeling in SUBD and then bringing it back. The taste plan is like something halfway between mesh modeling SUBD and NURBS but it has been patented and locked in by an evil corporation from the United States. So we don't know about that of course but the concept is not that difficult. So the whole idea of making NURBS editing more accessible is something we could work on and blend it. I don't know where I was. Just on the NURBS thing there's this awesome program called Moment of Inspiration I think it is, MOI. I think it's the guy who worked on Rhino. He made a NURBS modeler aimed at artists. It's incredible. So I suggest maybe you buy them out. That'll do it. I can't collect the money from me. Crowdfund. Hello, my name is Serge. I have actually three questions. I'm sorry. Oh yeah? Okay. So this is how it works. Okay. The most important one. The control C is sometimes really slow. I mean for me to crash blender and reassign the file then to use control C sometimes. So yeah. What are you using? What do you mean? What are you doing to make control C that slow?  No, but we use lots of maps and terrains and visualization for architecture and every time control C is crazy slow. This is an undo system and basically when you undo it's like opening the file from scratch. So what happens is all the object needs to be evaluated from scratch. There was all plans to keep all those objects in the undo state and then we can save undo much quicker. But on another hand the question is do you have a multi-core machine and did you check if all the cores are 100% busy because if they know then we can optimize something out? Currently we have 2.8 mites and things because of collections and things. Currently if you want to optimize your work think of using group duplicators linking in your data from other streams. But if you have like a pictorial and you don't work on it anymore link it in because all your undo will then skip everything you link in. So your undo then is restricted to what you have in the local stream. So you can create really complicated things and if you link it in it is not part of the undo system anymore and then you can keep working very fast. That's a good tip. So my suggestion would be actually if you can come up with a scenario with instructions that actually reproduce it reliably that then you file a bug with it that will be very helpful. So if you can make it slow on my machine while I'm using it then I can fix it. That makes it a lot easier. So if you have some sequence of operations that you do and then you press control C and then it's very slow that makes it easy for a developer to fix it. Since if there are more than four steps including the .land file yes much more handy. The art of bug reporting that's what they can talk about forever. So if you talk after the meeting you can ask them hey how do I make a good bug report and they know exactly how to do it. But the key thing is always that you don't consider it to be a support thing and start like I have a problem which is help me now you have to transform the problem into somebody else's computer. I have to make sure that the developer gets the problem. So he should be able to redo it and recreate it and then experience it and then you can get it fixed. But if the problem is only on your system but you have no idea how or what then you need help or you have to find out what's going on and then you can get fixed in .land Suggestion for this thing in particular when you are clicking report a bag it will take you to the developer's website to report a bag so that could be the first line the extraction I don't know if they are there could be yeah they are a video from I've reported twice but I'm doing the thing without reading because I was able to successful but for our form yeah the form okay also a question I'm doing particle system with the linked dupli groups so only the empty are showing in the particle system when I'm doing particles with linked trees, linked things and the dupli group only see the empty it will be in the 2.8 maybe with overrides it will be feasible to particle that rendering groups with the dupli groups linked from a tree library yeah only shows the empty and if I'm appending the trees I'm fine with that I'm using the local groups this is for Sergei question do you want to answer this is just a dependency correct a simple dot blend file and I'm looking to this but is it there is already a bug on the website but it's kind of a dependency or a recurrency or something stuck so it will be maybe after the session talk to him but the duplicating system on particle system will be reworked for the new dependency graph and for the new render engines and ownership model so there's definitely something we can look into and see what we can fix later on the full particle system will be redone from scratch anyway okay so do you have any plans to improve the texture so we are able for example to paint in the separate channels in the red channel in the green channel something like that we can do in substance painter or if you use substance then you have other opinion no well we had the g-slot this year but it was more on vertex painting not on texture painting similar code to texture painting to vertex painting so so we know that it might be improved but we don't currently have any priority on those improved because it's kind of good enough and we have much more important broken stuff and better to take care about but it's also not something it's also not rocket science and someone can just also help what is rocket science then I don't know rocket building rockets is oh aha I see so when are we going to make rockets as blender community we could finally do rocket science yes and then we put blender on the moon on Mars but seriously I know Campbell he worked on the texture painter mostly there is also a developer who is making an add-on so you see was he on the conference Andreas no I don't think but Andreas is doing fantastic it's really good it's a bee painter I think it's called you know it the add-on so he's really smart in thinking about the features that are useful for an artist to have so he's doing all the research and design and we are in contact with him and I connected him with Campbell already and the kind of stuff is fun for him it's not that much work so when we have 2.8 a little bit more on track Campbell is also working on new eye stuff and workflow that kind of topics but as part of the whole process next year I'm sure he will pick up stuff from Andreas and make sure that the painting and blender is upgraded not only that I'm also looking for people to really look at a better texture but procedural textures combining them with layers and notes make sure that we don't have to have a plug-in with substance designer but we don't need it that's the philosophy of the blender project sorry guys but that kind of workflow is fantastic and the blender, our old blender material system was actually almost something like that but then with a really bad interface so we have to make sure we have good real-time rendering we have to get a good interface with good layer definitions and note systems to manipulate your procedural coordinates and then we have a texture system in blender it's really not rocket science right so who's going to code that who thinks that's interesting to code it's a trick question ah you think it's interesting when you're done with Alembic you have other stuff to do I'm looking for people to work on this and there's one of the things we can do in the 2.8 period it's a question about a new collection of the 2.8 will they be linkable like groups and it will be usable in rigging for linking rigging well I mean it's more Dalai stuff currently but he's working on type of collection which is actually going to replace groups so you can choose a collection and decide to make it a group group collection and then that would be linkable and yeah yeah I mean it's supposed to replace groups in the end so wasn't it a collection links to a group yeah but proxy we nuke them proxy we replace with ID overrides it's a new system which will be more robust and hopefully much less hacky so we can maintain it easily and make it more powerful more easily to extend all too we want to get rid of proxies hello I have a question considering masking and rotoscoping I was using this for masking video and my problem was that when I applied a feathering that the key frame stored all the information and when I altered it the feathering was all over the place somehow the key frame for one mask handle all the information and including the feathering and when you want to alter this one handle it brings it all what is the normal feathering for I don't know maybe I was doing something wrong but what I would like to have and I didn't see it as an extra key frame for the feathering yeah that would be great I think key frame is currently stored per control vertex of the the spline itself and then everything else is kind of interpolated based on the distance but it shouldn't be popping up that much so if you have such a case please do a report and assign to me and I'll try to see what we can do from Blender 2 to support it better but at least there's one person still using plastic masking code one person in the world oh there are two two people using it three people using it and more people using masking in Blender so we have to keep making sure that it keeps working all the time it makes the render arrows go away it's really good is there anyone still using Blender internal for everything just to ask are you very unhappy that we are kicking it out or do you even know that we are kicking it out yeah I'm unhappy yeah I'm unhappy so we still of course it's not out yet the code is still there right when is it going to be actively kicked I can kick out it tomorrow yeah you can you can't do anything you want but will you okay challenge accepted so this around lunchtime I was meeting upstairs about the game managing and people said you said last year that we are going to kick out the whole game managing I said I didn't say that we're going to work on have a better game managing and the same thing with Blender internal that we improve real time rendering and what the PV is doing is showing that you can now get better quality rendering in real time than what you can get with Blender internal for the game managing we want to have a similar thing people see now what's possible and then you have to get a tool that allows you to make interactive content in Blender that's the game managing but it should be well integrated so yes we will keep working on that oh yeah but in 2.79 you will have Blender internal forever alright forever and ever so in the first version in 2.8 I don't think the old material system is still there then you have to notify everything I typically use quite a lot of objects in the scene let's say like 2000 and these objects are typically instanced so let's say I have 50 or 100 unique objects each of these objects may be instanced and even I have LODs to run faster but I still let's say I have 2000 or 3000 of them in a scene it may run like lower FPS and I'm wondering if these or the implementation of new viewport in 2.8 will bring some speed up to render much more objects I'm not sure that's exactly the problem but so there are multiple problems when you have multiple objects the dependency graph would not be happy when you have multiple objects but it's only during construction time adding an object could also be an issue because it does checks with all the other objects to compare the uniqueness with the name for the drawing we implemented new botched rendering which prepares all the instructions and it just real easily instance it's all over the viewport but that's something that Mike can explain better how it works and limitations yeah it should definitely if they're like exact copies you're just instancing these objects the new draw system should handle that pretty well and getting into different things like LODs I'm not so sure about that but basically if you have one object instanced 10 times in the scene like the CPU side computation of that once and then it can draw that multiple times and it'll be it should be much better than 2.7 if Blender internal will be replaced by EV will we be able to render in background we are looking into that problem because there are some limitations of OpenGL which on Linux requires having X11 session but we some other stuff like EGL we can avoid this or some other tricks but this is on a list to look into it is on the list to look into this is actually the same issue currently having the OpenGL playblast there are lots of requests to be able to do those in the background and it's going to be solved sooner than later well even better you could also have a little render farm with like 3 or 4 graphics cards you have one second render time per frame or something that's work got something done exactly I wanted to know about weighted normals for low poly models game models what's the situation because I know there's a Google summer of code I don't know if it's merge if everything is merge and if there's anything that the core developers need to do before things like mirror modifier not working with weighted normals as it was as an add-on is there any development going on and that does that summer of code project is being merged completely or has problems we need to review the patch I mean the guy submitted the patch like one week ago or something like that so the developer so I just need to review it first and then probably get a second pair of eyes on it and it should go in most hopefully like next months or something like that yeah I mean globally it's okay probably some details to be checked but globally it's okay we're not going to release 2.8 soon anyway so there's time right so when are we going to release 2.8 when are we ready hi we're I think all really excited about EV and kind of excited to try that out on the MAC side but I think from what I can tell EV is still broken on the MAC side and I was just wondering how that's going and also wanted to say thank you for the shadow catcher I got an idea that Clermont was not here Clermont is hiding yeah I think it was broken like 2 days ago and then it was fixed but I mean it might depend on specific graphics car combination but in general it shouldn't be broken so if it's broken reported but yeah it might just break once in a while while things are developing we don't accept bug reports on 1.28 I'm wondering if there's any way to stream EV into a window in my application that I'm developing so that I just have it as a display using kind of blender as a packing engine maybe it's an open GL window in my app that it is sending the display to I don't know just make a bug report I would guess and at least some screenshots or so I think for example what Nimble Collective is doing they have a remote desktop client and they stream all the graphics oh that thing no this kind of stuff is a lot of development is going on making virtual workstations and make sure that all your open GL drawing can be streamed we have a wrap to other clients and stuff and that's not depending on blender it's just an external program that makes sure your drawing gets streamed somehow to another application I don't know details about that it just kind of depends how you exactly do this for example as the excellence the wording through SSH will totally work for this if you use VNC it will also work if you use remote desktop then Windows forces you to use a software rasterizer which is no longer compatible with the requirement of open GL we need in blender and this is something we cannot solve from outside because blender just needs open GL to work properly so but there are ways around this and they are all outside of blender there's so much we can do about this wasn't X11 was also designed to do this X11 is designed to totally forward X11 through SSH tunnel it's not an issue it's just X capital for the SSH command and blender and it just runs so this question is specifically to Lukas I think first of all thank you for your work on video noiser you literally saved my life like a month ago for the animation project so yesterday you had a great talk but I think you didn't mention the denoiser at all for multi-frame animation denoising stuff so what's the state on that the state is I started working on it yesterday and it's coming the problem is it already was working in the google sum of code branch but the problem of the multi-frame denoising is you want to use neighboring frames of course you also want to use the frames that are about to be rendered and in blender the render pipeline works the frames are rendered it's composited saved it completely throws it away and does the next one so there's no way to do it as part of like click animation and it just denoises everything so what I'm working on right now is you would enable a denoising pass that will be in the pass options you render the animation to EXR then you press a special button that denoises the entire animation and then you press another button that runs the compositor on top of the denoised animation so the workflow is pretty bad which is why I decided to just drop it for the first release but I don't see any better way to do it as far as I know Pixar render man does the same so I guess if they couldn't come up with a better approach this is also a pipeline thing where the compositor also has to work with because people who use the compositor also want to have like a whole shot renderer and then have all the raw files around on a fast SSD and then all the EXRs in the composite have a preview on a whole shot and do all that stuff then you can do some denoising, whatever, compositing but denoising also works better if you know what color space you have and blue and that kind of stuff there would actually be pretty interesting for the compositor because it would mean that you can change your compositing setup and recomposite everything without re-rendering yeah so I guess it's not too complex to do, I have to look into it but first the denoising part should be done pretty soon I guess it's not rocket science, huh? Hi, I have a question from Twitter coming in so one common workflow is to have one screen for cycles rendering and one other for different draw modes how would that work in 2.8 since the render they did extend limit on the message by factor of 2 like recently so it might very well be not 140, but the idea is you would probably use two workspaces for that or Brecht can know better ideas I think by screen he means well he meant like 3D viewport because not what Blender would call a screen I suppose it probably means the window so on the left you have the wireframe so personally I think it's very important to be able to have like one 3D view in like solid mode and another in cycles rendering mode and so I'll make that work one way or another that's good news I think make it work according to design yes I mean the decisions are in their hands of course in their good hands but I think the strategy more is let's first finish everything we already know because I also got stuck almost in talking about how much we do for 2.8 and it's a lot, it's really big, it's very complicated so first we have to prove that everything we already did and what we are working on is working sign it off get a really good interface make it really usable and then you can move on and say okay so now we have all those engines and all those insane drawing modes how are we going to manage that more efficiently we need users and designers and people to give us feedback for those configurations besides the quiet Hi, so more question about the way to interact with Blender and with that brilliant Python interface I think this is really great because it allows on the one hand to go and look what's going on inside on the data but also to control things now there are two ways to do that Python one is through just the console the Python console and you type things and that works fine but then you can also have this text window there where you can type a small script and let that thing do things but it seems to me like that Python from the text window once you run it and it's done, it has displayed things you have no way to find out in the regular Python console what it just did it seems like it's two different Python worlds so to speak and at times it can be annoying because you can do things complicated through a script there and then you'd like to go and hack into it but you can't you have to see the text window the Python code which is inside it it's exactly as if you were executing a Python file so it's exactly the difference when you have the basic Python executor if you are launching the interactive command line or if you are just passing it a Python file to execute and then it executes this Python file and then it's just quit it's exactly the same thing so usually you use the text editor to do something similar to addons or UI it means you create something which will be kind of persistent inside Blender while the Python console is more like very quick test or very quick hack to the data structure but something which is not really you would call the Python operator inside the text editor and not inside the Python console usually another approach you could use for this is if you call the text block something.py then in the console they import something and that module will reflect whatever you have in the text editor so if you have your code in the text editor as a function you can just call it from the console and then hack on whatever that function returns there isn't a plan to improve the Outliner yes I mean the Outliner needs a lot of work and a lot of improvement the only problem is manpower to do it but I will need to do it for asset management we ultimately will have to have some control of the assets in the Outliner but I would first like to kind of rewrite it from scratch nearly because currently the way he's handling its internal events, internal operations and everything is very weird, very complicated and very non-conventional compared to the other parts of Blender so it needs kind of a bit of work and not very firm of adding features in current implementation so in the dark corners of internet I saw the planning from Dalai Filinto and he had improved an Outliner plan in the coming weeks because we need to improve Outliner for collections anyway so you should work with him together but we're definitely going to improve Outliner to support all the new features we're going to introduce at quite age just stay tuned so recently like this year I'm a software engineer so I've discovered oh you've got a Python API that's great and I started writing operators but I discovered you couldn't add modifiers and I wondered if there was sort of a conscious design decision behind that or or anything sorry I didn't quite catch the question so you can write operators in Python the modifiers? No you can't because the problem is that modifier is handling huge amount of data through meshes so and it's like run every time you change the setting every time you evaluate it so it's just not possible to do it you know I thought Gambo was looking at how to do mesh manipulation faster to have a couple of functions and have a Python script do a massive mesh stuff No not with Python please no Python has a stupid idea of global interpreter log which will log the full execution pipeline when you have multiple orders using Python modifier then you all of a sudden fell back into single-threaded evaluation obviously so please no So then what? Then I don't know Ah she codes Python go whatever you name it Open shading language? Also that's a general point we're actually just discussing this outside we often get requests for C++ API, modifier API, custom closures and cycles Blender is open source so that's important for commercial software because for example if you're using Maya and you want a modifier you can either wait for them to edit or you can edit just Blender you can just implement it right inside the software and that actually works pretty well so That's also what I always say but I know there's an extra reason why people want it it's just business right so if you make a plugin and you can sell it and have an infrastructure of a marketplace for things to do on the other hand we still will work on our modifier design to have modifier nodes for example the node system to work with modifiers or node system for constraints and rigging and at some moment we will have to look at a way to insert Python in this even when you don't like it No I know the big studio has the same exact idea of forbidding Python from this that's what I fully support and at the moment if it's odd you can put everything to someone else and do something more fun but if you have a node API you can find a way to have things plug over Yes but not with Python like someone puts closer to OSL what is just getting just in time compiled to native code that would be the proper approach OSL I think you would look at for this Kind of maybe not one to one but it has better potential How do you hear this It's on the last so yes we are making an open source project so you can basically implement any modifier you want but I think if you have modifier nodes you can infallibrate that how much need that is for custom things Same is for the compositor you have a list of compositor nodes and you think ah shit I would like to have this one or that one or that option or tweak something but always or easy something we can look at In my addon I use quite a lot of external libraries let's say like OpenCV and I want my user to not basically care about if they have to use or download these and that library and its dependencies so I use the PIP which comes with each Python version but it's not included in the blender Python, would that be possible because right now I have to have let's say Python 3.5 installed to use that PIP-XE and download the library with all dependencies and install it I missed the beginning of the question something about PIP installer executable so it's not part of the blender so I have to have Python 3.5 installed to use it to install the would be possible to have it in blender in a limited way because on Windows the default Python that you download from Python.org is a different build than our own build Python so any binary extension won't work as far as I know that changed but I understand that the question is if you could include PIP like PIP so that package management kind of thing and I remember there was some discussion on the mailing list about this recently but I don't remember the conclusion I don't think there's any specific reason why we couldn't include it but bring it up on the mailing list or look in the archives and reply it's a pretty technical issue but I think it's solvable so aside from the new outliner Windows that we're going to have on 2.8 are we going to have something like a schematic build another way to see all of our scene content but on a macro level I come from Softimash and my past days I use that software and I really love this schematic view they had because everything is so organized especially for rigging you know so aside from the outliner we're going to have another editor to see our scene every single thing we have on it from a macro level so it's like oops which we had like back in the days but I would love to have some editor work on visual law as dependency graph on all the objects and everything on whatever detail level you need because that would help a lot for personally for me and I believe it's a good tool for riggers and for riggers we can have even something more fancy where we can visualize rigs like properly aligned bones and visualize all the possibly dependencies possibly dependencies, hooks and everything it's just what I understood also for Presto and Primo the tools they have in the industry they visualize it in 3D so you see the whole character you see all the vertices and bones and everything rigging or highlighting to show you conflicts and things yes, that's definitely what sound like it's really interesting for us but it's somewhere in the list and it's not in the very bottom of that list but did you ever use to blend the old oops the thing that we had an oops we call that but it was a way to show all the relationships between data blocks and that's fun if you have five or six and seven suddenly you have like a thousand of them and then it doesn't make sense right to see a diagram that doesn't give you any information or extra control so that was not really the way to do it but we can do like from the interface you can solve this and there are ways we just need someone to do this I guess so I think at last year's conference reality I think requested from Sergei like motion curves for animation like improved ghosting is that still on your list or is it like technically impossible to manage because for animation especially like character animation it would be great to have somebody in our studio who's backing on his knees every morning and praying to the coders and please give me the motion parts what do you mean that for character animation I don't know why we don't do it why don't we do it currently with the new dependency graph there are lots of optimization disabled just because of the new way they organize stuff with the copy and write implemented that will come relatively for free and we can multi-tread the calculation was almost ideally across all the threads and you can understand because if you have a character and you have a little bit of a complicated character that takes a little while to actually frame calculated and for the motion parts you actually want to have real time complete animation calculated every frame because especially if you edit things you want the motion parts to update immediately but to update the motion parts it has to calculate the whole animation for the whole system and that's a problem in the current code and that's something that will be much more efficient in the future and that's a huge to do in that area of code how and what we need to do there and just need to how high is that on your list the motion parts it can be done quite simply after the copy and write is fully enabled in 2.10 branch and then it will enable lots of features and improvements in quite few areas like motion parts, dynamic paint and few others which I saw around when I was reworking all workarounds and everything he talks to animators every day so he will be reminded and I will remind him too you'll get it soon are we going to have a sort of freestyle real time render like OpenGL based? No 50% chance I know Tamito Kajiyama he has been contacted for this, he's looking into it to find out how to re-code it in a way that we can fit it in the new architecture I haven't read him his mails for a little while, I don't know the conclusion but I heard there was somebody else from China I suddenly recorded putting something online last week fantastic rendering of lines but so it's currently not on the we know we want it but there's so many things to do there is a freestyle there's a grease pencil project going on the whole idea of 2D animation our cartoon rendered animation is really hot we know it's in the focus but I don't know the answers I mean I think it will at least keep working at the level it is now I mean you'll be able to edit on a cycles render or on an EV render afterwards and it won't be real time but the real whether we will do real time or not is unsure but I think it's pretty sure that current functionality will at least stay but did you ever look at freestyle the code itself, how it works yes I mean wasn't it like a really weird port of Python stuff that became C code and then it started reusing the blender internal in a weird way and convoluted and it was developed outside blender first and so it uses a lot of non-blender specific stuff but yeah I mean but you use freestyle for things okay some lines but we definitely want to have a real time freestyle in the fuel port of course but we have to look at how and when the planning is unclear so a question about Python API when we ask for users data.users we get a numbers is it possible to get a pointer also you actually in one or two versions ago you won't get it from users but you can get it from I think it's data library you can request a mapping between ID and the ID it is using or whatever I don't remember the details exactly but you can check that in the API documentation I think it's data that users map yeah something I don't remember exactly the path but yeah something like that I don't know if it will be possible to use a panoramic camera in real time like the cycle one but in real time I don't know if it will be possible in the view port like in EV but with so you can render equirectangular frames in real time kind of or streaming real time frames to an Oculus or something this is actually what the HMD branch is doing as far as I know is there a Joey in the room yeah so actually for the VR implementation we do it open HMD I actually thought about creating a method to render in cycles real time on our equirectangular sphere and place the user in the middle of it so you just be able to look around as the equirectangular was wrapped around the sphere so I guess this could be implemented in EV and stuff like that as well for a real time type of simulation but it depends on what you want to do because I guess if you're using EV and VR or something like that you want to see the entire scene anyway so but ideally what you want is that the view port is simply working in every normal way but display in things like another panorama or for VR or whatever so you don't have to go to a special mode to be able to see VR that's what you are targeting at yeah right so yeah I guess this could be something that could be implemented quite easily actually I was not here since the beginning so I hope nobody asked for this question I have a question about the denoiser this is a very magical thing you just have to click on this magical switch and everything is ok but is it possible to imagine to have something less magical I mean something you can decouple from the rendering process because in fact you cannot easily blend things between the original rendering and the results of the denoising process so can we have something more like a compositing effect that is indeed possible the main reason why it is not a compositor node right now is on the one hand when I do it in combination with cycles it's easier to do GPU rendering and the other part was that it uses a lot of temporary data and if I do it right during the rendering I don't need to save it which means that you can save a lot of RAM but there already is a way if you enable the experimental mode to actually get the internal denoising passes in the compositor so it would technically be possible to implement it as a compositor node probably you wouldn't want the full algorithm that's used in cycles but something a bit less complex and a bit easier it is certainly as possible in the future I would have to look into it I'm gonna ask you directly I've noticed that when I'm emitting smoke from an object and I enable emit from texture and it puts a texture on I can never see that texture on the object is that something that's possible seems like that would be simple it should be possible but make a bug report that's not a bug that's like a feature request or something I don't know but it sounds like a bug like if you use texture but you want to see where it's going to emit from so you can like dial in if you're going to emit from like a cloud texture you can dial it in before you keep simming and watching simming, changing yeah animated right moving the depth yeah see tawn wants it too I don't know why they didn't code it it's really stupid this colders I would have done it hi I'm a silly just a question if I want to create an uber cool PBR shader which I have noticed in Unreal 4 is that possible for an EV is that on the list in OpenGL no OpenGL doesn't matter can I create an OpenGL shader in EV yes or no so you mean you write your own GLSL code or just combine some existing nodes because you can of course create a node group and the way you can with cycles or whatever but if you want to write your own GLSL code that's going to be more complicated because the way for example EV PBR built in shader works it relies on all kinds of other algorithms running we have to generate probes shadowing algorithms and all those things these are very much tight to all other parts of the pipeline so it's not as easy to plug in a different PBR shader so there has been some work on an API to plug in GLSL shaders custom ones but in general it's really difficult to be configurable hello so I also walked in partway through so this may have already been asked as well or it might be too general in which case Tom can stop me but as Blender devs you have a lot of insight into how Blender functions I assume some of you use Blender as well no? okay what are the things that you really dislike about Blender right now no no no so what's your least favorite feature of Blender the most hated thing what do you dislike about whatever so what do you really dislike? I don't know wouldn't even touch it with a stick oh you mean coach wise coach wise nobody touches the sequencer I think it's a very difficult question maybe you first have to ask them something positive like what do you like best it's not fun anyone has something you really really hate wow where do I even start at least it's on maybe cycles dependency graph notes sequencer curves outliner there's one thing that's really really good of course there's motion tracker really the only part that has been developed up to the standard of engineering you won't like to see in Blender so you're going to work on upgrading all the parts of Blender to that level excellent anyone has a hated thing particles particles particles Blender internal to be honest Blender internal the rendering engine tomorrow it will be nice this is like saying that you hate your grandmother what why you don't do that you say goodbye to her that's it okay the variable names in the code yes and also there's a little bit of formatting I think that could be improved in places say it again formatting variable names Google style meaningful variable names comments, things to papers you know what developers really hate do they hate documentation all of them accept him documentation is the most horrible thing so if there's anything everybody here you say I would like to help out or do something I mean there are all kinds of nice guys and girls they're coding working on Blender and they really like it if they get help presenting their work either if it's as a documentation as a video as tutorials, as little demo files or blend files showing off features it's a tremendously fun and useful thing to do really never hesitate just connect into a developer looking at the work he's doing and make mockups or tutorials or do whatever that is it helps a lot questions further? not a question a request what? I wanted to go back to the topic that we had directly before which is the shader pie nodes and stuff so I understood them that there would be some GLSL code that you define in the pie node and that you can simply plug in everything and stuff there was one of the original plans it needs really careful design because as Brett mentioned there is so much you can do flexible and shader because engine needs to know okay so this type of closure I need those passes for the open gel to work what we can probably do is like enable some procedural textures similar to what our cell does with some extra limitations no extra no GLSL defined closures and stuff like that some and one of the other idea was that render engine itself should be able to provide code for its nodes so we don't need to implement cycle shaders in the generic place but those designs are kind of on hold for now they are on the list they are somewhere on the mid-list me again yeah yeah this is my second item it's from the guys from our studio the animator the furious animator we have lots of camera bindings to the markers I really looked up on Google and everywhere but they didn't find a normal solution we can't import bindings between files I mean if I we use the same terrain lots of shots lots of cameras and we switch between them with markers and we can't import them between files and we can't grab markers with keyframes simultaneously which is weird and yeah so this is my second item in the code it's still called durian camera hack durian camera hack durian camera hack if they have durian camera hack all over the code don't on get microphone nobody hears you get microphone but I already said it now anyway so we wanted the most easy quick dirty hack to switch cameras and then we thought the markers are available and then we made the markers to switch cameras which is actually total abuse of a feature that was never meant to do that but then you can add features to markers unlimited right that was not the purpose but the feature of how do you edit cameras and switch them in a scene is really important I mean now for grease pencils the same thing you switch cameras all the time motion graphics for storyboarding things you want to switch cameras but I don't think we found a better solution on the market thing game logic and the viewport hmm game logic editor for camera switching the sequence is a typical thing you do with a sequence just drips and swipes and have the cameras move directly keyframing the active camera property directly keyframing the active camera property of the scene there is no active camera property it's a pointer how do you keyframe pointer she don't do that no, no, you don't do that you know that, you know that all you do any more here, that's one what performance improvements of selecting in the viewport can we have in 2.8 because it's still not that quick in 2.8 well I have about a thousand objects and 20 million polygons it takes about 5 to 10 seconds to select an object in 2.8 it does in 2.8 current version what graphics card do you use sorry what graphics card what operating system which driver do you use windows 7 nvidia quadro I don't know quadro yes quadro what I think it's latest I made fun of you but this is of course a serious problem 2.7 hap issues with intel or some other graphics card OpenCL select for 2.8 we are using so many new features that some selection might be slow but it's not meant to be slow it is not meant to be slow but there could be buggy drivers and stuff like this so if you have 2.8 and you find out a way to make selection very slow in an easy way submit it I'll talk to the guys you have your laptop with you but builder says do not report bugs about 2.8 we have enough on our desk would you kick my ass but can you try another computer and see if it's slow and if it's slow then it means that we probably can reproduce this and look into that but I guess if I didn't hear any developers complaining about this anyway it is supposed to be super super super fast if you have a slow selection in blender it's a bug and then we have to find out how to recreate it how to redo it and we look at it I have to wrap it up you still have your 20 questions one very last one okay my last question about tracks I think I tried to use it but I need to use sequencer to upload thumbnails and stuff and shots but will there be an option to do this with After Effects or similar because we don't use the sequencer I'm sorry one of the things that we're thinking about is to create a new shot directly from the web interface as well actually in the last few weeks I've been working on getting subversion linked to the cloud so that you can click on I want to have a subversion repository for this project that gets linked to the cloud if you also use attract it can install a hook for that so that those things are also linked and we try to integrate these things more and more so that as attract is familiar with the assets that you have we might also want to link them to wherever they are stored in the version which means that from the web interface we have enough information to actually construct a whole scene for you that we blend file with those assets that environment and then maybe even create that file for you so once that's done there will probably some API calls that you can also call whatever other software that can do HTTP calls Sibren, is it going to be like attract SDK thing similar to blender ID so what application can we use that functionality are there plans to have attract SDK pretty much using the same style of API as the rest of pillar so you should be able to use the pillar API SDK ok ok we have to wrap it up so thank you blender developers you can still have them have an afternoon here talk to them try to get them to work and solve more problems next talk


Hello everyone. We'll introduce ourselves here. I'm Brad Posorek. I'm the Systems Administrator with Oklahoma Medical Research Foundation, OMRF, as you'll hear us call it. I've been with OMRF for about 16 years. I've been in most of the infrastructure roles on the infrastructure team of IT, enterprise IT at OMRF. I'm Stuart. I am essentially a CIS admin also with a more of a background in Linux. I work in the Arthritis and Clinical Immunology department. Which basically means I'm actually in the labs doing computer work for for the labs instead of actually working for IT. Which basically means I do anything and everything computer related that is not handled by IT. I've been doing that for about 18 years. And we're here today to kind of share our story about SWIFT. So a little bit about OMRF. We are located in Oklahoma City, and we're an independent nonprofit biomedical research institute. Dedicated to understanding and developing effective treatments for human disease so that others may live longer, healthier lives. Just a little bit about OMRF. We're about 500 full-time employees in size. We run a pretty small lean IT shop. We have three system admins. Two of them are right here. The team of five dev apps, three help desks, security officers, CIO. That's it. We all wear many hats overlapping a lot of areas, help each other out a lot. A small fact about us here is that some people have a hard time believing that our IT department has actually run on a $150,000 capital budget annually. It's all we're given to bust it up, divide it up. We fight for it. Networking wants it. I want it. Everyone else wants it. AV wants it. Everything else we have to beg for. In the science side, we have about 45 lead researchers. They are grant-based funded. They bring their own money in, and they really want to do their own thing. This self-funded nature of them having their own money, some of these labs have big money. Some of them not so big. It's really created an us-first-them environment over the years. We're just constantly fighting administration versus science. My role in the IT role, we're focused on our core applications, accounting, procurement systems, payroll, networking, email, etc. We bring with that availability, business continuity, and a support model behind it. Many times this often brings project management and a longer deployment times to the table, much longer than what science wants to deal with. Science, like I said before, they bring their own pool of money. They know what they want. They need to do an experiment. They want to get it going so they can move on to the next step, find their discoveries. So they just go and buy their stuff, stand it up, move forward down the road to science. Oftentimes, this sets up what we call shadow IT. This stuff gets thrown up on lab benches and closets under desks, any place they can find to stick their equipment. For me, it's the typical usual business train model. I run Dell Equalogic Sands for all my stuff. I got two groups, a legacy group of PS6000s that I migrated off of. And another group, my primary group, is also Dell Equalogic. And it's dedicated to iSCSI on one pool and SMV NAS on another pool that runs Dell FS. Right now, I'm down to about 11 terabyte free on my NAS system, and that terrifies me. I could get a call this week that that's gone in full, or it might last me another couple of months. I really don't know in today's science. I need to add something to that. And on the iSCSI side of it, one of my main roles is the convolt backup system, where all my volumes are mounted, two terabytes of volumes into convolt. Well, I run disk to disk to tape, and really one of the key issues I had was my primary data and my backup data are all on one array, one nightmare away from being a full tape recovery. And I just can't fathom that, to be honest. So I need to find something I could grow into, expand as I needed it, and be able to do it in a cost-effective model. So from the other side, what I'm bringing to the table as a shadow IT group, just kind of giving you an idea of what I had before. Being the largest shadow IT presence on campus, mainly around the idea of supporting genomics workflows sequencing, mainly in the labs that I'm in right now. I run a pair of scale-out NAS systems. One of them is newer, smaller, faster, and under-support contract. The other one is older, slower, larger, but not under-support contract. This leads me to spending a lot of my time copying data between it as it ages off to get it off the smaller, faster ones. So we have more space there to continue with our analysis. So I say I have 200 to 400 terabytes of data. I don't really know how much I really have, because it's moving around all the time. On top of all that, I have to back it up somehow. So I actually run a third NAS system that I basically built myself. ZFS box sitting in one of these closets, just another building on campus that I then replicate some of the data to, again, as my supposed backup. Because the amount of data we have doesn't fit into the combat licensing that we have. And again, like I say, all of this I'm doing on my own, separated from IT. There's little backup for my actual data. There's no backup for me. There's no cross-training. It's all on my own. And like I say, I'm just a small portion of this. I'm the largest portion of the shadow IT, but there are other groups. You know, there's another lab that has close to 100 terabytes on a RAID 5 array. There's another lab that has another RAID array that has a drive in it that's failed that they have no idea how to replace the drive on, because they don't have the administrative password to that interface. It just hits there and blinks with a red light. So in getting this talk ready, I actually came across this little slide I had from one of our shares. It's pretty old, you know, 2010, 2011 timeframe. Just to kind of give you an idea of what all this stuff is there for. You know, this share, you know, we started out 2010 under 5 terabytes on this share. That's, you know, nothing. You can see the, you know, big spike there. We get some of our first sequencing equipment in and start running experiments and start running more experiments, you know, just keeps growing.  Contrast this with this little graph from one of the shares on our new NAS system.  They got it from running one experiment in one week and they needed to do something with it. And so now what took us, you know, about a year to generate, you know, we can do in, you know, less than a week without anyone knowing. It just shows up. Put it somewhere. Yeah, we have to put it somewhere.  That's not low. That's empty. That's no capacity. So what is all this hardware there about and that, you know, it's all about data. You know, that's, that's what it's all about.  It's always growing. You know, again, this is all about doing science.  And you know, like I say, we get that funding from our science side to perform these experiments, to generate these data. But that funding may come with money for me to buy some of these NASA rays that I've had in the past. But it doesn't give you the ongoing funding to maintain that NASA ray when that support contract goes up or, you know, you need to fix something that's broken in it.  It's just not in their vocabulary. You know, they've spent time, they spent money, effort to generate these data sets. Why, why get rid of them? What's the point of that? There is no point of that. So we're basically holding on to it forever. You know, I have stuff on my NASA system with timestamps before 2000. I have no idea what the data are, but they're there and I have to continue copying them around to different systems. So, you know, this leads to, you know, this mix of sizes of data and then types.  The backup data, you know, virtual machines, it's all there. And that's what we're doing for science. All our science is data driven now.  You know, the computer science stuff is coming to the bioinformatics space. That's what it's all about. On top of all of that, we have to protect this data somehow. You know, Oklahoma Medical Research Foundation, that's Oklahoma. You may think of, you know, a covered wagon, but Oklahoma is also known for a documentary that came out a few years ago. Some people think it's a movie, Twister. We are in Tornado Alley, so that is always a threat. And we've also learned of a new threat recently, of earthquakes actually. So, you know, we have to do something. You know, back to my older graph, there are under five terabytes. At that time, you know, my disaster recovery plan was actually, I told my investigator in my lab that, you know, if I see a tornado coming, I'll just drive down to the office and grab the drives and, you know, drive out away from the tornado. That doesn't work when you have a couple racks worth of NASS servers. So, you know, what all this means of all this data is we basically grew into a mess. You know, we have this historical warring, you know, on the bad days, just not communicating on the good days between the different departments and the different labs of everyone doing something on their own. You know, IT runs a SAN and a NAS system and they have their own backup, but that doesn't really work for the research computing side of things. I run a NAS system and need backup, but that NAS system doesn't really work for the IT side of things because my NAS system is too expensive for them and my backups are too large for them. So, you know, we're competing and duplicating effort and not working together on that.  You know, Brad is using tape for his backup. I actually looked into using tape for my archive data for some of my science data. I had to wait in line behind the business backups to get my stuff onto the tapes. I had to wait in line behind the business backups to get my data back off of the tapes when I needed to use that data again. I had to lose my data when both of my tapes failed.  Both tapes failed and so I lost some data there.  It's what tape is. And you know, on top of all of that, you know, we're both running out of space. You know, at one point last year, a year before last, for Brad's disc to disc to tape, it came to he had no space left and he needed some more for that. So I had to actually give him some of the space off of my science NAS. And then this question of, well, wait a second, he didn't pay for any of that. He charges me for storage and I'm giving him storage back for free. Shouldn't I be getting some money back somehow? Great model. You know, I don't, I don't really know on that. So it's like Sarah said, we both in IT and science ran out of space. We needed storage and we needed it quickly. I started down my ecologic path and we'll just buy another array, slam it in, expanded a couple hours later and I'm good to go. He went down his path, buy more Iceland, slam it in, expand. He's off to his science quickly as well. But what are we really doing there? We're paying and maintaining two independent systems. The cost is enormous to the foundation, to a nonprofit foundation. Stuart and I came together. We started talking. It's like, why can't we do this together? There's got to be something out there that we could both put in on that it could be useful to both of us, could be usable both of us, still perform it for him, cost effective for me. How do we move forward with this? So we both, we went off, got our, brought our normal traditional quotes to the table, eliminated them quickly, looked at some other scalable systems, and really OpenStack Swift quickly became the front runner in utilizing Swift Stack as its front-end management system. Some of these initial wins that we had on it was, my Commvault environment was very easy, very simple to deploy. It accepted it very well, formed very well. And Swift Stack had great Commvault documentation already out there for me. Provided an excellent page to grow model. We, as we fill it up, we just put in more commodity disk drives, expand it as we really need it. My CFO loves this idea. And the COO really loves that science and administration have come to the same table and are working together. Instead of fighting each other so much. One of my other big wins here is, it has 3x replicas helping to eliminate my tape environment completely. And really for the first time ever, I have a third location off-site in an actual DR facility that I can set up a Commvault DR replication center. Something I've never been able to do without having to buy a whole other system down there, and array storage down there to replicate too. Science immediately got it online and we're able to move off 50 terabyte of data immediately and archive off of their Isilon system to immediately ingest more science and get running fast. Kind of how this came together was in August of 2015. We put in our initial order. It was for six nodes. We did reuse one JBOD system to help initially save some money up front in our third backup region. There's about 250 terabytes total of the disk and the license purchase from Swift Stack. Roughly six months later, Stewart got a new project online, a new grant awarded for 100 more terabyte of disk and license and two more chassis for a separate genome project that his lab was running. And then our third major expansion was another 50 terabytes, just earlier this year, three more chassis nodes again off of a grant that he was awarded, and then I brought to the table the drives to start filling up those chassis. And then over the course of the whole year, as we've approached the best practices guidelines or needed more expansion, Enterprise IT has been able to fund that license and those drives, sometimes even sneaking drives in as office supplies. So a little bit more details on how that's all done hardware-wise. Like I say, we're using Swift there, standard 3x replica, so we have three zones in two regions. I say regions here in quotes because initially one of those regions was again in a closet in another building on campus.  We actually run a mix of drives. We reused some SAS drives we had previously. We've bought various SATA drives over the time and the last time we actually expanded some drives, the SAS drives were cheaper for us at the time, so we actually slammed in more SAS drives. We honestly spend more time when it comes to expand drives, bitching back and forth over what drive to buy, what's going to be the cheapest, what do we think is going to last the longest. We spend a week or more going back and forth looking at that than it takes to put the drives in when we expand it. We're using for the most part there all commodity consumer drives, which can fail obviously as anything and everything does. The extra thing we're doing that is using the full disk encryption, the Linux full disk encryption, so the drives themselves are fully encrypted. We did this for a couple of reasons. One, when we first did our install, Swift did not have native encryption. Two, it's just easier to keep it that way now and we don't have to care about the drives at all. They're throw away. Everything on there is completely protected if you don't have the key. Genomic data is considered personal identifiable information, so we have to protect it. We pull that drive out of the chassis. It's useless to anyone else without the key. For our one failure so far of a drive, we actually for fun went ahead and did the standard consumer RMA replacement on it, mailed it back to Seagate or West Indies or whoever it was, and they sent us another one. It didn't matter what was on there. For performance reasons, we're using read-write affinity, so we're targeting the local nodes in our local region just to maintain the performance and then everything else. Obviously we'll replicate off-site eventually. We started out with just simple round robin DNS between all the nodes because we were going with a full proxy account container on each node. That was done to meet some price performance and capacity requirements. We couldn't afford to do a dedicated proxy tier and we're continuing with it without the proxy tier just because it's easier to have every node do everything. Everything's the same. Later on we added in load balancing through the Swiftstack controller, which is what we're using to do all the management of everything. For anyone kind of graphically minded in the audience wants to see a pretty picture, here's some lines of everything. We have our two regions. One is in our main office in Oklahoma City. We sit on an Oklahoma City campus of the local university. The main campus is about 20 miles away down in Norman in Oklahoma and they actually have their own fiber between the two and are nice enough to let us utilize that. We actually have 10 gig down to our off-site now since that's been moved down there last month. Our two local zones on off the sides of the data center, that's all plugged into my high performance computing 10 gig network, and then I'm able to bridge out to the OMRF network now so I'm not separated by a little Linksys NAT router between my connection out there and then obviously the Swiftstack controller manages all that for, so Commvault and everything else then talks in. So what is all that fancy hardware actually get us? These are a couple graphs I pulled from the monitoring system shortly after we actually stood it up and the top one here is our HPC doing some test analysis and we were able to pull out a gigabyte per second of our archive data to reanalyze it and then we were able to push in about 600 megabytes per second, which for me really kind of turned it from just a cold archive to an active archive tier. It turned into something that's useful enough to do all our normal analysis from. The other side of this collaboration with the IT department, which is a traditional Windows-focused IT department, whereas like I said, I have a Linux background, open source, none of that stuff is a big deal for me, but to get something like this into the IT department was a little bit of a sell. A couple things that made that easier, some Ansible scripts up front, just a couple playbooks to do the basic stuff to set up your Linux and do our work for encrypting the hard drives. So Brad can run one playbook that will add a new node, get it all provisioned ready to go, and then when he adds drives in at any time, another playbook will find those new drives and throw our keys on it. The other big thing with that is the Swiss stat controller actually. Like Brad said, we all wear mini hats. On our hat rack, none of those hats is full-time open stack operator. None of our hats are even full-time storage operator. There's too much other stuff to do. So having the Swiss stack there to actually do all the work of installing, configuring, managing, and then most importantly, actually have a support number to call if something goes wrong or there's a question, which came up last year when I configured a couple things a little strangely, adding in some drives, and then immediately went on paternity leave. So having a number to call and take care of that made things easier for the IT department. The other thing that the Swiss stack gives us is we're not running the full open stack suite, so we're using just the standard off from there, but they have an integration with Active Directory, which then just from a simple couple of radio buttons lets the IT department manage all of the access to control straight out of Active Directory like they're normally used to using. There's no other different tool. So this is a quick shot of some of the accounts we have set up. So we can define all of our accounts to access our different scientific archives straight in Active Directory, manage all the control. Nothing weird for anyone to have to do. So as Brad mentioned, we got it set up in a week. We shoved the Commvault data in there. We shoved my initial archive data in there. That solved our initial pain points. But any storage can do that and just some data. There's nothing big there. And so what comes next is start using some of the fun stuff that you get with Swift. Do some programming against it. So what I started playing with was first start using some other stuff that can talk to Swift. Stand up a quick Docker registry. Everyone Docker, Docker, Docker. Everything's in a container. And for its persistence, straight in Swift. So that was easy enough. From there, it started to... I put on my developer hat and start to actually program against the API. So the first thing I did was just kind of get familiar with getting and putting. And so I actually wrote a quick back in for HashEvault for some secret stores. So again, that's not too fancy. Get and put objects. Nothing fun there. Next thing is start using it for some real business cases. For me, in the sequencing core, we regularly send 50 to 500 gigabytes out to customers. Previously, I was having to, again, manually copy this over into another share that was on a publicly accessible web server, send out a link to someone, then remember in a couple of weeks to go back and delete that stuff from my share because it was sitting on my NAS system. What does Swift have that can help me with this? Swift has a thing where you can expire data. So another quick script that I can tell it, here's a set of data, send it to this email address. It will create a container for me, upload all that data to the container as it uploads, set a delete flag in whatever timeframe I tell it, and then they get a link to a randomly named container and they can download the data. And I don't have to think about it. It just gets cleaned up afterward, and if they don't get their data, that's their problem because that's our policy. That's a policy thing, so it doesn't matter for that. So the next thing I started playing with in Swift was the temp URLs. And for that, what I started doing was I actually wanted to start backing up some of my other systems, but I didn't want to go tie-in to a Commvault license, and I just had a couple of small files on systems I wanted to back up. And normally, that would be okay, there's different things that can do back up and do it to a cloud object, but you have to give it credentials and save credentials in a credential file. That then means if someone get those credentials, they could read whatever data you've put in there or delete whatever data you've been in there. I don't trust anything, so I don't want my credentials sitting out on another system, even though it's in my own firewall. So I actually wrote a quick service that basically would hand out temp URLs on another authentication token, so I can just one-off back up things to a temp URL. I get to put URL for five minutes that's completely random based on another key. And again, it was just something quick and a day worth of hacking something together as fun. From there, like I said, we have the ability to use this as an active archive, pull our analysis data straight from it, and if you remember back to the Active Directory image, I have a lot of groups, and that was just part of it that was there, which means basically we've taken the approach to break up all our archives into separate projects and groups so we can more easily identify them and control access and know how much space they're using for any sort of charge back stuff we may have to do in the future, which means my users need to know what they have access to, and more importantly, they have to be able to access it with their Active Directory credentials. Again, I don't want them putting... well, I don't really care, but Brad doesn't want them putting their Active Directory password into some sort of environment file. So again, a quick little script of a program to interact with the list of tenets that are available and set up your basic auth tokens so you can then do some analysis, which then leads you to... okay, I don't want my users to manually have to download the files, run their analysis because a lot of our analysis software doesn't actually speak HTTP. It wants a standard file system, so we still have to pull these objects out. Again, I don't want them to manually download them, run their analysis, forget to delete that file because now I have the file sitting on my file system again, so I don't want them to be able to access it in the last space. So another quick script that basically you can name an object or a set of objects and a standard Unix command. It'll pull it down, pipe it through, run the stuff and delete the file behind you and just a quick thing of counting the lines in a file. So all that, just a couple of days of playing around with some development stuff. So it's all sunshine, rainbows, great stuff, but there are some troubles along the way. So, familiar with the idea of objects, how to use that and how do we want to use it, what's the best way to set up our containers and projects. That kind of then bleeds into some user error. Like I said, I had user error from the admin sign of adding stuff too fast, but I also have a user error from the standpoint I live in fear of yes, the data are 3x replica, but if I accidentally do a curl delete to the wrong thing, I could delete all my data. So we're getting around that right now with different accounts that have read and write access. The other problem was, like I said, the offsite. It took us over a year to actually identify a location that we could A, afford, B was far enough away and C had enough bandwidth. This little picture here is when we were doing some experiments on site using virtual PF sense to see if we could encrypt the data enough and have enough bandwidth, and then how slow we could get it. The virtual PF sense did fine encrypting with an IPsec tunnel for what we needed to do, and as we were slowing that bandwidth down, we found at about 500 megabit we would fall behind on the replication too far, so we knew how much we needed to have to actually do that. Going forward, like Brad said, the CFO loves the cheap and he wants all the stuff in the NAS systems now, so we're looking at, can we tear off of that? My ISLON has a thing where it can automatically tear, which would be nice, and all the users could use it. It technically only supports public clouds like S3, which then makes me think, maybe I could plug into the Swift S3 stuff. But then I'm tied into using that proprietary system forever, so I'm really looking forward to what comes along with proxy FS to just skip the NAS altogether. I'm looking at the data aspect of it. All the scientific data I put in there has tagged with the investigator, the sample, the disease, the project, the grant number, anything and everything we may ever want to look on, and so to now put a search engine in front of that so other investigators on campus can discover what data are already available to them. The last thing I want to start to kind of developing is play with some middleware, actually. Like I said, I fear myself that I'm not wrong, so I want to just do a simple, simple worm like middleware of if you do any destructive operation, just have to have an extra HTTP header on your client's call. Just something real simple. And so really what has that brought us to now that we're collaborating, we're doing everything together, it's no longer us versus them, it's us and them working together going forward. We're fleshing out our shadow IT departments, giving them the 3x replica, the disaster recovery, the business continuity aspects and hopefully saving them money in the long run but not having to buy this stuff up front and putting more money back into their scientific research. Some of the takeaways, real quick, so far we've got about 420 terabyte of object data sitting in three regions. We've come together, presented a viable joint effort using OpenStack Swift and SwiftStack as the front-end management and proved not only can it work and science and administration can work together but we did it providing a more cost-efficient model providing more benefits that can be achieved again like I said, with more traditional storage efforts. I will say personally up front that this was a very daunting project initially as it brought it to me being traditionally a Windows environment traditionally trained but Swift has not been so bad and really the SwiftStack portion has brought a great sense of ease to me. Thank you. Any questions? Very interesting talk. Thank you very much. Can you say a few words about how you manage the distribution of resources between operations and the mission side in terms of like are you working this as a condo or how are you dividing the costs for long-term maintenance and provision of capability? On this OpenStack model we kind of who bought the last round who gets the next round type thing? It's a mix. The initial purchase that was kind of a 50-50 split. We both needed it and we were both going to get half of that 250 terabyte that we initially bought. Since then some of those expansions again just kind of the way some of the stuff works it's easier for the science side with their larger grants to buy the chassis you have to get the whole chassis at a time even if it's empty which is a sizable cost and so we've gotten some of those and then we get capacity at that and then like Brad said it's easier for them to kind of buy drives one off on their own I can bury those in operating expenses he can put those under his budget and so long-term what we've worked out with the administration is they'll maintain the licenses going forward is the idea and then what we're presenting to our other investigators is like the guys that had 20 terabytes of USB we can either just take those USB drives and slam them in there or you can buy other drives bring your drives to us, we'll put it in there we'll cover the rest of the cost again to try to get these data out of the labs out of the closets and something that's protected okay the other guy and this maybe goes back to the same question of how do you break up costs do you use the accounts to sort of group like projects or departments can you see like how much storage is in accounts over the past month or six months or something like that yeah actually that's again kind of why I have so many accounts set up accounts from the SWIFTAC SWIFT level is with the SWIFTAC interface it can easily give a report back in terms of the usage buy the top level account without having to go in and run extra accounting against containers individually so we can say okay this project is named the first investigator for this purpose so we can know exactly okay you know investigator X you're using this total against these you know five accounts that you have in there for your five different projects thanks for your talk you mentioned that you have I think you've termed it an eternal retention policy have you have you used different storage policies to kind of create like cold tears for moving data to doing that for most of the stuff at this point no I do have some different SWIFT policies for the customer delivery I am using just a 2x replica for that one because I don't care about their data I probably shouldn't say that in case anyone wants to use our services and so that you know I don't need to make that third replica because again it's a short-lived thing that doesn't go out so I'm only using 2x on that but for everything else we haven't done much of that I don't want to go into the exploration of other policies you know erasure codes or anything like that that's one of the things I am actually waiting for is multi-region aware erasure codes so maybe we could move that for some of the really old backup data but still have be sure that there's enough offsite to fully recover if it comes up good stuff we're kind of looking at something similar and one of the questions that we have is and it may relate to the relatively high load on the environment and then you lose your link to your offsite and your main site goes down and you rely on your offsite and then the offsite and the online they both eventually come up and have to resynchronize and how long something like that takes or is the amount of data that you're loading low enough that they can get back to sync relatively quickly our daily churn is low enough at this point that it hasn't been an issue like I said it was just three weeks ago, four weeks ago that we actually did get the stuff offsite and so that third region was completely offline for 24 hours as we drove it down I had unhooked the day before so I could get everything on track and we had to throw it all in trucks because we were cheap, we had to do it ourselves and physically drive it all down there and hook it all back up and so it was offline and everything came back and resinked relatively quickly, it wasn't an issue our daily, the Commvault stuff is the biggest driver of daily churn usually and it hasn't been an issue I don't know if we'll get another big whole genome project and have 70 terabytes show up in a week or something and have to put that in but it's supposedly a 10 gig link and we can use it all so we'll just let it churn for a while so at this point all the data we're putting in the scientific side is technically private to our own investigators but that is something that would definitely be something to look at is turning it into a repository and make that wider available if it can be or agree to by that investigator Thank you