cs344 ยป

Contents

Whats your job

Alright, I'm happy to be here today with Ian Buck from Invidia. Ian, welcome. Hey John, how are you. >> Doing great. Can you tell us what your job is at Invidia? >> Sure. I'm the general manager of computing software at Invidia. Basically, what that means is, I'm in charge of all the things for using GPU's that are not really graphics or video related. Things like simulation and problem solving that you don't really do in a game but we're recycling those cycles to use them more for, for us, kind of supercomputing kind of applications. >> And you've had a lot to do with the development of CUDA. >> Yeah, I started the CUDA team many years ago, going on six, seven years now. Finishing my work at, Stanford to do it for real at NVIDIA.

How did you become involved with GPUs

So tell us how you got involved in GPUs in the first place. >> It goes way back. I, I, was actually an undergrad. I was working on the, some of the original SGI Octanes with OpenGL. And discovering that actually some of those OpenGL Octane machines for doing graphics, you know the kinds of image processing you do in OpenGL could actually be used for computational. The simple computational things like fluid flow. So my, one of my undergraduate thesis was to study fluid flow and simulation in the open geo pipeline on an old SGI Octane. And then went on to Stanford, did a few other things but came back to it eventually and that built a career out of it.

Brook Project

So, the project that you're, I guess, best known for when you were a graduate students, is this Brook Project. So can you tell us a little bit about that and, this was a very important step in sort of the, the progression of GPU computing to something that we can do commercially today? >> Yeah, when, when. Back in the Direct X 9 days, of computer graphics, GPUs started becoming truly programmable. People could express real programs, fairly short ones, maybe only four to eight instructions long, that can actually, you know, solve a shading problem like, face, skin, shader, fairly complicated stuff. A bunch of us in Academia, no-, noticed that the performance of these GPUs was huge compared to CPUs of the day, the number of floating point operations you could do was in ordered magnitude larger. So we studied and looked at how you could do, use GPUs for more general purpose things, by hacking graphics API with Direct X. The research, while, while, while a lot of people show that, that it is doable, and, >> But it was really hard. >> It was a really hard to program, in fact you needed a PhD in computer graphics, to do even basic mathematical kind of stuff. In fact there were folks in finance, that were hiring you know, game programmers to try to hack GPUs to solve financial problems. The research I did, was basically study the more fundamental programming model around GPUs. What was it about GPUs that made them programmable and, and in parallel, but also, present a more cohesive programming model to general, you know programmer, to let them think about GPus in a more general way. In the end, the project was called Brook, it was an actual software, you know, programming language for how to program in a general way GPus, on top of its existing graphic APIs. So users could think about, you know more general data concepts, instead of triangles and pixels and textures. What sort of things did people write in Brook? >> We wrote, Ray Tracer to sort of study, you know could, could a GPU actually implement a Ray Tracer, which is a fun thing, because lot of people said it couldn't be done. We did, triangle meshing, we did a lot of it, simulation, and grid kinds of stuff, thermodynamic problems, heat transfer problems, flow, fluid flow kinds of problems, For the most part, it worked, it, it demonstrated the, the performance benefit, it was still early, you know they, they weren't fully general, so you did have to work around some of the, the hiccups, but from a research perspective it achieved its goal showing the potential of GPU computing.

Starting the CUDA project

Nvidia called and said, you know, we'd like you to come here and start looking at how we can apply this in a, in a more product, I guess, orientated sense. >> Yeah, NVIDIA was a great partner in the research. They were giving us access to the hardware and some of the early specs, so we could think about where they were going and how it influenced our research. When we're all said and done, I, you know, I got the opportunity to come to NVIDIA and do it for real. I started with the Cuda group with one other guy, and it was just the two of us. And we hired in a compiler and library and QA guys, and we, we slowly built our, our way up. This time, really taking the learnings from Brook and starting over a little bit. And also influencing some of the Harvard design, the Harvard architecture, you know, what could we do the generalize the architecture just a little bit to make a little easier. >> Mm-hm. >> To eliminate those roadblocks we discovered with Brook, and then eventually come up with our sort of Cuda 1.0. >> So, how long did you take from the day you got there and you started on this project, to the time it launched? >> It was probably good 2 years of development work that we were, we were pushing to do this work. You know, lot of it was spec writing, kind of coming up with paper specs, writing us some sample code examples, seeing who is usable, easy to use. Picking between of variety of different approaches for how to express parallelism. Eventually, we latched on to a very basic idea of threading because we knew that most, you know, C and C++ programmers understood the concept of a thread at least at the simplest level and if we could latch on to that we could actually present a program model that was familiar. Very early on in the Cuda project, we did a road show, we went out and we talked to every customer who wanted to talk to us saying, tell us your problems around computing, how can we help you? And one of the big sentiments was, they didn't want to learn a new language. >> Mm-hm. >> And you know, while it was tempting as a researcher to come up with a new parallel programming language of the world, they just wanted an easy way to get access to the performance, and they had a bunch of C or C++ or Fortran programmers. So, when we set out to start Cuda, we made sure it's not a new language. It's just a couple of simple extension to C and C++. So that anyone who knows those languages plus has a basic understanding of threading could get access to the GPS performance.

Most Interesting projects youve seen

What are some of the most interesting projects that you've seen done with CUDA from, from the beginnings, all the way until today? >> Yeah, you know, one of the great parts about this job is that I continuously learn about fields of science and computing that I had no idea existed. Whether it be, you know, defense or oil and gas, or finance. One of my more favorite projects is the work, work we have done in like CT scanners. A CT scanner is actually take I don't know if you know how this work but it's a series of X-rays they take at different angles, and you're taking 2D X-rays and the machine actually composes the X-rays together and were actually, recreates the 3D model of what's going on inside of you, lungs or heart or body or whatever. Well it's really cool to get that 3D vision. The bad part actually is the radiation is kind of harmful. In fact, you want to kind of use the least amount of radiation but the more x rays you get the better the view of the problem you have. So what one, this one group is doing is actually, they're using GPUs and they've been able to reduce the amount of, Of x-ray radiation by something like 20 x. Really making it much more safer as a procedure. And applying the power of the GPU to take many more lower-dosage radiation shots. And combine them together to not only get a clearer picture with less radiation but you get it really quickly and. In one sitting. You don't have to home and come back to figure out if you have x, y, z problem or issue. So it's really cool to see that kind of stuff truly affecting lives.

Coolest features in CUDA

So if you fast forward a few years, there're been a lot of new ideas that have come into CUDA, between the time we launched five-six years ago and today. So what are maybe the two or three coolest ideas that you think have come in since the time we launched. >> Well certainly we've added a lot of support for managing concurrency. There's no short of parallel in these, in these codes if people want to move to the GPU. So we've added support for con, better support for concurrency, for streams. We didn't move some of that from what starting out from software support, actually it's now. Baked into the hardware with the Keppler and the hyper cue. It, those are pieces of silk and they're actually inspired by some of the stuff we were doing in software, originally with Kudo two point oh. Other stuff we've been doing is integrating closer with the memory hierarchy. In Kudo four point oh we did the UVA work which is unifying the virtual address place. Of the entire machine, such that they, this was work we did, sort of tightly integrating with the operating system, such that a pointer, bi pointer value, the system can know if it, if memory is living on the CPU or living on the GPU. So there's no separate memory spaces anymore. You can simply dereference a pointer, and it goes to the right machine, the right Speaker:CPU or GPU or your peer's GPU. Make, that made multi-GPU programming way easier than, than we had before Kudo four, four point oh. >> Yeah, I mean as a professor I think usually what I say is the most common error is accessing memory on the wrong side of the machine. >> Yeah. >> You think you're de-referencing a GPU pointer and it's really on the CPU. >> Yeah. >> That's certainly very helpful and going out and doing those. >> Uh-huh.

GPU impact on mobile world

So, the world is becoming more and more mobile and so the expectation is that the sorts of things we do with big GPU's today maybe one day we'll see being done with something you carry in your pocket. So where do you see GPU computing having an impact on sort of the mobile world? >> Well like I said, computing is just sort of everywhere. And it's certainly coming to the mobile space. I think cell phones today are replacing our laptop PCs, and there, there are opportunities and there are new mobile device. Something that is always on, carrying it with us, we're interacting with people and friends, as well as ourselves with the device. And that's another opportunity for computing to make a difference. Camera is one area. The one thing you do see is that, you know, cell phones are trying to take over the digital camera market. And there are things that you can do with a digital camera with a processor, with an interactive test screen things like computational photography and computer vision with the camera that just wants to be done on the device, and really changes it from a point-n-shoot thing, to a more dynamic interface, to something interactive. And I hope to see some really cool stuff happen there.

What to do with 100 times more compute power

So, I mean, that's an area we can probably use a lot more computation and use it profitably and, and actually see the difference. So, one of the questions I always ask my students when I'm teaching is, what are you going to do with 100 times more compute power? So, how about in the more, you know, high performance computing area? What are people going to do with 100 times more flops? >> Well, there are some problems which have limitless amounts of computational need. Things like understanding disease. You know, the work being done by folks like BJ Pading at Stanford. Where you're trying to simulate the actual progression of a cell as it gets attacked in the disease. He's got a project working on Alzheimer's, where he's trying to actually witness a nerve cell, and watch a nerve cell die at the atomic level. You know, simulating every gene, and protein, and cell membrane, as it gets infected, and, and, and, and eventually deteriorates. Just think how many atoms are in a cell assuming that in real time or at least in a timeline where you can actually understand disease down to the actual chemical level. It's really going to provide a cool breakthrough for how we treat those things and understand the path, the very complicated pathways and then design the right kind of drug that can get in there and stop it at the right point and keep people healthy. I think that's one of the one of the more cooler things we are going to be doing in, in computers.

Surprising uses of CUDA

So what's really surprised you over the last five years, in terms of how Kudo's developed and how people have used it? And I'll throw one thing out there, I, I think that the day I was disclosed on CUDA, right before it launched, if you'd asked me at the time I would have said, you know in five years, maybe people aren't going to be programming so much in CUDA. Instead people are going to build higher level languages on top of this, and that's what we're going to be using. But CUDA's really proven to have legs and, and is a very efficient way to keep these machines, busy with lots of work to do. So, you know, I certainly would of lost a bet I made then. What have been some of the things that have surprised you? >> I think one of the things that surprised me the most, is just the diversity of fields, and really, you can't predict it in some, some regards, but you can. There are, there are applications for computing in tons of spaces, and you, we have a vision of, you know, if computation is free, what would that mean, and what would it mean for everyone, and we can inspire ourselves. But there are all these short term things that we can go achieve with GPUs, that were simply just not, no one thought of, there may be a small group of people out there that were researching them, But you, we, by giving them this sort of, this processor where computation's effectively free and memory bandwidth is 10 to a 100x faster than what you're used to, they're just unleashed. Astra GPU, there's a actual conference now, on doing astronomy simulations with GPUs, and they were the, one of the first few people to jump on the bandwagon, because they have a really big problem. They're trying to simulate galaxies and black holes and star formation and they were just stuck, they couldn't, they, the simulations were so hard and there, they'd maybe get to run on a super computer for a brief a few cycles at a time. But now all these guys are doing it on their computing, right at their desktops and actually prove theory, about how stars form, how a galaxy forms. I mean this is, I think we, no one ever knew about existed, this kind of problem that existed, because this little community is off to themselves. But we discovered these things, and we, we're so excited to see that we make a difference.

Advice to Students

What advice to you have for students that are taking this course that are starting to learn about parallel computing? So what can you say to them that they should be thinking about as they, they grow and mature in this area? >> Well first off they picked a good course, [LAUGH]l, because this is definitely an enabling technology. By having experience and, and expertise in GPU computing can let them be a, A vehicle for extracting, you know, that kind of flops are free, computation and unlock some of the things that are, that researchers or industry or even commercial space haven't been able to do yet because they just don't have the bandwidth. That don't have the compute power, horsepower. Cuda, NGB/g computer, in some cases is meant to inspire, I mean just think about if, if computing was free, if you didn't have to worry about, you know, the flops or the, you know, bits per second, what could you do with the kinds of problems in your space or in your field? Being part of that revolution is totally fine and I see it everyday when I work customers and, and selling GPUs or talking to developers. I think they should be, not only just learning the technology but >> Thinking about what they could do with it and where they could apply it. >> Yeah. Thanks so much for spending some time with us today. I appreciate it. >> Sure. Thanks John.