RenderMan traces ray streams of a few thousands rays (at least not millions like Disney's Hyperion), right? Does that really help that much with reducing the shading costs? It probably makes a large difference for shading the initial hit point, but nowadays it's not uncommon to trace paths of more than 10 bounces. It seems very unlikely to me that it gives a significant speed-up with that level of incoherence and the high complexity of production scenes.
According to Per, the primary rays are traced per block, giving coherent accesses to the high resolution textures. Secondsry rays, especially with higher orders of reflections, are accessing higher mip map levels where it is safe, and as I understand it, these mostly fit into memory by virtue of being filtered down.
However, I don't think they actually do 10 bounces of diffuse reflections just yet. But this is just a guess. I simply doubt that it makes enough of a difference to bother with that.
Blocks of primary rays tend to be about 16x16 by default, though this is tunable. MIP maps do greatly help with textures for incoherent rays. But we are also able to use the same trick for geometry; secondary rays may use coarser tesselations than primary rays. Per wrote about this some time ago, but it's something we still leverage. [0]
The number of bounces we do is fairly flexible. [1] We actually default to an upper limit of 10 total rays per path, with an initial limit of 1 diffuse bounce and 2 specular bounces. Depending on the geometry that's hit these initial limits may get extended until the total hits the upper limit. There are cases where the higher limits make a noticeable difference to the look.
Some context: During a panel on production renderers at SIGGRAPH [0] it was mentioned that most rendering research was not really usable for production rendering, as those algorithms are unable to handle the complexity of production scenes. Matt Pharr (one of the authors of pbrt [1]) asked the panelists to release production datasets for research purposes. Walt Disney Animation Studios is now following up on his request by releasing these two datasets. Yining Karl Li of Disney [2] and the readme for the Moana dataset [3] have some more info.
It's absolutely amazing that Disney is doing this, especially as the license is quite permissive and also allows the usage for non-research purposes.
As an amusing aside, Brent and Rasmus got us a scene from "Meet the Robinsons" back in late 2006 that led to our first real on-the-fly subdivision work. Working with Brent and Dylan at Disney, it ultimately led me to support correct subdivision in Arnold.
At Stanford, this fed into our micropolygon/subdivision work, though we couldn't use the Disney scenes as that was only for me, Dylan and Utah. Luckily, Marcos at Solid Angle put me in touch with the folks from Zinkia Entertainment, which was much more challenging than anything we would have created by hand. But we couldn't share that with anyone else either.
That this dataset is actually publicly available may easily push forward the subdivision/rendering community. Modern subdivision research folks (Loop, Niessner, etc.) have blazed past traditional evaluation with some amazing approximation work, including sharpness. Production renderers still don't those except for previewing though, because it's not that big of a win and because researchers haven't had access to scenes like this.
I'm super excited to see this scene (or portions thereof) replace Big Guy :).
Thanks for providing that context. It really sets the scene on why Disney decided to do this, and it’s peaked my interest. Didn’t realize how challenging this was..
There's very little info in the Disney paper on geomapping, do you have any ideas on how they do it?
I guess that you don't want to page in full models (as you either need a very large amount of memory per thread, which means smaller ray batches, or you need to have a shared cache, which means waiting on other threads). So you'll probably need something similar to Toro (as described by Pharr et al.), which just doesn't seem very efficient to me (as grids have improved little compared to BVHs).
Easiest way is to have a two-level BVH, where the lower level just has overall AABBs of every object in the scene which contains a "geometryInstance", each of which then contain another object-space BVH of the triangles (or other primitives).
This is trivial to do, but you take a pretty big penalty for not being able to look down to the primitive (microtriangle, microcurve) level for the lower level BVH, so the quality of it isn't as good. So for stuff with hair on skin, you've pretty much got two objects overlapping which sucks for traversal performance. There is a way around this, but it involves storing a pointer to each primitive in the BVH which is expensive and mixed transforms / motion blur get complicated with this.
The more complex way of doing it would just be to use the single level BVH and partially build it - given the lengths Disney are going to in order to sort and batch the rays, it sounds like they can constrain stuff such as all rays they send in a batch are going in a similar direction anyway, so you can cull a lot of stuff.
These days in VFX we're pretty much just fitting stuff into memory anyway - there's no other option. PRMan does support geometry paging in theory, although Arnold doesn't and VRay only partially does (with VRay meshes), but performance absolutely sucks doing it, so buying more RAM is just the easiest/cheapest solution all round.
Disney are only really doing deferring / reordering to such an extent because of their love of PTex which sucks with random access IO, so...
Yup, I meant paging geometry. The problem with just doing a two level BVH is that some models can be very large, while others may be much smaller, which makes it hard to reserve memory for the geometry cache (even if it just fits one model per thread). Which means that you can store fewer rays in memory, and I'm not really a fan of constantly writing rays to disk. I was thinking of just cutting off subtrees of the BVH and storing those on disk, which makes it possible to assign just a few megabytes of memory to each thread for caching geometry. Not sure how efficient that would be though, probably a bit faster than rebuilding part of the BVH every time (especially when using high quality BVHs with spatial splits). Shouldn't be too bad with large ray batches, sorting and ray streams.
They do mention some tests with "out-of-core rendering of scenes with massive amounts of geometry" in the paper, but there isn't much info on it. AFAIK their patents mention streaming in geometry too.
I know that buying more RAM is probably easier, but it would be interesting to render very large scenes on commodity hardware. I guess paging in geometry is just too much of a hassle, you constantly have to stream in geometry to memory; to compute surface derivates, to do subsurface scattering, to sample arbitrary geometry lights, etc.
Why would you need per-thread geometry caches? That doesn't really make sense (memory duplication)...
What you'd do is just map the geometry + bvh into a serialisable format (like vraymesh) such that they can be fread() in one go pretty quickly. You could put heuristics in to not page out very small meshes...
But unless you batch and re-order rays (and I'm not convinced doing this is worth it if you're doing more than 3/4 bounces, as the incoherence just makes things a nightmare), there's no point really doing this (unless you're happy with very slow performance - even mmapping stuff is slow).
The per-thread caches are not strictly necessary, but you don't want threads to block because they can't stream in geometry to memory. You want some upper bound on the amount of memory each thread needs for geometry (each thread could try to ray intersect a different mesh). Dividing geometry into fixed sized chunks gives you this upper bound (max N chunks per thread).
Ray re-ordering is indeed a nightmare, but sorting very large ray batches (millions of rays) into coherent ray streams is less of a hassle, and it should enable coherent geometry access (which amortizes the cost of those reads, not sure to what extent).
But that doesn't really scale - unless the per-thread stuff is small (and then you'd have to page on a sub-mesh level - i.e. split up the mesh spatially and page it as you traverse through it for internal scattering or something), if you double the number of threads, the memory used for threads by doubling the number of threads will obviously double as well.
For stuff like hair where you need loads of scattering events or for SSS doing that (paging subsections of shapes) is just going to thrash any geometry cache you have unless they're quite big.
And I'm still not convinced by ray sorting: it means your light integration is extremely segmented into artificial progressions as you expand down the ray tree, and makes anything regarding manifold exploration (MLT or MNEE) close to impossible to do well.
Well, if I opened any article that has a like or a share/recommend button, I did technically send a request to Facebook and got something in return. Does that count as "using Facebook"?
Google has a search engine monopoly (at least in Europe), this gives them the obligation not to abuse this. If they're using their monopoly to promote other services (such as when directly show Maps in the results, without offering other providers of maps the same option) or if they're scraping content without linking back (such as with Yelp), they're abusing this monopoly.
There is no such thing as a "search engine monopoly".
The word "monopoly" doesn't mean "lack of viable alternatives", except colloquially. (Colloquial uses of the term are not appropriate in context; that is equivocation.)
The same can be said about Windows, yet both the European Union and the United States accused Microsoft of abusing their monopoly by bundling Internet Explorer with Windows. This antitrust against Google is really similar.
While I share your sentiment, the case with MS was different because they were coercing hardware manufacturers into not bundling Netscape by threatening to revoke their Windows licence. Google aren't revoking access to their other, primary product if you use Yelp, Bing Maps etc.
They gave Yelp two options: either disable crawling in robots.txt, which means that Yelp won't show up in Google's results at all, or accept that Google will copy Yelp reviews without linking back. This is quite similar to what Microsoft was doing in my opinion.
The only issue is that people were forced to pay for Windows when buying a PC for running a different operating system. PC hardware OEM's paid royalties to Microsoft whether or not users installed Microsoft systems, and the OEM's passed down this cost indiscriminately. That's an example of monopoly power: users wanted to buy just a PC, without a hundred-something dollar OS, but couldn't.
As for bundling of IE, Microsoft has a right to put whatever they want into their software release. People who use Windows and then complain that it comes deeply integrated with a Microsoft web browser can be safely regarded as supreme morons. An OS needs a browser in the internet age. Ubuntu comes bundled with Firefox. (Browser choice is good for Windows; Microsoft only hurt themselves by integrating IE in annoying ways into the OS. Many people are happy Windows users, yet browse the web with an alternative browser; Microsoft should be happy they are on Windows.)
Moreover, the history of Windows is littered with entitled whiners whose application or utility became irrelevant when Microsoft realized that an OS should ship with that kind of thing and produced an equivalent. According to those litigious nincompoops, the OS vendor has no right to bundle things such as a TCP/IP stack, disk compression or anti-malware utilities, because it threatens their "add-ons" which were there first.
The whole IE thing is like going after Toyota because they install cruise control, air conditioning or decent audio in many of their vehicle models. And cruise control, AC, and audio systems are something you can have installed after-market from third parties, so by Toyota bundling such features into cars, it ruins their business. Moreover those features are slickly integrated into the car (e.g. dashboard/steering wheel controls) and hard to remove and replace with the generic parts. The bundled factory features thus enjoy unfair advantages in integration.
Google, by contrast, only runs a program on some machines in the cloud that only respond when you resolve their name and send them a TCP/IP SYN packet. Every search starts with someone resolving "google.com" (or a related regional domain) and making the first contact. From your IP address, you can similarly reach any other search engine on the planet, if you're in the free world, at least.
I think this is aimed at middle/high school students. It probably can be interesting for adults, but I'd really recommend picking up a book such as Fundamentals of Computer Graphics by Shirley and Marschner [1].
Yeah it seems to be aimed at teens, but this is so far beyond most excellent high school students. They're dropping in terms and concepts which you don't touch on until the final years in an undergrad, with a focus on graphics.
The CG by UC Berkeley on EdX is a great starting point. One of the exercises involves building a simple ray tracer. A slightly more advanced source on ray tracing is http://www.scratchapixel.com/. Of course the best way to learn about ray tracing is by reading Physically Based Rendering: From Theory to Implementation by Pharr and Humphreys.
Pixar has ditched RSL in favor of BSDFs written in C++, so there is no real reason for learning RSL anymore. Open Shading Language (OSL) is a new shading language which is getting more and more use in the industry, but it's completely different by design.
Yes, you're right about RSL. What I've meant is that it is (was) a good learning tool. Maybe something like shadertoy today with an appropriate course would be apt?
PBRT is a great resource too. I don't know if they opted out of literate programming? I can't stand that for some reason. I've leafed through PBRT though. I've learned (back in the day, hah) from Shirley's Ray Tracing book and bibles that Andrew Glassner wrote (Principles of Digital Image Synthesis) which I still consider awesome resource.
Yeah it was a great learning tool. pbrt is a good learning tool, but sadly it already implements most of the basics. Courses at Cornell and ETH Zürich used a small rendering framework called Nori [1], which is a pretty good learning tool. Shadertoy is nice, but perhaps a bit annoying to work in (as it involves a lot of tricks). I guess the best way to learn is to write your own ray tracer, but that takes a lot of time.
The current edition of Physically Based Rendering still uses literate programming, but most of the maths are separated from the code, so you can skip over most of the code. Even if you can't stand I can really recommend reading it, as it's a great book. Shirley's Realistic Ray Tracing is still very useful these days, but obviously it's lacking the state of the art. If you really want to avoid any code you should read Veach's PhD thesis [2] and then some papers on BSDFs [3], shapes [4] and ray tracing acceleration [5].
The REYES algorithm is an algorithm that can very quickly produce batches of shading points (with the required information to shade them such as the viewing direction). It's not really a hack and you can still use path tracing (with BSDFs) to correctly shade these points. In fact this is what Pixar did for Monster's University; they used the REYES algorithm to determine shading points visible from the camera and then shaded those using BSDFs and path tracing. The problem with this is that you basically have to store the scene twice: once for rasterization and once for ray tracing.
Isn't that complimentary to what I said though? You could presumably get the same result by just path tracing the whole thing, so all you've gained is speed at the cost memory. I didn't mean speed hack in a pejorative way.
I understand why you might want to do this in practice, but it still doesn't seem to be that important to teach as an introduction.
The difference between Reyes and path tracing is this:
- Reyes batches camera-visible geometry into grids of micropolygons and microvoxels. These micro-elements are then shaded using any method you please (which, in the case of Pixar movies since Monsters University, has been raytraced physically based illumination). The micropolygons are projected to the image plane and stochastically sampled, much more akin to traditional rasterization. There is no raytracing for camera-visible geometry, and all projections must be linear.
This style of doing things was to exploit locality in screen space. It assumes that the image can be broken into "buckets", and the information outside the bucket can be destroyed when the bucket is finished. This used to be true when RenderMan was first designed, but now that everybody is raytracing their illumination, it is less of a useful optimization.
This also means that shading happens at discrete times. Usually everything is shaded at the shutter-open instant, so specular highlights will be smeared in an unphysical way, because their motion is not properly correlated with the geometry motion.
I should also point out that Renderman keeps an internal radiosity cache in Reyes mode, which is basically just stored colors of illuminated geometry. This cache is queried by raytracing when computing indirect illumination, and a cache of the whole scene is kept for each bounce. Pixar only uses one or two bounces in Reyes mode due to the heavy memory cost. Note that view-dependent indirect illumination is not possible with this scheme-- Second bounces (and deeper) are always diffuse only. (It is possible to avoid this, but it tends to be slow because Reyes makes all kinds of generally-untrue assumptions about its ability to batch shading which fall apart in second+ bounce shading).
- Path tracing fires rays for camera-visible geometry. In most path tracers, including RenderMan RIS, each ray hit generates a new shading exec. This means that time-jittering for camera motion blur also trivially produces time-jittered shading, so (among other things) moving specular highlights will be correct, and view-dependent indirect illumination is possible. Because of this live-shading aspect there is no radiosity cache, and so much deeper levels of indirect illumination bounces can be used (compare 8 bounces to Reyes' typical 2, for example).
Also, because camera-visible geometry is raytraced, it is possible to do interesting things like nonlinear projections.
* Discarding data when finishing a bucket means that you have to produce final-quality pixels before moving on. Path tracing allows the renderer to produce an initial, crude pass over the pixels and then refine from there. This leads to quicker initial feedback and makes interactivity possible.
* Object instancing is much more practical with ray tracing.
* Before hybrid REYES/ray tracing, you'd have to bake out shadow maps and other passes before you could get to the beauty render. Even with ray tracing you might still be baking point clouds for GI. With path tracing you can do it all with one render pass.
I guess I didn't fully understand your post then, I thought you meant that the Reyes algorithm is an "inferior estimation" and that it might be beneficial to add it to a renderer, but neither is the case. But it isn't an inferior estimation (just a different way of computing things) and you don't want to use the Reyes algorithm in practice as scenes either barely fit in memory or they don't fit in memory at all (and in that case you want your memory usage to be as efficient as possible to avoid reading from the disk as much as possible).
In "regular" Reyes (and I may just have this wrong - I was unaware that the hybrid technique you describe was common) I though you were a) locally approximating surfaces and b) applying local shaders. These shaders are typically inferior estimates of the BSDF at that location compared to other techniques, but can be very memory efficient.
In the hybrid approach you describe, you avoid at least (b) above by doing the shaders by path tracing, say, but localized to where your rasterizer has discovered. So you aren't benefiting in anything but speed, so this seem more an implementation detail than anything fundamental.
So I don't see how dropping this sort of detail from any introductory course is anything but sensible. There should probably be a discussion of space/speed/distributional issues in general, but at a higher level.
Again, I'm not questioning why anyone would want to implement it, just why anyone would find it odd to leave out from an introductory course.
http://kesen.realtimerendering.com/sig2018.html