Render Farm Management for Digital Media Pipelines



Try It FREE 30 Days


EDU Site Subscription

Microsoft Azure



AWS Google Oracle



“We’ve rendered over 5 million jobs and still counting…” - ReelFX
AWS Google Oracle

All articles:

Renderfarms Past, Present and Future


While celebrating our 10th anniversary, CG Society asked PipelineFX CEO, Richard Lewis, for his thoughts on the history and future outlook of rendering and renderfarm management, as well as the growth expectations of this aspect of the industry.

PipelineFX has been celebrating its 10th anniversary, so it seemed like a good time to catch up with the company’s founder, Richard Lewis, and get his thoughts on the history and future outlook of rendering and renderfarm management, as well as the growth expectations of this aspect of the industry.

Rendering is still doing what it was doing 10 years ago: growing. Breakthroughs occur, expectations develop, and companies respond to meet them. But it is really through points of disruption that we have seen the biggest gains being made. For rendering, these have been: the introduction of the multi-core processor, the huge decline in the cost of both content creation and rendering software, and the rise of commercial render managers.

The advantages of the multi-core processor become abundantly clear when you stack up old costs against new ones. For instance, while working with Square Pictures on its Final Fantasy: The Spirits Within feature film (2001), I helped deploy a system that utilized more than 1,000 single-core Pentium 2 and 3 CPUs and several hundred MIPS processors to render, what was at the time, the single most complex CG movie ever made. That renderfarm was also enormously expensive, occupied an entire floor of an office building, and needed tens of thousands of dollars per month in electricity and cooling.

It is also a dinosaur by today's standards, as CPU technology has advanced to the point where, at a fraction of the cost, a small studio can attain rendering power equal to or greater than the kind Square had, thanks to things like high-density processors, memory density, and advancements in blade server technology.

The trend of better, faster, cheaper has extended to software as well, which has driven down the costs associated with creating content. I remember selling Alias PowerAnimator when the suite (modeling, animation, and rendering) sold for $45,000. If you add in another $50,000 for a tricked-out SGI Octane (which some companies did), you would quickly find yourself spending nearly $100,000 for every artist in the animation department. Enter the desktops of today, which are far more powerful and can be acquired with a suite of high-level software for 10x less than those historical prices. This has expanded not only the reach of rendered content, but the opportunities for professionals who can create it in a very real, and very tangible way, as well.

Images from NASA’s Conceptual Image Lab

Images from NASA’s Conceptual Image Lab

Free the System
The problem is these jumps in technical capabilities haven't always addressed pipeline efficiency at the same level we now see via commercial render managers. Even their rise was pre-dated by a period of disorganized R&D. When we started our company 10 years ago, there were just a few supercomputing apps, some open-source grid systems floating around, a few commercial render managers written and supported by individuals, and a lot of custom code. Studios were struggling with vendors that didn't understand digital media applications, and R&D departments were building and supporting custom render managers at great expense.

Today there are finally a few viable render managers for sale, and they are supported by teams of software engineers who come from digital media and understand both the apps and the render pipeline. Rendering has never been so accessible to so many, and this is due to the lower cost and increased support provided by commercial render management software companies and their products.

Images from NASA’s Conceptual Image Lab

Images from NASA’s Conceptual Image Lab

Look to Efficiency
At its best, an efficient render pipeline will minimize the ever-expanding render time and allow for more cycles of the creative iterative loop. By "creative iterative loop," I mean the entire design process from start to finish, including all the successive revisions that force companies to re-render each new step in order to get to their final product.

Being that this loop is the foundation of a higher-quality product, most companies and artists want that leeway. Or if quality is not the issue, then an efficient render pipeline allows an organization to take on more work with the same resources, leading to more profit. And profits, especially in the visual effects industry, are hard to come by these days. This makes rendering efficiency very important; or, as I like to say, "business critical."

Right now, we are at a point in rendering's history where the value of renderfarm management can be easily seen and felt by system admins. In the past, this wasn't the case-but over the last decade, we've seen several key improvements.

The new ArtistView GUI in Qube!

The new ArtistView GUI in Qube!

More studio insight into actual utilization.

      This is a simple education issue. Most studios don't really know how much of their computing assets are truly being used each day/week/month or per project. A common thing we hear from our customers in the beginning is, "We need more render nodes!" But most studios just need to use what they have more often and more effectively. There are a lot of things you can do before you start to actually need more hardware and software, and if your utilization rates are not over 90 percent, you don't need more computers.

Increased middleware integration with render management. It's only been recently that we've begun to see production tracking, asset management, preview/playback, and other pipeline-critical systems being integrated with render management. Most of these apps were traditionally developed in-house. Today, they are services offered by third-party software vendors like ourselves, Shotgun Software, Tweak Software, and so on, so companies can utilize their entire pipeline from the get-go, in the most efficient way possible.

Better and more training for admins, TDs, and artists. In 1986, I started an AutoCAD user group. I had to. There was only a Command Reference Guide at the time-no books, no classes at a community college, no private CAD or animation schools. We had to meet and tell each other what we had figured out on our own about the software. I feel like render pipelines are still at that stage now. Try searching Amazon.com for books on render pipeline engineering, or customizing your render pipeline, or anything about render pipelines, and you get nothing. PipelineFX was the first render management vendor to offer certification training in a render management application for this very reason-the knowledge needed to be spread. Through these classes, more than 250 technical professionals around the world have been certified. This is a start, but there are still thousands of organizations that depend on rendering every day and don't employ staff that have any specialized training in the subject.

The Future Is Cloudy
The search for new ideas around the pipeline has also become a big piece of the rendering landscape, in part because the industries that rendering touches have become more organized and more aware of how important rendering is to their day-to-day work processes.

The biggest trend we are seeing right now is cloud rendering-considered by many to be the next panacea. To be honest, the hype around the cloud reminds me of the hype around that "storage as a utility" idea which was touted as a paradigm shift around the year 2000. Remember when we were all going to plug Ethernet into the wall and get Tier 1 data storage at high-performance speeds for super-cheap prices? Never happened. All those companies went out of business.

The Internet, and the cloud, have not changed the rules of economics. There's price, performance, and quality. You only get two. If the cloud is fast (performance) and truly scalable with all the software available for rent (quality), then it's not going to be cheap; or if it's cheap, it won't have fast performance and high quality. For the most part, cloud rendering will be successful in 911 situations-times when you need 200 more machines for two months and you can't afford to buy them or find the time to set them up even if you could afford them. In that case, it's worth it to pay more (sometimes a lot more) to get the work done on time.

The cloud can also work for the occasional user of rendering, but if CG is your business, you are going to need in-house rendering capability so your capacity can be something you count on and use over and over again.

Western X discovered the advantages of using renderfarm management while outputting these furred animals.

Western X discovered the advantages of using renderfarm management while outputting these furred animals.

Offline Rendering Is Here to Stay
The enthusiasm shown for cloud rendering is really just indicative of how people approach technology, in general, nowadays. They want new, they want meaner and leaner, and they want to believe there is a silver bullet out there. In terms of rendering ....



To read original published article, please visit CGW.com

Comments are closed.