Qube

Render Farm Management for Digital Media Pipelines

 

 

Try It FREE 30 Days

 

EDU Site Subscription

Microsoft Azure

 

 

AWS Google Oracle

 

 

“We’ve rendered over 5 million jobs and still counting…” - ReelFX
AWS Google Oracle

All articles:

9 Tips for Rendering in the Google Cloud

1. Pick your "Zone"

Knowing your zone can help you control how fast your data is transferred to and from the cloud, how much you pay for transfers, which equipment may be available to you, and even your environment’s stability.

blog-google zones image

Your data transfers more quickly when it’s closer to you ­ just like driving your car from one spot to another. Since all cloud providers offer their resources by geographic region, you should start by looking at the region you are in or that is closest to you. Within Google regions, they offer zones, and you can select zones within a region that may be physically closer to you. The closer you can get, the faster your data will transfer and the less it will cost to complete those transfers. And if you have to transfer data from one cloud resource to another, know that transferring between zones in a region is cheaper than going across regions.

Learn about the zones available to you as it can affect the hardware and services available to you. Certain zones have Ivy Bridge or Haswell processors where others don’t. In addition, zones may have different quotas on the available services like firewalls, IPs, image storage, etc.. Last but not least, for maximum uptime, Google highly recommends having instances spread out to several zones in the same region because zone maintenance and outages should be expected.

2. Pick your Machine Type

Different machine types can offer a varying amount of processors or RAM which may render different applications more or less efficiently. It’s best to do a set of test renders of your typical work on several different types to find the sweet spot in terms of price and performance.

blog-cloud-machine type image copy

For scale, look at Instance Groups. You can define a template for your instance image, so adding more processors to your standard render farm image is as simple as increasing the number of instances for that template. You can even set the instances to scale up automatically as you need additional resources. Instance Groups automatically load balance between the resources in the group, and if an instance crashes, it will automatically reboot and pick up where it left off.

3. Leverage “Preemptible” Instances

blog-cloud-preemptible VM instances image

If you are scheduling render jobs that can afford the possibility of being paused or stopped for a period of time then Preemptible VM instances are a great bargain, running at a much lower price. By using these instances, you can save, on average, between 30­ - 50% of the hourly cost for rendering instances. You can save quite a bit of money across one or more projects, but with some added risk since there is no guarantee that the instance will be not be pre­empted.

4. Preview Jobs First

Optimize your use of cloud nodes. If you are going to be paying to use off site resources, then make sure the jobs are correct before you start the meter.

approved-red-stamp-5x3

If you are using our render management solution, Qube!, you can leverage our Preview Frames feature to automatically split a job into two parts: the first​ part​ test renders​ a sample set of frames immediately​ (locally)​​​​​,​ and​ ​the remaining frames are ​in a separate job which can optionally wait ​for user approval to continue (in the Cloud).

This way, you don’t start the meter on throw away work, and as you get information about the completed preview frames (render times, memory footprint, etc...), you can better determine how many cloud instances will be required to finish the project on time.

5. Don’t keep the meter running

A best practice in minimizing cloud costs is to pause or stop the instances when they are not in use. It can mean the difference between a large monthly bill versus the kind of cost you were expecting.

taxi-yellow cab

When rendering, it’s best to automate. For example, Qube! offers the use of pre­- and post­flight scripts that can be run at the start and end of every job. Write a script that starts your instance to run every single time a new job is dispatched to a cloud resource, and do the same to shut them down or pause them when your job completes. Your instances will only be up for rendering, and you will not be paying for cloud resources that are sitting idle.

6. Let the network expert get you connected

blog-cloud-vpn tunnel

Connecting your network to the Cloud requires some technical expertise that’s best handled by network engineers or system administrators. It’s not that it’s especially difficult, it’s just that it takes some time and testing. It’s getting better now that Google has added a more “out of the box” VPN product and there is a growing online knowledge base where users are sharing their experiences, including a forum on Rendering in the Cloud.

7. Cache is King

In any network environment, whether it’s on-premise or remote or a mixture of the two, the quality of experience and overall throughput depends on the strength and speed of your available cache. It makes an even bigger difference when you are using cloud nodes.

Avere’s vXFT is an extremely easy way to initiate the transfer of data in and out of the Google Cloud. blog-avere-vXFT image

Mount the Avere cluster along your file path, and your files will start transferring without changes to your pipeline. A standard three node cluster can serve well over 100 cloud nodes, and the proprietary algorithms insure the most used files are available as the priority on the fastest disk.

8. Bill More Effectively

man-graph-bill effectively
Because Google pricing is always online and available, you can forecast for a particular project how much rendering capacity you may need before you get started. And since pricing of cloud node usage is so much more straightforward than all of the baked in costs associated with on-premise rendering, you can estimate the work needed and make sure the actual, appropriate cost is passed on to your client and covered in the invoice. You can improve cost forecasting accuracy by using previous show render data to estimate the type of work needed on a new project. Costs may include the instance, storage, caching,  VPN, render license, and render management license.

You’ll cut down on comments like “did I estimate correctly?” or “I need to go back to the client with additional charges” or “our numbers don’t add up ­ we must be missing some cost for rendering.”

9. Test now!

Sign up for your free Google Cloud Platform trial and TEST NOW!. You want to test the Cloud and figure out your integration BEFORE you have a desperate need for more render power. Once tested you’ll be ready when the need arises so don’t wait!

Just_the_little_Qube_Cloud

For more information on rendering in the Cloud see:

Comments are closed.