Production Diary

Tools, tips
and techniques

Techniques, tips, and code documented during the course of production work. VEX snippets, Python automation, and FX breakdowns from real film projects.

January 2017 · VEX · Production Diary
Houdini VEX Snippets
50+ production-tested VEX snippets - spirals, point clouds, matrices, volume operations, culling, and more. Bundled as wrangle presets for day-to-day efficiency.
May 2014 · Python · Production Diary
Houdini Python Snippets
Python automation for production tasks - batch parameter changes, bundle management, render farm job control, and node graph manipulation.
June 2014 · VEX · FX Breakdown
Pyro Clustering with VEX
Splitting pyro containers into sub-domains using VEX - a technique used on Jupiter Ascending for simulating and rendering smoke trails at scale.
October 2018 · VEX · Crowds · RnD
A Few Notes on Houdini Crowd Simulations
Production notes on ragdoll setups, bullet solver hybrids, constraint networks, agent attributes, and rotation limit configuration.
September 2017 · VEX · Wire Solver
A Few Wire Object Notes
Key attributes for wire simulations - klinear, kangular, damping, targetstiffness - and how to remap them along curves for art-directable results.
July 2014 · RnD · FX Breakdown
Pebbles and Dust
RBD, pyro, and particle dynamics interoperability in Houdini DOPs - debris library instancing, dust emission, and multi-solver collision strategies.
Production Diary

Houdini VEX Snippets

January 23, 2017

I typically bundle these snippets as presets in a wrangle node as I get to use most of these functions from day to day. This generally gives me a good head start for similar wrangle tasks and also aids my efficiency.

Spiral From Line

//SPIRAL FROM LINE
//CIRCLE OF REFERENCE AND LINE MUST LIE IN THE SAME PLANE
//(ZX) PLANE IS SUGGESTED
//RADIUS OF CIRCLE SHOULD IDEALLY BE THE SAME AS LENGTH OF LINE
//Use uvtexture node to compute uv attrib on points
//Texture type is set to Row & Columns
//Get velocity vector along line
int total = npoints(0)-1; //last point
int after = total-1;//penultimate point
int ntm = @ptnum+1;
vector npos = point(0,"P",ntm);
v@v = @P-npos;
//get vel of last point
if(@ptnum==total){
 v@v = point(0,"P",after)-point(0,"P",total);
}
v@v = normalize(v@v);
v@up = chv("up");
v@side = cross(v@v,v@up);
//Matrices
matrix3 xform = ident();
float freq = chf("frequency");
float angle = @uv.x * freq;
vector axis = v@up;
rotate(xform,angle,axis);
@P*=xform;

Limit the Velocity of Crowd Agents

//Modifying the 'v_agent' array vector attribute
//Use inside a SOP Solver
vector myvel[] = v[]@v_agent;
float mult = chf("mult");
foreach(int i; vector k; myvel)
{
    if(length(k)!=0.0)
    {
        if(length(k)>chf("max_speed"))
            continue;
        else
            myvel[i] = myvel[i] * mult;
    }
}
v[]@v_agent = myvel;

Flow Vector Along Geo (Cross Product)

vector up = set(0,1,0);
vector side = normalize(v@N);
vector flew = cross(side,up);
vector flow = cross(side,flew);
v@side = flow; //visualize flow vector

Attribute Transfer Color from Second Input

int handle = pcopen(@OpInput2, "P", @P, chf("rad"), chi("num"));
vector lookup_P = pcfilter(handle, "P");
vector lookup_Cd = pcfilter(handle, "Cd");
i@many = pcnumfound(handle);
if(i@many>0){
     @Cd = lookup_Cd;
     v@P = lerp(v@P, lookup_P, chf("mix"));
}

Rest Position from (Array) Name Attribute

//Set rest position from array
//This two part code uses two attrib wrangle nodes
//It generates an array that contains the name attribs
//of selected pieces and stores this array attrib on a single point

//Part 1: Store names
i[]@pts = pcfind(1, "P", @P, chf("rad"), chi("num"));
string nnme[];
s[]@nme= nnme;
string tok;
foreach(int a;@pts){
  tok = prim(1,"name",a);
  push(@nme,tok);
}

//Part 2: Set rest position
string names[] = point(1,"nme",0);
foreach(string nme; names){
  if(s@name==nme)@P = v@rest;
}

Quaternion Rotate

v@N = set(0,1,0);
matrix3 xform = ident();
float amt = chf("amt");
vector axis = set(chf("axisx"),chf("axisy"),chf("axisz"));
rotate(xform, amt, axis);
p@roti = normalize(quaternion(xform));
v@up= qrotate(p@roti,v@N);
v@N*=xform;

A Random Rotation Matrix

matrix3 myrot = ident();
float amt = fit(rand(@id+chf("seed")),0,1,chf("minamt"),chf("maxamt"));
vector axis = sample_direction_uniform(rand(@id));
rotate(myrot, amt, axis);
@P*=myrot;

Shrink Wrap

int handle = pcopen(@OpInput2, "P", @P, chf("rad"), chi("num"));
vector lookup_P = pcfilter(handle, "P");
i@many = pcnumfound(handle);
if(i@many>=1)v@N = @P-lookup_P;
else v@N = set(0,0,0);
@P-=v@N;

Cull Based on Ray Direction

int handle = pcopen(@OpInput1, "P", @P, chf("rad"), chi("num"));
vector up = {0,1,0};
vector dir,pos;
float angle;
i@many = pcnumfound(handle);
while(pciterate(handle)){
    if(i@many>1){
        pcimport(handle,"P",pos);
        dir = pos - @P;
        angle = dot(normalize(dir),up);
        if(angle>chf("mix"))removepoint(geoself(),@ptnum);
    };
};
pcclose(handle);

Rotate by Matrix

float xrotamt = radians(chf("xrotamt"));
float yrotamt = radians(chf("yrotamt"));
float zrotamt = radians(chf("zrotamt"));
vector min, max, centroid;
vector xrotaxis = set(1,0,0);
vector yrotaxis = set(0,1,0);
vector zrotaxis = set(0,0,1);
getbbox(0, min, max);
centroid = (max + min) * 0.5;
3@xform = ident();
rotate(3@xform, xrotamt, xrotaxis);
rotate(3@xform, yrotamt, yrotaxis);
rotate(3@xform, zrotamt, zrotaxis);
v@P -= centroid;
@P = @P * 3@xform;
v@P += centroid;

Move Points to Surface (SDF)

float vsample = volumesample(1,0,@P);
vector dirtosurf = volumegradient(1, 0, @P);
@P -= normalize(dirtosurf) * (vsample - chf("offset"));

Quantize Position - Cubify

float ps = chf("scale");
@P.x = rint(@P.x/ps) * ps;
@P.y = rint(@P.y/ps) * ps;
@P.z = rint(@P.z/ps) * ps;

Volume Frustrum Cull

vector pndc = toNDC(chs("camera_name"), @P);
float pad = .2;
if(pndc.x<0-pad || pndc.x>1+pad || pndc.y<0-pad || pndc.y>1+pad || pndc.z>=0){
  @density=0;
}

Velocity Along Path

int total = npoints(0)-1;
int after = total-1;
int ntm = @ptnum+1;
vector npos = point(0,"P",ntm);
v@v = @P-npos;
if(@ptnum==total){
 v@v = point(0,"P",after)-point(0,"P",total);
}

Get Centroid

vector min, max;
getbbox(0, min, max);
vector centroid = (max + min) * 0.5;

Group Roots and Tips of Curves

i@root = (vertexprimindex(0, @vtxnum) == 0);
i@tip = (vertexprimindex(0, @vtxnum) == (@numvtx-1));
setpointgroup(0, "roots", @ptnum, @root, "set");
setpointgroup(0, "tips", @ptnum, @tip, "set");

Orient Attribute from Normal

matrix3 m = dihedral({0,0,1},@N);
@orient = quaternion(m);

Simplex Noise on Velocity

v@v *= chf("vel_scale");
vector freq = chv("freq");
vector offset = chv("offset");
float amp = chf("amp");
vector nn = xnoise(@P * freq + offset);
nn = fit(nn,{0,0,0},{1,1,1},{-1,-1,-1},{1,1,1});
nn *= amp;
v@v += nn;

Connect Points

//Run over detail
int num = npoints(geoself());
int vtx_limit = num-1;
for(int i=0;i<vtx_limit;i++){
    int prim = addprim(0,"polyline");
    addvertex(0,prim,i);
    addvertex(0,prim,i+1);
}

Reduce Points

if(rand(@id)<ch("kill"))
    removepoint(geoself(), @ptnum);

Tighten Points

int handle = pcopen(@OpInput1, "P", @P, chf("rad"), chi("num"));
vector lookUp_P = pcfilter(handle, "P");
v@P = lerp(v@P, lookUp_P, chf("mix"));

Delete Lone Points

int handle = pcopen(@OpInput1, "P", @P, chf("rad"), chi("num"));
i@many = pcnumfound(handle);
if(i@many<chi("num"))removepoint(geoself(),@ptnum);

Scale by Matrix

vector p = getbbox_center(0);
float sx, sy, sz;
sx = sy = sz = chf("scale");
matrix idan = maketransform(0,0,{0,0,0},{0,0,0},set(sx,sy,sz),{0,0,0});
@P -= p;
@P *= idan;
@P += p;

Cull Points Based on Speed

float vlen = length(v@v);
if(vlen<chf("speed"))removepoint(geoself(),@ptnum);

Interpolate Line Into Circle

//Input is a line, resample for more points
//UV texture SOP as point attrib, rows and columns
int num = npoints(0);
vector lineStart = point(0,"P",0);
vector lineEnd = point(0,"P",num-1);
vector lineDir = lineStart-lineEnd;
float l = length(lineDir);
vector up = set(0,1,0);
vector side = cross(normalize(lineDir),normalize(up));
if(dot(up,normalize(lineDir))<0) up=set(1,0,0);
float t = @uv.x;
float pivot = chf("pivot");
float lineLength = length(lineDir);
float bendFactor = chf("bend_Factor");
float circleRad = lineLength / (2 * PI);
vector circleCenter = lineStart + (-lineDir * pivot);
circleCenter += up * circleRad;
float angle = PI + bendFactor * (1.0 - (t+pivot)) * 2 * PI;
vector posOnCircle = circleCenter + set(cos(angle),sin(angle),0) * circleRad;
@P = lerp(@P, posOnCircle, bendFactor);

Look At: Generate Radial Velocities

v@v = set(1,0,1);
vector pp = getbbox_center(geoself());
matrix3 mm = lookat(pp,@P);
@v *= mm;

Bend Curve

//Activate curveu attrib in resample SOP
@curveu = chramp("ramp",@curveu);
float bamt = chf("bend_amt");
vector benddir = chv("bend_dir");
@P += benddir * bamt * @curveu;

Median Point Number

int npt = npoints(0);
if(npt%2==0)
    i@midpt = (npt/2) -1;
else
    i@midpt = (int(ceil(float(npt)/2))) -1;

Point Cloud Density

float rad = chf("rad");
int num = chi("num");
int handle = pcopen(@OpInput1, "P", @P, rad, num);
vector lookup_P = pcfilter(handle, "P");
int many = pcnumfound(handle);
i@many = many;
f@viz = float(many)/float(num);
@Cd = vector(f@viz);

Transform by Primitive Intrinsic

matrix xform = primintrinsic(@OpInput2, "packedfulltransform", 0);
@P *= xform;
Production Diary

Houdini Python Snippets

May 4, 2014

This is an attempt to document and share a few of the Python code that helps in automating several tasks during the course of production.

Create a Primitive Group for Each Polygon

node = hou.pwd()
geo = node.geometry()
for primm in geo.iterPrims():
   grpName = geo.createPrimGroup('Gurupu_' + str(primm.number()))
   grpName.add(primm)

Set Keyframe Values as Point Attributes

node = hou.node('/obj/CRACK_SIM_7/carve1')
geo = hou.pwd().geometry()
parm = node.parm("domainu1")
firstkey = int(parm.keyframes()[0].frame())
secondkey = int(parm.keyframes()[1].frame())
first = geo.addAttrib(hou.attribType.Point, "first", 0)
second = geo.addAttrib(hou.attribType.Point, "second", 0)
for pt in geo.points():
    pt.setAttribValue(first, firstkey)
    pt.setAttribValue(second, secondkey)

Change a Parameter in All Mantra Nodes

parent = hou.node("/obj/ROPS")
for node in parent.children():
   node_type_name = node.type().name()
   if node_type_name == 'mantra':
      param = node.parm("matte_objects")
      param.set("endurance render")

Pattern-Based Node Operations

parent = hou.node("/obj/ROPS")
sstring = 'thrusterA'
temp = 'endurance_render'
for child in parent.children():
   if sstring in child.name():
      param = child.parm("matte_objects")
      if param is None:
         pass
      else:
         param.set(temp)

Remove a Node from a Bundle

node = hou.node('/obj/test')
bundlename = 'testy'
for nn in hou.nodeBundles():
   if nn.name() == bundlename:
      if nn.containsNode(node):
         nn.removeNode(node)

Replace String Subsets in Parameters

def forceObjects():
   for node in hou.selectedNodes():
      param = node.parm("forceobject")
      searchStr = param.evalAsString()
      matchStr = "thrusterDelayedBB"
      replaceStr = "thrusterDelayedBC"
      if matchStr in searchStr:
         replStr = searchStr.replace(matchStr, replaceStr)
         param.set(replStr)

forceObjects()

Rename Alfhou Nodes to Ancestor Names

import hou
for node in hou.selectedNodes():
   ans = node.inputAncestors()
   suff = ans[0].name()
   node.setName("alfhou_" + str(suff))
FX Breakdown

Pyro Clustering with VEX

June 29, 2014

'Clustering a pyro container' means splitting the container into a number of sub-domains rather than using one single container. I used this technique while simulating and rendering smoke trails on Jupiter Ascending.

The logic behind this technique is quite simple and interesting. The basic idea is to have the clustered containers created only once and at specific points during the simulation, then the pyro solver takes care of their auto-resizing/expansion up until the next creation point/frame.

While working on this in London, I used VOPS to implement the algorithm. However, I tried to understand the logic and re-implemented it in VEX using the point wrangle node.

Pseudocode

compare present cluster value to the cluster value on the next frame
if (cluster value at next frame == value at present frame)
{
  set the 'deleteme' flag to 1;
}
else
{
  set the 'deleteme' flag to 0;
}
//First frame check:
if (current frame == first frame)
  set the 'deleteme' flag to 0;

VEX Implementation

float sframe = ch("./sframe");
float cframe = @Frame;
int thisCluster, nextCluster;

addattribute("sframe", sframe);
addattribute("cframe", cframe);

import("cluster", thisCluster, 0, 0);
import("cluster", nextCluster, 1, 0);

addattribute("this", thisCluster);
addattribute("next", nextCluster);

if (thisCluster == nextCluster)
{
  addattribute("deleteme", 1);
}
else
{
  addattribute("deleteme", 0);
}

//First frame check
if(cframe == sframe)
  addattribute("deleteme", 0);

addvariablename("deleteme", "DELETEME");

In this implementation, I created a custom parameter that represents the first frame. The algorithm compares the cluster attribute value at the current frame to the value at the next frame. If they match, the instance point is flagged for deletion - it's not needed because the container already exists. The first frame is always preserved.

Production Diary · RnD

A Few Notes on Houdini Crowd Simulations

October 25, 2018

Ensure your collision layers have a sensible and accurate representation.

Ensure your 'agent configure joint' has a sensible and accurate representation of rotation limits.

A Ragdoll sim is a crowd solver / bullet solver hybrid with the bullet solver controlling the ragdoll behaviour.

The heading and up vectors are used to ensure the initial orientation of the agents. If you have an initial velocity, it overrides the heading.

Calculate Orient from Normals

matrix3 m = dihedral({0,1,1},@N);
@orient = quaternion(m);

Calculate Heading and Up from Orient

matrix3 rr = qconvert(@orient);
v@up = set(rr.yx, rr.yy, rr.yz);
v@heading = set(rr.zx, rr.zy, rr.zz);

Agent attributes can directly be modified during a simulation within a SOP solver.

Increasing the number of substeps on the bullet solver helps with weird stretching of ragdolls.

The Hard Constraint Relationship (usually Pin Constraint) and the Cone Twist Constraint Relationship hybrid is used because rotation limits from Agent Configure Joints set up cone twist constraints. For any transforms where rotation limits were not set (or if there are multiple root collision shapes), pin constraints prevent the ragdoll from immediately separating.

Because constraint attributes set in SOPs act as multipliers to the DOP parameters with the same names, a good workflow is to set the DOP parameters to one and set the desired values with the matching SOP attributes.

Reducing the "Error Reduction Parameter" on the Hard Constraint Relationship seems to get rid of the undesired stretching of ragdolls.

Initialize Basic Agent Attributes

vector @v_agent[];
vector @w_agent[];
vector v = chv("vel");
vector w = radians(chv("angvel"));
int n = agenttransformcount(0, @primnum);
for (int i = 0; i < n; ++i)
{
    append(@v_agent, v);
    append(@w_agent, w);
}

Primitive Attributes on the Constraint Network

@softness = chf("softness");
@max_up_rotation = chf("max_up_rotation");
@max_out_rotation = chf("max_out_rotation");
@cfm = chf("constraint_force_mixing");
@bias_factor = chf("bias_factor");
@relaxation_factor = chf("relaxation_factor");
Production Diary · RnD

A Few Wire Object Notes

September 29, 2017

Visualize the width of the wire object(s) and be sure it's a reasonable size.

Wire points must be reasonably spaced apart. Not too many points and not too few points.

Some key basic attributes that allow better manipulation of the wire object in SOPs are: width, klinear, damplinear, kangular, dampangular, targetstiffness and targetdamping.

Think of the wire object as having two distinct behaviours - stretching (klinear) and bending (kangular). Depending on the material, the wire object can oscillate while stretching (damplinear) and/or oscillate while bending (dampangular).

Use a uvtexture node to get a normalized mapping of values (0-1) along the wire object. Set Texture type to 'Rows & Columns' and attribute class to point.

Remap UV to Attribute

f@f_uv = 1 - @uv.x;
f@f_uv = chramp("uv_ramp", f@f_uv);

If you notice a wiggly behaviour on the wire object, it is most probably because of the oscillating forces. Either the wire is oscillating too much while stretching or bending. Introduce some damping - damplinear or dampangular as the case may be.

If you want the wire object to be reluctant to any of the forces, play with the targetstiffness and targetdamping attributes. The wire will then quickly go back to rest. targetstiffness will make the wire reluctant to move from its initial position while targetdamping will make it appear reluctant to oscillate.

Create Multipliers for Key Wire Attributes

f@klinear = fit(f@f_uv, 0, 1, chf("min_klinear"), chf("max_klinear"));
f@damplinear = fit(f@f_uv, 0, 1, chf("min_damplinear"), chf("max_damplinear"));
f@kangular = fit(f@f_uv, 0, 1, chf("min_kangular"), chf("max_kangular"));
f@dampangular = fit(f@f_uv, 0, 1, chf("min_dampangular"), chf("max_dampangular"));
f@targetstiffness = fit(f@f_uv, 0, 1, chf("min_targetstiffness"), chf("max_targetstiffness"));
f@targetdamping = fit(f@f_uv, 0, 1, chf("min_targetdamping"), chf("max_targetdamping"));

Remember to substep when needed (also 'Max Collision Resolved Passes' on the wire solver).

RnD · FX Breakdown

Pebbles and Dust

July 24, 2014

This RnD demonstrates the use of Rigid Body Dynamics, Computational Fluid Dynamics, Particle Dynamics and their interoperability all within Houdini DOPs.

Rigid Body Dynamics

My previous RBD simulations suffered from the debris pieces being perfectly flat and polygonal around their surfaces. After generating and caching my initial debris simulation, I used the for-each SOP and the corresponding 'name' attribute to apply a bit of noise (Mountain SOP) and subdivision to the individual debris. This made them look a bit more rocky and less flat/polygonal.

Smaller Pebbles and the Pebble Library

To add more variation, I took the original debris simulation into a for-each SOP, scaled them down and moved their centroids to the origin. Using the file SOP, I cached about 20 variations to disk, suffixing them with 'opdigits' - pebble1, pebble2, pebble3, etc.

These smaller pebbles were then brought back into the RBD simulation with a RBD Point object by adding an instance attribute onto a point cloud:

/obj/pebble_library/pebble`trunc(fit(rand($PT),0,1,1,21))`

The point cloud was generated by scattering points in a region for the initial pebble position. The total number of scattered points determines the total number of pebbles in the system.

Fluid Dynamics: Dust Simulation

The Pyro Solver was used for the dust emitted with the debris. I always prefer using the pyro solver over the smoke solver as it offers more flexibility and controls for the shape and character of simulations.

To isolate the source debris (pieces that fell to the ground), I used an expression in the delete SOP: $TY<0.1 - selecting only pieces below 0.1 units. The desired pieces are sent into the fluid source SOP and cached.

To get dust-like results: pump in just a tiny bit of viscosity and bring down the turbulence significantly. I also keyframed the buoyancy so the smoke begins to rise only when debris has hit the ground surface.

RBD / Fluid Collisions

For rigid body elements to collide properly with emitted smoke, the simplest/faster approach is to use a merge DOP with its collision property set to 'mutual'. A more precise approach would be generating a collision field from the debris and plugging it into the pyro solver.

Particle Dynamics

For grit particles interpreted as sand, I used the POP/DOPs interaction with the POP object. Particles were emitted off the RBD debris, inherited part of the debris velocity, and had gravity applied - making them move downward in the direction of the falling pieces.

Particle / RBD Collision Strategy

Two problems arose when merging the particle stream into the rigid body stream: particles kept sliding on the ground plane, and particle velocity was affected by debris velocity (causing explosions).

Fix for sliding: set the static friction value for particles to about 2 on the POP object.

Fix for velocity interference - use three merge DOPs:

merge 1: particle -> ground plane : collide relationship = mutual
merge 2: debris -> ground plane : collide relationship = mutual
merge 3: merge1 -> merge2 : collide relationship = No change