Fix for bug [#31613] Cycles 3d viewport material display mode + skin modifier related crash

When in material display mode, mesh_calc_modifiers() calculates the
orco DerivedMesh, which uses a different CustomDataMask. In
particular, it does not necessarily include the current modifier's
requiredDataMask, so those layers might get set to NO_COPY. For the
skin modifier, this resulted in a crash when the modifier internally
copies the DerivedMesh and the output does not contain the expected
MVertSkin layer.

Fixed by adding the requiredDataMask to the orco DM's CustomDataMask.

Also added a debugging function to customdata.c:
customData_mask_layers__print(CustomDataMask mask);

This will print out the names of all the CD layer types in the mask.
This commit is contained in:
Nicholas Bishop 2012-05-28 21:02:44 +00:00
parent d3c6c6babd
commit 3d98da9b0f
2 changed files with 14 additions and 1 deletions

@ -1651,7 +1651,9 @@ static void mesh_calc_modifiers(Scene *scene, Object *ob, float (*inputVertexCos
orcodm = create_orco_dm(ob, me, NULL, CD_ORCO); orcodm = create_orco_dm(ob, me, NULL, CD_ORCO);
nextmask &= ~CD_MASK_ORCO; nextmask &= ~CD_MASK_ORCO;
DM_set_only_copy(orcodm, nextmask | CD_MASK_ORIGINDEX); DM_set_only_copy(orcodm, nextmask | CD_MASK_ORIGINDEX |
(mti->requiredDataMask ?
mti->requiredDataMask(ob, md) : 0));
ndm = mti->applyModifier(md, ob, orcodm, app_flags & ~MOD_APPLY_USECACHE); ndm = mti->applyModifier(md, ob, orcodm, app_flags & ~MOD_APPLY_USECACHE);
if (ndm) { if (ndm) {

@ -1193,6 +1193,17 @@ static const char *layerType_getName(int type)
return LAYERTYPENAMES[type]; return LAYERTYPENAMES[type];
} }
void customData_mask_layers__print(CustomDataMask mask)
{
int i;
printf("mask=0x%lx:\n", mask);
for (i = 0; i < CD_NUMTYPES; i++) {
if (mask & CD_TYPE_AS_MASK(i))
printf(" %s\n", layerType_getName(i));
}
}
/********************* CustomData functions *********************/ /********************* CustomData functions *********************/
static void customData_update_offsets(CustomData *data); static void customData_update_offsets(CustomData *data);