Hide Button by Classification

This seems straightforward, but I'm not having any luck.

We've modified the Document ItemType with an additional Classification category to link out to documents produced in another system.  For this, we want to hide the "Open in Office" button.

Have followed the CUI Admin Guide and the Configuring Solutions guide for Removing a button, with the modification of inputting the classification into the Command Bar Item "Classification" field.

But that doesn't remove the button...  Is there somewhere else this goes or how can this be done?

Thanks!