If I understood correctly the premises of the question, what the user should decide is:

 1. If the user wants to consider one of the following options
 2. If the answer to 1 is Yes, then choose one of the options

The possibly outcomes could be:

 - Option A
 - Option B
 - Not applicable

**Original proposal**: [[demo]](https://jsfiddle.net/4nh2795s/show/)

Considering you are using [toggle buttons](https://material.io/guidelines/components/buttons.html#buttons-toggle-buttons), for the third outcome you could give the possibility to unselect both options (adding an extra note to clarify this possibility or an *"(optional)"* next to the label).

<img src="https://i.sstatic.net/6W3J3.png" width="280" />

**Second proposal**:
Make the user decide 1 and then 2. The checkbox enables/disables the buttons. In this case the toggle buttons can act as radio buttons where one needs to be selected.


<img src="https://i.sstatic.net/DxddX.png" width="280" />