If attendees have the right to not vote, simply check the "Abstain may" option in the voting details pane. Attendees will then be able to opt to abstain when giving their response.
​

⚠️ Only the editor to modify the vote.