For starters dont make it so obvious that an alert message or an important notification about the user potentially making a mistake can be just skipped in the button clicking frenzy but think about how you can highlight the error or concern so that users do notice it.To quote the smashing magazine article
Pretty much every application has some form of feedback messages.
These are little messages that pop out when there is an error or
warning or perhaps when an action is completed successfully. Designing
these messages correctly is important because you don’t want to
confuse or startle your users when there’s nothing to worry about. A
good practice here is to do a couple of things. First, color code the
different types of messages. Messages that notify users of successful
actions are usually colored green. These employ the traffic-light
analogy of green meaning “Go.” Warning and error messages are colored
yellow. Same traffic-light analogy here: yellow means slow down and
wait. You can also distinguish between warning messages and error
messages by coloring errors red and warnings yellow.
The second thing to do is add a unique icon for each message type.
Icons can convey meaning instantly, without the user having to read
the message. For example, a tick icon can symbolize completion of a
successful action. An exclamation mark in a triangle is a warning
sign. People will instantly recognize that this message warns them
about something and will pay attention.
That said since the action happens once a year or not at very regular intervals, having a confirmation dialog should be fine as you need to inform users of the potential dangers of performing that action.To quote this ux matters article
When a user chooses a command that can destroy the user’s data, always
display a confirmation message box
Lastly I recommend reading this excellent article on how to write effective communication messages. To quote the article
A good confirmation main instruction must require thought. Remember,
the entire point of an effective confirmation is to give the user a
reason not to proceed, so that reason should be clear. (Making the
generic “Are you sure?” confirmation clearly pointless.) The risk of
the action or the unintended consequence should be extremely clear.
Provide all the information
A general rule: whenever you ask the user a question, you provide all
the information required to provide an informed answer. Be sure to
give specific risks, object names, contexts, etc. required to respond
in the confirmation itself.
Design responses to create a mental speed bump
Ordinarily, a good dialog box is designed for efficient decision
making, where users can often respond without reading anything but the
response buttons. While this works well in general, the entire point
of a confirmation is to get the user to stop and think about whether
or not to proceed. (If proceeding requires no thought, don’t confirm!)
There are two facts about user behavior that we need to work with
- We can’t force users to read the main instruction.
- Users do read button labels before they click them.
Given these facts, there are three ways we can write the response
button labels to create a mental speed bump:
- Choose specific responses so that the reason to not proceed is obvious even without reading the main instruction.
- Use Yes/No responses (with the safe response the default). While there’s no guarantee, users generally read what they are responding
Yes or No to before clicking.
- Add “anyway” to the label, as in “Proceed anyway.” Anyway is a very provocative word that gets users to stop and think. “You mean there’s
a reason to not proceed? Better find out what it is first.”