Actually, you have 3 visual states: MouseDown(ActiveClicking), Checking and Unchecking. In WPF the action command is usually fired at MouseUp. For obvious reasons like DragAndDrop, which is pretty common at Desktop Apps.
MouseDown - Checking and Unchecking: gives users a blue indicator where the action takes place. Regardless of which interface you use: mouse pointer, touch or keys.
Mouse Up - Checking: shows the final state.
Mouse Up - Unchecking: shows the final state and the blue indicator, where the action might take place (important for control by use of keys).
This might look confusing at first glance, but only if one looks from web perspective at it. In web realms one is used to OnClick(MouseDown) as start the action command. But this might change as it has drawbacks with DragAndDrop and Touch (hold and gesture).
We might learn - it makes developer life easier if the command fires only at MouseUp/onMouseUp.
And other (well-known) learning - blue is a good indicator for users with color vision difficulties. And indicator for the active one (out of all interactive elements) is a must for keyboard useage and good for overall orientation.
And - it promotes learnability of interactive elements as you can try click and hold/touch and hold in order to see how it reacts (DragAndDrop or context menu), but you are able to cancel the intended click by move mouse away and release button upon a not clickable area.