The simplest questions often have complex answers. If you’re making a game, it sounds like you may be making a custom interface, so you probably need some general principles to guide your control design. Whether to trigger an action on mouse down or mouse up depends on the control and the action and how the user will interact with it. This is probably why your library gives you a choice. To decide, balance efficiency (making input fast and easy) with tolerance (making input error-resistant or easily corrected) for the particular case. The key questions are:
What should dragging do? Even if you have no drag-and-drop functionality, you still have to decide what dragging will do.
What should holding the button down do? Again, even if no command is made, there often should be something happening.
Generally if mouse selection merely changes focus, then focus changes on mouse down, not mouse up. This is good for efficiency, allowing the user to use dragging to complete some kind of command without having to “re-grip” the object (before an object can be dragged, it must have focus). It presents little tolerance issues since accidentally changing focus generally is easily corrected and doesn’t change any underlying data. Obviously, if the UI responds to mouse down, then you have to take the position of where the mouse pointer was on mouse down. Windows, icons, database records, slider controls, and text boxes all receive focus on mouse down.
On the other hand, if mouse selection completes a command or changes a data value, then tolerance is more of an issue. In those cases, generally the command/change triggers on mouse up. I see no reason to put a time limit on when the users must mouse up to trigger the command -let them think as long as they want.
Triggering on mouse up allows you to use dragging to cancel the command or change to another command. Activation of controls like command buttons, check boxes, and links are all canceled by dragging the mouse off the controls. Dragging-off-to-cancel also works for pulldown and dropdown menu items. Users can also change their selection of the menu items by dragging to another item while holding the mouse down. In this case, take the position at mouse up so you execute the correct command or can tell if the user is canceling. Since dragging itself is not used to execute a command, there is no appreciable efficiency loss, but tolerance is improved.
The one exception is for cases where you want to support auto-repeat for a command, where holding down the mouse button on the control repeats the command at about two times a second. Here logically the command must be triggered on mouse down. Scrollbar buttons, the scrollbar track, and spinner buttons all work this way. It would also make sense for buttons for Zoom In/Out or Find Next/Previous. The efficiency gains from auto-repeat are obvious, but to maintain adequate tolerance, it should be reserved for actions that are easily reversed (such as scrolling and Find Next/Previous). Auto-repeat for Delete sounds great for clearing spam out of a user's inbox, until the user deletes 15 important emails by accidentally setting the phone down on the mouse.
Generally, selection that puts a control in an intermediary state should occur with mouse down. For example, while activation of menu items occur on mouse up, pulldown and dropdown menus open on mouse down rather than mouse up. This provides feedback on the user’s action and also allows the user to immediately drag to select an item, making for efficient input. It also allows Mac users to use Windows menus in the manner they’re used to. Tolerance is not a big issue since simply opening a menu is easily reversed and effects no command or value change. An exception is the context menu, which opens on mouse up in Windows. That’s because MS wanted to allow the user to drag and drop objects with the right mouse button (try it in Windows Explorer or Word –it’s pretty cool).
If a command triggers on mouse up, you often should provide some sort of intermediary feedback on mouse down so the user realizes what command they’re about to execute. For example, command buttons appear “depressed” on mouse down. So while a command executes on mouse up (and thus you need the mouse position at mouse up), you also need to track wherever you've provided feedback on mouse down so you can clear it on mouse up -including if the user has moved the mouse off the control on mouse up. Feedback on mouse down may not be necessary if you already show such feedback with mouse hover.
Appropriate use of mouse down and mouse up can do a lot to make a UI feel fluid and “intuitive,” which is probably important for making your game fun. Most users probably couldn’t tell you how a particular control responds to mouse down and up, but good design still benefits them. I think most users will drag off a button to cancel its activation. They don’t even think about it -it just seems natural to them, even though this doesn’t work with natural (physical) electric buttons –they activate once depressed. I've never seen a user attempt to activate a command button by clicking outside a button and sliding on to it, even though that would work with a physical button. "Intuitive" can be weird.