In a hardware button there are multiple variables to consider, including materials, resistance of such materials to pressure, elevation of the button (more elevation equals to higher reaction times) and so on. You'll also need to consider the location of the button, which fingers will be used to activate such button and many more.
The point is that there isn't a formula, but many formulas (such as Fitt's law or Accot–Zhai steering law ), and of course, direct measure of user interactions on real life prototypes.
Once you solve the problem, the 300 ms time you mention (as in the answer from Luke Sawczak) is an industry standard, and you should get as close to that number as possible.
If you value accessibility, it is recommended to choose a slightly higher time (even if you add a bit more resistance) to help people with physical and neurological disabilities. Keep in mind many users can't control their movements, so this is a very important concern. You can adjust the time from within the interface, of course.