It seems to be a simple problem, but i cant figure it out
Lets say, I would like to know if there is some point to implement new feature. If we have to focus on the feature or not. Lets assume there is no possible some kind of test like questioning the users or whatever else. Its function will be easy, something like - for example "webcam for ecommerce for users that are paying premium account".
To be specific, I have 1500 premium users. I can tell "Feature is used when atleast 75% of clients use it". Great! We would like to run Fake Door test, where we implement just the button for webcam and when user click on it, we show him "we are implementing this feature right now, stay with us" or whatever else (i know, fake doors isnt the best method, but it is not the point of this). I will "test" it for 14 days. In 14 days, 350 clients will come on my site and they see this feature. 265 of clients clicks on the button.
What can I say about this feature? It seems like I can say "Yes, we have to implement it, because 75% of users will use this feature" (75% of 350 is 262.5 < 265) => H0 (Atleast 75% use this feature) seems to be ok. But it is not truth at all. Because there can be HUGE error (I tested ONLY around 23% of clients).
What I am trying to achieve is:
I would like to say - "With 95% confidence, 75% of clients will use this function, so we can implement it".
I am lost of all confidence intervals, confidence levels and sample sizes, etc etc. Can someone help me how to get the confidence step-by-step and explain me, what can I count from those numbers (1500 premium users at all, 350 users saw the feature, 265 users used the feature).