It depends very much on your application. Each method has strengths and weaknesses, so you need to weigh up each one and decide from there.
A paginated list is easy to navigate to a specific point, and would probable be best where you have many items and someone is likely to need to jump to a specific one.
Infinite scroll is good for a simple interface where it is most likely that someone will want to see the earlier items, or where there aren't many items in the first place. Think of a blog here.
Clicking to show more is best in similar situation to infinite scroll, but where bandwidth is an issue. I only have to load up what I want to load up that way, but I have to click another button. You need to decide which will be more important or, even better, ask your customers.
I am facing a similar issue in an app I'm working on, and my current solution is to use a slider both for navigation and positional information. I have yet to test this out with customers, but I think the idea has merit.