It depends: where do you expect the application to be used?
If most users will be in a relatively quiet, indoor environment, the conference room should be close enough, in my opinion--even if it's not exactly the same (at home vs. work, for example).
However, if the expected environment of use is quite different, testing in a conference room won't be adequate.
Imagine, say, a mobile travel guide app. It might work great in your test. But then in real life, you might discover:
- Small controls on the interface worked fine when the phone was sitting on the conference room table, but not when navigating with one hand while walking.
- The color scheme isn't readable in bright light outdoors.
- Audio features are useless because of traffic noise.
Consider the features and expected use of your app.