UX has, and cost, performance, but UX design is separate from performance design.
should [we] start considering stuff like [...] performance issues as part of UX
since it does influence the overall user experience of the user.
As you state, performance and loading times affect User Experience. The answer to the yes/no-question is obviously yes. So the next question is, how should this fact affect how we work with UX?
If performance is UX, then what differs programming from design?
Imho, performance should be respected along with any other constraint. But there are many reasons to regard UX and performance separately:
- Performance is improved by optimisation, while UX is improved by design.
- Performance can be changed without changing the design.
- Performance depends on tons of factors, many of which are not in the UX field, and it boils down to a number of seconds or milliseconds. UX in general does not boil down to a number on my chart.
- Performance may be seen as a part of UX, but a more proper way to see it is that UX has a performance. UX design needs to take its performance issues into account, but designing for UX and designing for performance are two competing tasks.
Trading performance for UX
I am a programmer, and the further down the process I optimize, the better. If you are prioritizing optimisation of performance before most of the UX design is set and ready, you are doing something wrong.
To me, one part of UX is unloading the human mind, cognitively, and many times that means heavily burden the hardware. For example:
- doing a zillion calculations to present a relevant search result, instead of requiring the user to type search engine optimal keywords. This takes much longer than just displaying a bad result, like bad search engines do. So Google compensates this workload with quite performy hardware to still give a fast UX.
- animating a folder or program collapsing or expanding (1, 2), instead of letting the user herself figure out where it went or comes from.
Animations and calculations burden performance a lot but the amazing UX effects of a well designed animation or intelligent search result is indisputable. This is the trade-off we seek. UX cost performance. And we can only hope that our hardware is fast enough, and that our programmers are skilled enough, to match our UX ideas.
Yes, my job is to implement the design at a low performance cost. But performance can be improved later. If performance issues later require sacrificing UX, that should be respected along with the budget, deadline and other constraints. We can predict some of the performance issues at design time, and if so, of course use our knowledge about it. But it is much harder to know the exact performance constraints on before hand, especially when exploring new technologies and designs.
So in most projects: Design UX first, optimize performance later.