I'm a fullstack developer rather than a designer, for me, the UI design is matter of hobby. I always work with agile methodologies, mostly of time, Kanban. The User Story is just a feature of the software and generally does not contain any prototype of how it can be or look.
Each User Story travel across some stage. For example, in SCRUM and Kanban, the first stage is Development. But this stage doesn't means only code, this stage covers from design to implementation. Is normal to divide a User Story into small tasks, this tasks must be estimated in points or time and this causes the User Story value.
So, once your User Story to development into your Sprint backlog, the user changes to this (for example):

As you can see, the User Story needs divide in tasks, so the team assign one member to one o more tasks needed to complete it, and of course, include designers.
Remember: the team must work together even if are distinct areas. This assures you a solid software.