Identify how the UI should work, and see where it fails to do that
You essentially have to hold your site up against a definition of proper function, and see where it falls short.
Use a list of usability heuristics to evaluate the essential function
In 1994 the N/NG released a list of 10 Usability heuristics. These are basic rules of thumb, guidelines to be used to assess if your site is checking the most basic of boxes.
This article gave rise to a legion of related articles that expand on these points with examples. Here are just a couple:
Find and use a comprehensive UI review list
There's an exhaustive, if a little dated, list of UI components and they should function at this link:
With a little searching you might find a more modern list. In any case, you need to go through your site and see where it's failing to meet the standards described.
A point about 'ugliness'
You'll note that none of the above refers to how ugly or pretty the site is. Yes, when you drill into the lists you'll find points about aesthetics, but ultimately it doesn't matter how a site looks if the navigation is confusing or broken.
It can be argued that a clean modern look and feel communicates more professionalism than one that looks like it was made in 1998, but that's just on first impression.
Users are the opposite of marketing and product managers in that they care much more about function than about form.
A good site can be "ugly", providing it clearly communicates its content, and facilitates fuss-free use.