Are there any "rules" or recommendations in UX Design for displaying a lot of data on a webpage? Or how to select the most needed information to display data?
Much of the time, it's most useful to summarize your data into visuals and higher-level categories that allow you to drill down to the specific data. This is why infographics are so powerful and why sites like mint.com are loved.
Categorizing your data in mutually exclusive, collectively exhaustive (MECE) ways will also eliminate redundancies in what you're displaying while also making sure what is displayed is thorough. For example, make sure to segment your data into say, 3-5 categories, but also provide an "Other" category that can catch anything that may not apply (this works well so long as the Other category is relatively small).
You could then choose to present the most often clicked categories above the fold.
A quick google turns up this article: 14 Beautiful Content-Heavy Websites for Inspiration
It offers several principles, reworded here:
White space – allow the content (and your visitors’ eyes) room to breathe
Boxes, borders & graphical planes – Segment the information into visual categories
Intuitive search method – Let your users jump straight to the info they need
Grids – Although not always necessary for comprehension, keeping content within a rigid, consistent structure helps reduce the effort required to process it
Strong information hierarchy – Establish a consistent design language using content types (blurbs, excerpts, call to actions)
Visual hierarchy – The relative importance of different content areas and elements can be visually implied in many ways, ranging from typographic treatments (headlines, sub-headings, pull-quotes, etc.), to image sizes and saturation, placement, etc.
I would add one more important thing that ties into the hierarchical points.
User-Centric Content Decisions - Try as best you can, via user testing and analytics, to determine what content users really want/need and use that information to influence related decisions.
Design principle: Organizing information https://uxplanet.org/design-principle-organizing-information-343a7ef936a8
UX Design Guidelines For Content Heavy UI https://theuxblog.com/blog/design-guidelines-for-content-heavy-ui
How to strategically plan for data-heavy UX design https://getflywheel.com/layout/plan-data-heavy-ux-design-how-to/