How often does this information change? When it does change how does it change?
For example, if you add a new training course, you'll presumably need to decide which job roles it's relevant for. But do you really need to see the whole grid to do that, you might only need a page for the new course with a list of roles to pick from.
Likewise, if you invent a new job roles you might only need a page with a list of the courses to pick from that could be provided for people in that role.
Another alternative, if you don't actually need to see everything all at once is to have two lists like this...
download bmml source – Wireframes created with Balsamiq Mockups
Selecting something in the left list "activates" the right one by adding checkboxes to add the relevant courses. You might want to have another view with the lists the other way around, or you could just click on the items in the right hand list to "activate" the left one with checkboxes, although that's a little less obvious perhaps. You could garnish those lists with "add course" and "add role" buttons as needed.
Essentially, rather than thinking of the data model, think about what the common tasks are you need to do with the system, then design interfaces for those very specific tasks. Don't try to design one generic interface to solve everything - that way leads to creating a monster! ;-) If you can tell us what those tasks are then you can probably avoid that huge grid altogether.
For example, maybe you need to compare a bunch of roles or courses to each other. That's ok, but likely it would be easier to pick the roles/courses you wanted to compare and then get a more limited display just for those, rather than facing the whole grid.
If you really do need to see the whole grid all at once then @Benny's suggestions are great.
The only thing I'd add is are there any natural groupings of job roles or training courses at all?
If so I'd collect those together so you have (for example) a "Management" block of 5 columns, then a small gap then a "Technical roles" block of 7 columns and so on.
You could do the same with the training courses if there is some natural grouping for those.
If you can group things, then you can close groups to hide rows or columns group by group, rather than individually.