The basic process would be:
Decide what you want to know after the study. "Evaluate usability" is not enough, as the concept of usability is too complex. It has many aspects, and you can't evaluate them all with a single study. For example, achieving the research goal
create a list of issues which have to be fixed in the next design iteration requires a very different study type from the goal
I want to find out if my product is better suited for beginner developers than my competitor's.
Evaluate the resources you have to conduct your study. The limiting resources will be usually the UX expert's worktime, the calendar time until a result has to be delivered, and the budget available for recruiting participants. You can save some UX expert time if you also allocate a budget for specialized tools, but it is not certain that there will be tools available for the type of study you want to conduct. You don't have to pay money for a statistical tool. Tullis and Albert suggest Excel, and it is enough if you are aware of its gotchas.
Design the study. Decide how many participants you need to collect the metrics you want, or, if very limited by recruiting budget, see how many participants you can get and choose metrics which are reasonably good for answering your research questions but can be measured with the number of participants available.
Organize the study. Prepare a workplace for the participants (if not doing unmoderated study from home - then prepare a way for them to have remote access to your system), prepare task descriptions, prepare data needed for completing the task, prepare questionnaires if needed (electronic ones are much more convenient, but a survey tool also costs money). Recruit participants, make appointments.
Conduct the study itself.
Evaluate the data. You will need statistics knowledge for it, else the results you get will look good but be actually totally meaningless (e.g. if you don't know statistics, you might decide to calculate an average value from the answers on a Likert scale). You don't have to be a wizard in statistics, but you have to know what types of data there are, what are basic descriptive statistics, what are confidence tests, and what is linear regression. Multivariate regression is more ambitious, but gives answers to questions which your managers will probably find most interesting. We are talking about maybe 4 to 8 credits worth of entry level statistics here. Of course, if you only have decided on a very specific study design, you can concentrate on the stuff you need for evaluating this one study only.
Make a nice presentation out of your results. Whatever your statistics tools spews out as raw data, it is normally not suited for human consumption. You have to document how you conducted the study, what your results are, what conclusions can be made from them, and what conclusions cannot even if they seem logical.
There are books on how to do this all. "Measuring the user experience" by Tullis and Albert is one possibility which is aimed at practitioners, not academics, and it covers all of the steps above. (There are whole books written on each of the steps too, but this probably goes too deep for you).
For step 3, you will probably need task descriptions around which you will structure your study. They are supposed to be created in a project independently of any studies. If you don't have any, you will have to create them first, there are books about this too. For some types of research goal, it is also possible to do a study without tasks, but 1) this may not be applicable to your case, 2) you can't well evaluate the "test coverage" of this type of study, in the worst case you will think that everything is OK with your software only because your participants didn't think of trying out some tasks which your software supports badly and 3) you will need an alternative classification for structuring the results, and most non-task based ones (e.g. "they have trouble with items from the Edit menu and items from the Tools menu) do not let you recognize the connection between usability issues.
The above is not specific to developer tools, you can use it with any type of software. Discussing specific details connected to the fact that it is a developer tool is practically impossible at this level of abstraction.
Update in case the above seems overwhelming
If you have done any psychometry, econometry or similar in the past, the skills are very transferrable. If you find out that you need an intern to help you (it is lots of work), try to get one with experience in these fields, it is more likely to find students who have had courses in these than ones who have been involved in software usability.
Maybe you are now asking yourself if you need to go through all of this if what you want is a simple study. The problem is: there are simple studies, and they can indeed be done with little effort. But you need the background knowledge to know if what you did was right. Empirical studies of any kind have this unpleasant quality: results from bad studies look just as good as those from good studies, and you need somebody with enough knowledge to look at the procedure and see if what you found out is indeed good insights, or garbage.
It could be an easier way to emulate something other people already have done, this way you know that your design is right, which is the biggest problem for a beginner. Look around on Google scholar for papers about evaluating developer tools, read the methods section, and see if you want to find out about your tool whatever the authors found out about theirs. If yes, do it the same way as they did.