Two things I'd recommend:
Have a conversation with them and find out how passionate they are about what they do. This is a good indicator for whether someone's a good hire for any industry, but specifically, you want to gauge how well they're involved and up to speed with events. I find people who know what's going on, what's new, what's cutting edge and from all those things what to use and when to use it are more useful than people who learned the theoretical way to build a UI in 1995 but didn't keep up with the real world.
Get them to redesign something you don't like very much. This is important because one of the core skills of an interaction designer is refactoring, which should be familiar to you as a programmer. Redesign isn't about throwing everything out and starting from scratch, it's about knowing what to change, why to change it, and understanding the impact of that change on the userbase, client, and engineering team.
So, for instance, I'd start the first part with questions like "What do you like about Hunch?" or "How do you feel about the new Google Images search?". This way you can figure out if they even know what's going on in UI design. "Are you excited about designing for mobile platforms?", etc. Ask them to tell you about their favourite app or game and explain what's so great about it from a UI/IA point of view. Get them to sell something to you - their passionate monologue/rant about something will give you a good sense of whether they know what they're talking about or they're just making stuff up.
The second part is harder given that you're not a designer yourself, but like programming interview questions, it's not so much about the end result as it is about the process. So make sure that when you ask them to redesign something, it's small enough in scope that they can go through that process with you. Pay attention to how they do that: do they whip out a drawing pad and start sketching out ideas? Do they say "well, do you have a computer with Photoshop installed?". Do they just start asking questions? All of these things will tell you how they communicate, how they learn, how curious they are. If they're good, then before just making rash decisions about changing a UI, they'll want to understand it as best as they can. That means understanding the business logic, vision/strategy, and engineering decisions.
I understand my advice is non-conventional, but I've always subscribed to the notion that if someone's a good, passionate, honest person, they'll do more for you in the long run than if they can name 5 famous designers and 10 UI books off the top of their heads. In the interview, you need to not only test them for UI trivia, but also about how they do their jobs. This method is really good at doing that.