The biggest thing is going to have to be the people on each site, and in each community. StackExchange would be wildly different if there were a totally different type of community in place, despite all the infrastructure the site creators offer. In terms of user interface, though:
Something I find profoundly unpleasant, although probably not what you're looking for. If I'm not logged in, Quora's homepage tells me one short sentence about what it is, and then tells me to make an account or log in. It doesn't give me any sort of preview or hint at how the system works, or anything. StackExchange.com, if you're not logged in, kind of gives you a list of topics you can check out.
Furthermore, Quora seems to focus on just the two colors: blue and grey. StackExchange has blue, grey, hits of orange, and other colors that pop up and do exciting stuff when you mouse over this or when there's something to notify you of.
Quora's interface seems very clean, polished, and calculated. StackExchange has some of that, but offers a little more... Character, in part with the color I described above, and in part with things like the sketched-on up and down vote buttons, and stuff like that.
SE puts a lot more stuff around each post. User info is a whole badge thing, and the vote score is much larger. There's a whole lot going on around the first post (the one that starts the topic) in SE. Furthermore, it seems that in Quora, the original poster is not expected to elaborate on the question at all, if he doesn't feel like it--I don't think I like that, but whatever. Finally, SE allows comments on posts--but only that one extra level. I find the comments to be handy, but there's a degree to which they restrict you--they cause each "answer" post to carry much more significance, and you can't really just contribute to a conversation very easily, unless in reference to one particular post. SE, in this way, is not about conversation as much as it is about finding the answer--relative to Quora, that is. In this vein, I'd point out that SE allows you to pick an answer as the one that answers the question (a la Yahoo answers), whereas Quora does not seem to be interested in closing a question thusly.
They're both very nicely designed web interfaces, and they both use a lot of similar design elements. I prefer SE, but maybe because I'm used to it. Oh well.