I can't see him as charming, either. I can't see any witch or wizard being "seduced" to the dark side because of his charm. Fear of him or desire for power, sure, but not charmed.
Charming? I certainly don't see that side of him in the movies, although I think Fiennes makes an excellent Voldemort. In the books, Tom Riddle could certainly be charming. I've always seen Voldy as a very charismatic individual- someone who could easily lure other wizard/witches to the dark side, especially if they looked down on muggleborn wizards to begin with.