Be human and honest with him. Take the time to do it, point the reasons you've got to fired him, but also point the good side [he makes excellent coffee, he's a funny guy, or whatever]. Don't just blame him or blame you.