We all get nervous sometimes, but common... he knows you stutter. He did not say anything about it. Then what's the problem? Try to be calm, breathe and talk slowly. If you want, you can say to him that there's a probability that you stutter more because you might be nervous. That's all!! Life's to short to stress about this things. Communication is the key. Go there, met him and have a good time!