In the book Life on Delay: Making Peace with a Stutter by John Hendrickson, he goes into how stuttering might be closely tied to OCD. But to answer your question, no. Stuttering has nothing to do with you being bad at math. Sorry to say but you're just a regular person who's just bad at math.