Huh, yeah, that sounds pretty cool! There's sometimes talk of a "novelty" or "distraction" effect, which can reduce the amount of stutter. It'll be most effective in the beginning of the activity (whatever it is), but is also very likely to fade later on, once you get familiar with the activity. VR definitely isn't "real life", so perhaps the mode of experience requires a certain amount of concentration or effort from you, in a different way than what you're used to - or perhaps it's just different enough to "catch you off-guard".