As a scientist stuttering myself I can say: don't worry. I have never ever received negative feedback from fellow scientists, organizational staff, (potential) customers or other people involved in conferences. In which field do you work? I don't know how it's handled in e.g. social science, but at least in the STEM field, people are usually focused on the contents of your research, not if you are stuttering while presenting. We scientists have the duty to present our research. Research not published or presented has not happened. Don't let other people hold you back from filling out the job you have chosen.