I would worry about your mis-manager than your coworkers. Coworkers dont have any input in performance reviews unless they are seniors.