Okay, the mediator may be dead wrong. Not unusual for someone who probably is hearing the facts of your case for the first time. If the observation is a valid one, it's time to reevaluate your negotiating position. If it's not on all fours, ignore it--or, better, persuasively rebut it. If it just ticks you off, you had better look deep inside yourself and determine why it disturbs you so much.
Remember, the mediator will be doing the same thing with the other side, and, often, confidentiality in the caucus prevents him or her from disclosing what he has said or has been said to him. Also, the mediator does not suffer the gladiatorial enthusiasm "enjoyed" by the litigators. He or she frequently sees things the lawyers have shoved out of their consciousnesses in order better to prepare for battle. The battle, of course, is the trial, the last resort in our judicial system from which mediation is committed to do its level best to save the parties.
A footnote to all this. Mediators recognize when they are being had. They have to be on their toes, when accused of bias (hopefully a rare occasion), to avoid demonstrating their lack of it by reversing the perceived bias. When I was in general practice, nothing made me more nervous than having a "friendly judge" on my case. I always believed, if he was my friend, he had to be honest, and, if he was honest, he might unconsciously bend over backwards not to show a bias in my favor. I don't have to tell you what that could have resulted in.