That's a question too tough to answer. Everyone needs to know that ANYTHING can happen at any moment ... whether it's 35mph, 75mph, or over 100mph. First and foremost ... RESPONSIBILITY. Know your surroundings, know who you're riding with, and know the conditions. I've seen my speedo hit 113mph, and I know that's plenty fast for me. Would I take more ... of course I would!!! But I'm more than happy with it. I also know there's only about a half dozen places where I'd even consider trying to hit that speed.
Just because you have a machine capable of it does not entitle you to use it unless you know damn well you're in a place where you're safe, as well as the people around you. I cringe when I hear people say "Yeah, I hit 100mph on my sled ripping down this straight stretch on the trail!" Ever think what could happen if a deer jumps out? There just isn't enough time to stop if something jumps out of the woods.
I don't think it's ever a matter of "too fast" ... I think it's a matter of "when is the time right to see how fast I can go".