Tom Brady's endless list of accolades include seven Super Bowl rings, three NFL MVP awards and 15 Pro Bowl appearances. But despite all his winning, Brady views his losses as the most important moments of his career.

During an appearance at the Fortune Global Forum this week, the winningest quarterback in NFL history said he could never have achieved his success without first experiencing setback after setback.

The 47-year-old broadcaster recalled years spent at the bottom of the QB depth chart, first in high school and then in college at the University of Michigan. Constantly losing out on the starting job in favor of other quarterbacks, he said, instilled in him a relentless desire to improve himself and prove his doubters wrong.

"I had to compete my butt off for five years in college. I had to compete my butt off in high school for four years," he said. "Those nine years of playing prepared me to be a professional."

Young athletes today are done a disservice by switching schools or athletic programs when things aren't going their way, Brady said. In his view, his career would never have been possible had he not had to fight for playing time.

"There's no way I would have had this success had I not gone through all the challenges of high school sports and the challenges of college sports," he said. "I look at so many of these young athletes who are never forced to go outside of their comfort zone. What do they do? They're told how great they are their whole life. They have people who coddle them. They never have to push beyond their limits."

To achieve true success, Brady believes you need to be willing to go out of your comfort zone and, in some cases, to fail.

"If you fail and then you figure out a solution with the people you work with to overcome that failure, you gain a lot of self confidence," he said. "And if you gain self confidence, the next opportunity you have to succeed, you have a much better opportunity to do just that."

Rather than viewing that failure as a setback, Brady described it as an opportunity to improve.

"To me, failure is amazing," he said. "It teaches you and really forces you to look inside yourself about what you can do better."

Looking back at his early struggles as a football player, Brady said he wouldn't change a thing.

"We've all faced our own adversities," he said. "And we look at all the hardest things that have ever happened in our life and the biggest challenges we've ever faced. We look back at those and we realize those are absolutely the best things that could have happened."

Want to earn more money at work? Take CNBC's new online course How to Negotiate a Higher Salary. Expert instructors will teach you the skills you need to get a bigger paycheck, including how to prepare and build your confidence, what to do and say, and how to craft a counteroffer. Start today and use coupon code EARLYBIRD for an introductory discount of 50% off through November 26, 2024.

Plus, sign up for CNBC Make It's newsletter to get tips and tricks for success at work, with money and in life.