Fisher, who won one national championship and three ACC titles during his eight seasons at Florida State, signed a 10-year, $75 million contract with Texas A&M last offseason.
It was at LSU that Fisher and Woodward first met and formed a relationship that proved instrumental in the two later joining forces at Texas A&M.
During an interview with TexAgs, Fisher acknowledged his affection for Woodward while also noting that "in this business, change happens."
"Scott was one of the reasons I came out here, as far as the introduction and the comfort level I had with him," Fisher told TexAgs. "But after being here, this place is phenomenal, and I plan on being here a long time."
He later added: "And hey, we got a job to do. And I wish [LSU] the best, but we're going to find another great athletic director because this is a phenomenal place. And we're going to go on, we're going to win championships and do the things we're going to do."
R.C. Slocum, the Hall of Fame coach and winningest coach in Texas A&M history, was appointed interim athletic director. In a statement, Slocum said, "My role is to keep the momentum going forward until a permanent AD is hired."
Fisher led the Aggies to a 9-4 record last season, during which he said he saw starting quarterback Kellen Mond learn how to make big plays in pivotal games.
"We're working on how precise and how advanced he and the team can get," Fisher told TexAgs. "We're working on his accuracy and his ability to get the ball out of his hands."
Fisher told the outlet that players were "so uneducated about how we wanted to do things last year" but pointed to Mond and Quartney Davis as examples of players making great strides.
The Aggies open the season at home against Texas State on Aug. 29 before traveling to face the defending champion Clemson in Week 2.