Rose, a native of Irvine, Kentucky, is a 1958 alumnus of Transylvania University where he served as an assistant coach after graduation under C. M. Newton. He then took a similar position at the University of Cincinnati before returning to his alma mater as head coach and athletic director and recorded 160 wins in eight seasons.
In 1975 he became the head coach and athletic director at UNC Charlotte, where in three seasons he took the 49ers to one NIT championship game (1976) and to the NCAA Final Four (1977). In 1977 he was named The Sporting News National Coach of the Year, the Sun Belt Coach of the Year, and Charlotte's "Citizen of the Year". In three seasons Rose's record at Charlotte was 72-18 (.800).
Rose left Charlotte for Purdue University in 1978 and led them to the Final Four in 1980, and is one of only ten coaches in NCAA history to take two different schools to the semifinals of the NCAA Tournament. Rose left Purdue after two seasons after compiling a 50-18 record and finished his coaching career at the University of South Florida.
After retiring from the college ranks in 1986 Rose served as an assistant coach for four NBA teams: the San Antonio Spurs (1986-1988); the New Jersey Nets (1988-1989); the Milwaukee Bucks (1991-1992), and the Charlotte Hornets (1996-2001). He also served as the Bucks' vice president of player personnel in the mid-1990s.
Rose and his wife reside in Charlotte, North Carolina, where they attend nearly every Charlotte 49ers basketball game. On June 7, 2007, Rose was hired by the Charlotte Bobcats to be part of head coach Sam Vincent's staff.