Segel was born and raised in Los Angeles, California. He was a state champion basketball player in high school and also plays the piano and wrote a song for his character, a drummer, to sing to the lead female character, Lindsay, in an episode of Freaks and Geeks. Lindsay was played by Linda Cardellini. She and Segel dated until February 2006.

Segel is known for his role as "Freak" Nick Andopolis on the short-lived NBC comedy-drama series Freaks and Geeks, about a group of suburban Detroit high school students circa 1980. He has also had recurring roles on CSI: Crime Scene Investigation and Undeclared. He currently plays Marshall on the CBS sitcom How I Met Your Mother, which debuted on September 19, 2005. Feature film appearances include Slackers, SLC Punk!, The Good Humor Man, and Dead Man on Campus. In 2007, appeared in Knocked Up, starring Katherine Heigl and Seth Rogen and directed by Freaks and Geeks producer Judd Apatow.