Fraser was born in Indianapolis, Indiana, the son of Canadian parents Carol, a sales counselor, and Peter Fraser, a former journalist who worked as a Canadian foreign service officer for the Government Office of Tourism. Fraser has three older brothers: Kevin, Regan, and Sean. His family moved often during his childhood, living in Eureka, California, Seattle, Ottawa, the Netherlands and Switzerland. Fraser attended the private boys' boarding school, Upper Canada College, in Toronto. While on vacation in London, Fraser attended his first professional theatrical performance in at the West End. He graduated from Seattle's Cornish College of the Arts in 1990. He began acting at a small acting college in New York. He originally planned on attending graduate school in Texas but stopped in Hollywood on his way south and decided to stay in Los Angeles and work in movies.

His surname is properly pronounced "Fray-zer," though some pronounce it "Frasier" (as in Kelsey Grammer's televisio