Miller was born in Chipping Norton, Oxfordshire, England; the son of Joy Marie (née Palm), a special education teacher, and Wentworth Earl Miller II, a lawyer and teacher. Miller's father, a Rhodes Scholar, was studying at Oxford at the time of Miller's birth. Miller has stated that his father is of Jamaican, German, English, Jewish, and Cherokee Native American Indian ancestry, and that his mother is of Russian, French, Dutch, Lebanese and Syrian descent.

His family moved to Park Slope, Brooklyn, New York, when he was one. He retains dual citizenship. He has two sisters, Leigh and Gillian. Miller attended Midwood High School in Brooklyn, New York, USA. He was a member of SING!, an annual musical production that was started by Midwood. He graduated from Princeton University with a bachelor's degree in English Literature. While at Princeton, he performed with the a cappella group the Princeton Tigertones.