Orr grew up in Columbia, South Carolina. He earned a bachelor's degree in English literature from Princeton University in 1996, and subsequently a law degree from Yale Law School. While still a law student, Orr published a review in Poetry Magazine. While practicing law, Orr has written reviews and essays for Poetry Magazine, The New York Times, and other periodicals. Orr was awarded the 2004 Nona Balakian Citation for Excellence in Reviewing of the National Book Critics Circle. In 2005 he became a columnist for the New York Times Sunday Review of Books, where his On Poetry column appears occasionally. He was the Hodder Fellow at Princeton University in 2006-2007.