Crary was born in Preston, Connecticut, where he attended the public schools and graduated from Trinity College, Hartford, Connecticut, in its first class in 1827. He studied law, was admitted to the bar, and commenced practice, while also assisting in editing the New England Weekly Review at Hartford. He moved to in Marshall, Michigan, in 1833.

He was a delegate to the State constitutional convention in 1835 and upon the admission of Michigan as a State into the Union, he was elected on October 5 and 6, 1835 as a Jacksonian to the Twenty-fourth Congress. However, due to Michigan’s dispute with Ohio over the Toledo Strip (see the Toledo War), Congress refused to accept his credentials and he was seated as a delegate until Congress admitted Michigan as a state of the Union on January 26, 1837. He was elected as a Democrat to the Twenty-fifth and Twenty-sixth Congresses, until March 3, 1841.

He served as regent of the University of Michigan 1837-1844 and founded the public-school system of Michigan, then serving as a member of the State board of education 1850-1852. He was editor of the Marshall Expounder for several years and a member of the Michigan House of Representatives 1842-1846, serving as speaker of the house in 1846. He died in Marshall and is interred in Oakridge Cemetery there.