The Library Company of Philadelphia, founded in 1731 by Benjamin Franklin and a group of his friends, was the nation’s first public circulating library. Today, the Library Company is an independent, non-profit research library of national importance comprised of over half a million rare books, manuscripts, prints, and photographs documenting every aspect of American history and culture from the colonial period through the end of the 19th century.
http://www.benfranklin300.org/bf300.htm
Harvard is the oldest institution of higher learning in the United States (see: first university in the United States), founded 16 years after the arrival of the Pilgrims at Plymouth. Harvard College, established in 1636 by vote of the Great and General Court of the Massachusetts Bay Colony, was named for its first benefactor, British-born John Harvard of Charlestown, a young minister who, upon his death in 1638, left his library and half his estate to the new institution.
https://en.wikipedia.org/wiki/Harvard_University
Clearly, many private libraries owned by various wealthy people existed early in the US's colonial history. The location of the "first" one of these is most likely a mystery.