James Fenn Clark was born on January 3, 1823, in Calcutta, India.

He was a physician in Royal Leamington Spa, England, and a supporter of missionary work. He came from a middle-class family whose fortunes were closely linked to Britain's trading and colonial empire. His father Hezekiah Clark (1792–1868) was a surgeon in the Bengal Medical Service in India (1822-1855). As a young man, Clark returned to England for schooling and he settled in Leamington. He played a prominent role in the local community, helping found the Leamington Provident Dispensary in 1860 and becoming Treasurer of Leamington College. He was also an active Christian, supporting his local church and serving on several church committees. These included organizations active in spreading the Christian message in the British colonies, notably the Church Missionary Society and the British and Foreign Bible Society. He travelled extensively, collecting many natural history and ethnographic items from places including India, China, Russia, Norway, Africa, Australia and the East and West Indies. These would form the nucleus of the Leamington Spa Art Gallery and Museum. He was also an accomplished amateur painter.

In 1851, he married Sophia Morris (1830-1855). In 1862, he remarried Adelina Josephine Roberts (1830-1916). He died on January 7, 1910, in Cheltenham, Gloucestershire, England.