The Turner House Gallery was built in 1888 by wealthy Penarth resident, James Pyke Thompson, on land previously occupied by the thatched farmhouse of Taylor's Farm.He named it after the painter J.M. Turner and used it to house part of his private art collection, which included paintings by J.M.W Turner and Dante Gabriel Rossetti as well as some examples of fine porcelain.

The gallery was opened to public view on the one day, Sunday, when his collection could be seen by the largest number of people, and thus became known as 'The Sunday Gallery'. The gallery was originally housed on the first floor of the building with the downstairs area being used for the custodian's accommodation.

The building is constructed of red brick with moulded brick string courses and dentil cornice. Red sandstone is used in the detailing in Italianate early Renaissance style and the roof is in red tile with a longitudinal light. There is a projecting arched centre bay with a plain semi-circular panel with the painted inscription 'Turner House Gallery'.

Following Thompson's death in 1897, the gallery gradually lost the public's interest and attendance figures dwindled. In 1921 the Trustee's decided to hand over Turner House and its contents to the National Museum of Wales to exhibit works from the reserve collection and temporary exhibitions of the National Museums & Galleries of Wales.

In 2003, responsibility for programming and administration at Turner House was taken over by Ffotogallery, the national development agency for photography in Wales, where it now acts as a significant new space for contemporary photography and as an additional exhibition and educational outreach facility alongside its Cardiff base at Chapter Arts Centre.