The exterior of St Mary's successfully blends elegant Perpendicular lines with rustic stone, partly fieldstone and partly a type of deep golden green sand known as carstone, which came from a nearby quarry known as The Butts.
The 12th century church sits close by the Manor House , in rolling Bedfordshire farmland. In 1407, Sir John Cockayne, Chief Baron of the Exchequer, purchased the estate of 1500 acres, which included the church. The Cockayne family line continued until 1745 with many gravestones (and brasses) in the church. The succession continued with the Cockayne-Custs until 1897 when the estate was sold.
Most important in the history of the church is the advent of the Hon and Revd Henry Cockayne Cust who succeeded in 1805. At this time the church was in a poor state of repair. It is said that . when Henry was taking the Christmas Service, snow was falling on his head. Henry began a major restoration of the church. He restored all the church roofs and rebuilt the south aisle. He reduced the size of the chancel and added a new, and spectacular east window with glass by Willement.
Henry was a great traveler. He visited Flanders and Brussels and saw magnificent woodwork for sale following Napoleon's sacking of churches and monasteries. He sent his agent to purchase selected furniture to install in the church. Most impressive is the woodwork from an Abbey in Oignes. This comprises sixteen large panels which show medallions of saints, These are sited in the chancel over rows of pews, with misericords. The woodwork is set on both sides of the chancel in collegiate style, as are the pews in the nave, also from Ognes. Other woodwork includes a fine altar rail from a church in Flanders showing four carvings of aspects of the Eucharist.
Nearly all the windows in the church are by Willement, including the fine west widow which depicts St Peter and the Keys of Heaven. In the north aisle is a 14th century stained glass window showing four saints from the northeast of England, The organ loft rail, recently restored. shows an array of medieval musical instruments.. Finally, the churchyard contains has a monument to WE Henley, the author of the poem 'Invictus', and whose daughter, buried here, was the inspiration for the name 'Wendy' in Barrie's story of Peter Pan.