This rural graveyard traces its history to the 1880's when the first burials occurred. There were six interments that year. Shortly thereafter, the Mt. Zion Presbyterian Church was constructed on the same hill, just east of the early graves on land deeded by J.T. Baldridge. This place of worship was also known as the White Church. The proximity of the church influenced the burial site's name. It too was originally known as White Church Cemetery. The early church and burial ground predate the naming of the Sharp Community. The Mt. Zion congregation united with the Sharp Church in 1917, and the building was eventually demolished. However, the graveyard continued to be known as White Church Cemetery for many years. It eventually assumed the name, Sharp Cemetery for the community that was located just to the west. Presently, the east side of the plot where the church once stood is now considered the "new part" of the cemetery, and is replete with many graves of former residents.