The mosque sits on a site once occupied by a Byzantine church that may have been built over the remains of an earlier temple. The church was converted to a mosque after the Muslim conquest. The minaret was destroyed in an earthquake early in the 11th century. In the 12th century Crusaders built a church that was destroyed by the Ayyubids. In the 13th century a new mosque was built by the Mamluks, but destroyed in Mongol invasions. Though the mosque was soon rebuilt, it was destroyed again by an earthquake toward the end of the 13th century. The Ottomans rebuilt the mosque in the 16th century. It was severely damaged in bombings during WWI, but restored in 1925.
In 2014, the mosque was severely damaged during aerial bombings of Gaza.