The Jewish cemetery was established in 1774 at the instigation of the court Jew Jacob Elkan. In 1775 it was first occupied by a deceased "little daughter" of Jacob Elkan. In 1805 Jacob Elkan was buried. In 1808 Gabriel Ulmann (1743-1816) added another plot to the cemetery area. After the last burial of Caroline Elkan in 1890, whose heirs moved to England, the cemetery fell into disrepair. Already in the early 1930s the gravestones - partly smashed - lay on the ground. 

Until 1881 the dead of the Jewish community of Rappenau were buried in Heinsheim . Since then, there was a separate cemetery on Siegelsbacher Straße, 150 m behind the current municipal cemetery. The first person buried in the cemetery was Thekla Herbst (b. 1868, d. May 29, 1881; below tombstone no. 12)  
  
Three Polish prisoners of war and two children of Christian Russian forced laborers were also buried in the cemetery in 1944/45. The area of the cemetery is 4.79 a.