The German term antisemitisch was coined in the late 19th century specifically as a word for Jew-hatred, and that usage carries on to the present, including its English cognate antisemitic.There is no "semitic" race that antisemitism opposes; the term "semitic" itself is historically used by those same antisemites as a name for Jews.
Insistence on the literal meaning of the term, instead of its common meaning, is itself arguably a manifestation of antisemitism, as it seeks to distract from the specific form of prejudice the term connotes.