Sa'id ibn Ubaydah said:Ibn Umar heard a man swearing: No, I swear by the Ka'bah. Ibn Umar said to him: I heard the Messenger of Allah (ﷺ) say: He who swears by anyone but Allah is polytheist
Hadith No. 3251
Sunan Abu Dawud
Gradings:
Al-Albani:
Sahih
Muhammad Muhyi Al-Din Abdul Hamid:
Sahih
Zubair Ali Zai:
Isnaad Sahih