Having turned the tables against most of them, Yusuf Khan had enemies lurking around him everywhere. Earlier working for the Arcot Nawab and British he earned the wrath of Mysore, and had slaughtered most of all rebellious Polygars who were anti-British, and the remaining were on the prowl. Now the Tanjore, Travancore, Pudukkotai, Ramnad, Sivaganga kingdoms joined with the British and the Arcot Nawab to attack Yusuf Khan, who by this time had proclaimed himself independent ruler of Madurai and Tirunelveli. In the First siege of Madurai in 1763, the English could not make any headway because of inadequate forces and the army retreated to Tiruchi citing Monsoons.
Meanwhile the Nizam Ali of Hyderabad once again proclaimed Yusuf Khan as the Rightful governor, while the Arcot Nawab and the British issued death sentence for Yusuf Khan as “to be captured alive and hanged before the first known tree as like a dog”.