Hi,
SQLyog uses PuTTY Private Key Files (ppk format) for public/private key authentication. So, please make sure that the key file you are using is in ppk format. If you are using ppk format and still getting the error, make sure your pass-phrase is correct. You can try the same with PuTTY(http://www.chiark.greenend.org.uk/~sgtatham/putty/download.html).
If you are using OpenSSH keys, refer the following link to convert them into ppk format
http://meinit.nl/using-your-openssh-private-key-in-putty
Also please visit the below SQLyog FAQ link for common SSH error messages.
http://webyog.com/faq/content/22/113/en/ssh-error-messages.html
Regards,
Vishal P.R