Database Sync Error/timeout

[optinst]

Good day.

First would just like to say Thank you to the Webyog team for the SQLYog tool as it has become the tool of choice used for MySQL database access.

We have recently tried to do a database sync of a large table of date (> 17 million rows).

We see in the 'Completing the SQLyog Database Synchronization Wizard' dialog that the 'Source:' and 'Target:' fields show 'Retrieving table data…'. We realize that this is because it is assembling the data to sync.

I have been running the query now for about 15 minutes and will continue to let this run but in the past we have seen errors related to timeout/connection lost and I am concerned that's what will happen with this one. I use Wireshark to sniff the connection on port 3306 to see that it is currently still active and it appears as though it is caching the data locally as large packet size of data are being streamed.

I have a few questions regarding this:

1) Assuming that the data is first cached and then sent to the Target or that it is cached in chunks and then each chunk is sent to the Target, could the 'Source: ' field be updated to reflect its progress rather than both fields simply saying 'Retrieving table data…'. This would give some indication as to what stage and progress it is currently in rather? (My assumption is that it is all cached locally first for all rows and once all rows are cached in then inserts them into the Target as my TCP trace shows large packets from the Source to my host machine running SQLYog, with no connections out from my host machine to the Target yet (still retrieving rows to insert I assume).

2) What is the location of the cache file on the host machine? i.e. %temp% directory in windows, etc? I have checked %temp% and see a few files that were created about the same time as I started the sync but nothing that is increasing in size dramatically. There is a 0 byte file which may be the cache file but this would imply that the cache file is never being with each write which I cannot see as this would be risky as there would be no way to recover the data up to the point at which a failure occurred.

3) Is there any way to resume a long database sync if it should fail rather than having to restore the sync over? I have tried saving a Sync task, running it and upon failure re-running it but it appears to just start a new version of the task (as expected) rather than be able to recover a previous run.

4) Our DB admin is away and so if the error ends up being a MySQL server-related setting, can anyone tell me what the MySQL server setting may be? I can't see it being related to any typical inactivity timeout b/c I still see large packets of data exchanged b/w Source and Target but do not know the various settings in the MySQL server that may lead to a timeout.

Thanks for any help you can provide.

Best Regards.

[Author]

Posts