Can't Sync 7.6g Table

[milandred]

i'm been using SQLyog for Windows for around of 2 months, and it is working fine for the small tables, but i need help now, i can't sync 1 table with

7.6 and above gigs of data.

i'm using the Synchroniztion Wizard!.

everything looks fine, it is able to count the source and target rows, but after 3 or 4 hours it stops

while still comparing them… and it says to check the logs, for the errors.

but the log file only record this.

Table SrcRows TgtRows Inserted Updated Deleted

========================= ======= ======= ======== ======= =======

`Subscribers` 17274994 17274986 There was an error while execute a query. The query and the error has been logged in the log file. 0 0

Total time taken – 7595 sec(s)

`Subscribers` 17274994 17274986 Error No. 2013

Lost connection to MySQL server during query

0 0

it seems that the connection got lost… how can i keep the connection active?

i'm using v5.21

thanks for you help.

[peterlaursen]

Please answer:

1) How do you connect?  direct, SSH or HTTP ??

2) Is a a shared server (ISP) or is the server configuration under you control?

[milandred]

peterlaursen wrote on Dec 26 2006, 11:18 PM:

Please answer:

1) How do you connect? direct, SSH or HTTP ??

2) Is a a shared server (ISP) or is the server configuration under you control?

i'm using application for windows (SQLyog Enterprise -Database Syncronization Wizard)

my internet connection at work is a T1. My internet connection at home is DSL PRO. I guess Direct.. i'm not using SSH or HTTP.

i've tried both, my work, and home internet and the same results.

also, i was using v5.20 and before installing v5.21 i uninstalled v5.20 completely.

also i tried with several computers.

I can tell the my internet is active, because i u sing the ping command, while doing the Sync to see if my network is having packet loss.

no, it is not a shared server…

thanks.

[peterlaursen]

No HTTP, reproducable on several computers etc. etc.

This looks like a data specific error.  

I think it is one particlar row of data causing this and not the amount of data.

Can you identify the row where it fails?

What was the last row that was succesfully inserted/updated?

[milandred]

peterlaursen wrote on Dec 27 2006, 09:36 PM:

No HTTP, reproducable on several computers etc. etc.

This looks like a data specific error.

I think it is one particlar row of data causing this and not the amount of data.

it is not a specific data.

i tried with different databases, and the same.

the problem is that SQLyog can't handle this amount of data at once..

to fix the problem, i did select 3 or 4 or 5 rows at a time, it took me like 4 or 5 days but fixed the problem.

the developers should find the problem and fix it, my databases is increasing fast.

Can you identify the row where it fails?

What was the last row that was succesfully inserted/updated?

[peterlaursen]

Please answer: how many ROWS are there in the table?

(“SELECT count(*) from …”)

[peterlaursen]

Quote:

the problem is that SQLyog can't handle this amount of data at once..

It does not!  Let me explain how it works.

1) using the concat_ws() and md5() functions it generates a checksum of 1024 rows (ans this is done by the SERVER – not SQLyog!) and compares for the two databases.  

2) If there is no difference we proceed to the next 1024 rows

3) if there is a difference 1) is executed recursively untill the rows that are different are identified.

4) Those rows are now INSERTED, UPDATED of deleted.

So it simply does not matter if there are 1024 rows or 1024*1.000.000 rows.  What is inside SJA at a particular time is the same.

If it is not DATA-specific, then it must be a server issue, that it only allows for a connection for 200 minutes (for instance – 3-4 hours ~ 200 minutes).

We cannot do any progress without a test case!  So if you want us to keep on working on it you will need to provide one!  We can then easily test with a broad set of servers!

BTW:

Quote:

to fix the problem, i did select 3 or 4 or 5 rows at a time, it took me like 4 or 5 days but fixed the problem.

How do you select rows with SJA DATA SYNC.  Or did I misunderstand something basic?

[Author]

Posts