Sync Query Problem?

[Sweens]

Hi

 

(Hope I’m posting this in the right place – novice forum user!)

 

One table in my database tracks visits to a website – the total records count is now about 80million.

 

I run a daily query for the synchronisation, so that (hopefully) only the newest records are added – so

 

1) I only look at the id column (not all columns) in the ‘columns to compare’ and 

2) The ‘SQL Where’ column is id > 79,900,000 (for example – I change this value regularly)

 

However, it takes FOREVER to run – it looks like it is scanning all records instead of only those where id>79,900,000. 

 

Am I doing this correctly?

 

I there a better way of doing what I want – some way of only adding new records? I’m not bothered about the old data, as it won’t have changed.

 

Any help much appreciated.

 

Chris

[Sibin A S]

Hi Chris,

 

Can you please tell the total number of columns in the table that you have been trying to sync. Also tell us the SQLyog version and MySQL server version.

 

Regards,

Sibin

[Sweens]

Thanks Sibin

 

There are 79268055 records in the source and 79109275 in the destination.

 

Using v12.2.4

 

EDIT – Sorry, misread your post; there are only 10 columns in the table.

 

(I am thinking I should dump the oldest records and just use that last 2 years!)

 

Cheers

Chris

[Sibin A S]

Hello Chris,

 

We will try reproducing this with similar number of rows. Also you can dump the oldest records if they aren’t required for you as it also decreases the load on the server as well.

 

Regards,

Sibin

[Sweens]

Thanks for investigating, Sibin.

[Author]

Posts