peterlaursen wrote on Sep 11 2004, 11:37 AM:
I just wonder: why do you store data permanently in .CSV-files?
Why not a table in a database ?
CSV-files are meant for backup and import/export purposes.
It's no problem to have two or more MySQL-servers on the network –
they just have to use different port-numbers.
I'm sorry if I missed the point!
I do not believe that I said anything at all about storing data permanently in a csv file. Our inventory is in fact stored, manipulated, and processed in an MS Access database. The data is then exported to csv and then imported into mySQL.
When you are dealing with over 1,250,000 products this makes for a rather larger csv file if you are to leave it in one chunk. We have 13 major categories of products, so we break the inventory exports into 13 seperate csv files. I have found it to be MUCH easier to manipulate chunks of approximately 100,000 items during an import (speed, possible errors, etc) rather than 1,250,000.
So I guess if you cannot see the rationality of that then I am sorry. Perhaps if you were to attempt an upload to your servers of 1,250,000 products (which by the way encompasses just a hair over 198MB of data) at one time you would see the reason for splitting them up.
Thank you.
