[Bug] Import Numeric Typing

[premixedpie]

I think I have encountered a strange bug when importing CSV data using the external import tool. I am having trouble getting it to type numbers with decimal places correctly, even when manually setting it with the map options it won't get the decimal place values. It seems to only happen after a certain amount of entries in the CSV file, at least from what I have been able narrow it down to. Maybe I have overlooked something but after cutting down a larger file to try and find out the problem this seems to be the cause.

I attached 2 simple CSV files that should reproduce the problem. bug_test1.csv should import fine typing the last column as a double and using the right value. The only difference from bug_test1.csv and bug_test2.csv is that bug_test2.csv has one more row before the number with a decimal. bug_test2.csv now types the last column as an int and even when changed to a double like bug_test1.csv sets it or setting it to decimal(4, 2) it won't add the correct decimal values. If you need the full larger file that has the same issues with many more entries let me know and I can provide it.

Edit: I should have used these test files, just rows with 0 instead of the different types. I think it better illustrates that the 1 additional row seems to cause some problem.

[attachment=1599:bug_test.zip]

[sathish]

Hello,

This is an issue with the ODBC text driver. For csv there is no data type associated with the values so the driver has to perform a sampling on the data to identify the data type. This is 25 by default. so If you dont have any “double” value inside that 25 rows it will identify as integer or whatever apporiate. However you can modify this in ODBC configuration wizard in the source. So you have to change the sampling rate manually.

Please follow the steps below to change the sampling rate:

Start-> Run-> ODBC data source administrator-> User DSN-> Select the text driver->Click Configure->Options-> Define Format->Rows to scan- change the format to larger value(say 100)

Please refer the screen shot for more information.

Thanks,

Sathish

[premixedpie]

Excellent that was what I was looking for. Thanks for letting me know this can be changed and sorry about the bad bug report!

[Author]

Posts