• After carefully formatting my csv and triple checking my data, I’m getting the following:

    The file newimport2.csv has been uploaded.
    New columns imported from the CSV file.
    1447 records added
    187 records skipped due to errors

    I cannot find any errors of any kind in the log file (debugging is on). The log only seems to show each successful import.

    I also used an online csv checker and it appears clean. There are no blanks in my data fields (I added N/A for no data) I also removed all quotes from my data in case there was an issue there. Manage Database Fields has all my fields. I originally had 2 dropdown lists, but even when trying as text-line, I get the same result.

    Suggestions?

Viewing 5 replies - 1 through 5 (of 5 total)
  • Plugin Author Roland Barker

    (@xnau)

    It’s hard to tell what the specific problem is here. That message indicates that the query to store the record failed, but I don’t know what would cause that.

    What we need to see is an example of a query that failed. All the queries are in the log, but it may be hard to tell which ones are having the problem.

    What I suggest you do it create a CSV that has a single record, and example of one of the records that failed. Clear the log, then import the record. The query that was used to store that record will be printed to the log, and it can be analyzed for the problem.

    Thread Starter archerdesigns

    (@archerdesigns)

    Ok, but are the records added in sequence? I’m having a hard time determining which ones failed. If they are, I can simply look at record 1448 or 1449?

    I did not find the word “error” or “fail” when searching the log. Is there something that will only appear on a failed record?

    Plugin Author Roland Barker

    (@xnau)

    It is not the kind of failure that is due to an error, it just means the query did not succeed in storing the record.

    The import is performed in order. The way to tell if a record import failed is that it will be missing from the database after the import. So, starting from the first record in the CSV, check that each one was imported. Eventually, you will find a record that failed to import. Construct a new CSV that includes only that one record.

    If importing the record that failed previously on its own works fine, that suggests that the CSV may be too large to import all at once.

    This is a bit mysterious, but looking at the failed query will help.

    Thread Starter archerdesigns

    (@archerdesigns)

    I made a little progress…turns out the database doesn’t seem to like some special characters:

    ° degrees
    µg micrograms
    ± plus/minus
    * asterisk

    But I found 3 records that have none of these that consistently fail. 2 are almost identical and all 3 don’t have anything that stands out that looks “bad”. A verbose of the log doesn’t say much or even tell me which field is failing the import. I cleared the log, then attempted to import those 3. This is all it gives me:

    Log file initiated at: 5 April 2021 10:50 pm UTC
    [04/05/21 10:50pm UTC]
    PDb_Session::get_alt_session_id obtaining session id by alternate method: b1b2f8826ed0007027e233d3aa75adaf
    Log file initiated at: 5 April 2021 10:50 pm UTC
    [04/05/21 10:50pm UTC]
    PDb_Session::get_alt_session_id obtaining session id by alternate method: b1b2f8826ed0007027e233d3aa75adaf
    Log file initiated at: 5 April 2021 10:50 pm UTC
    [04/05/21 10:50pm UTC]
    PDb_Session::get_alt_session_id obtaining session id by alternate method: b1b2f8826ed0007027e233d3aa75adaf
    Log file initiated at: 5 April 2021 10:50 pm UTC
    [04/05/21 10:50pm UTC]
    [04/05/21 10:50pm UTC]
    Participants_Db::process_form storing record:
    [04/05/21 10:50pm UTC]
    [04/05/21 10:50pm UTC]
    Participants_Db::process_form storing record:
    [04/05/21 10:50pm UTC]
    [04/05/21 10:50pm UTC]
    Participants_Db::process_form storing record:
    [04/05/21 10:50pm UTC]
    PDb_Session::get_alt_session_id obtaining session id by alternate method: b1b2f8826ed0007027e233d3aa75adaf
    Log file initiated at: 5 April 2021 10:50 pm UTC
    Plugin Author Roland Barker

    (@xnau)

    It looks like we’re missing the actual error. The query object is not showing a query at all, so something pretty basic went wrong there. php errors like that often don’t end up in the plugin debugging log because they are happening before the logging is initialized.

    Catching those errors will require you check the php error log on your server.

    As long as the CSV file is using UTF-8 encoding, characters like that are not a problem (I tested this). Characters that are meta characters in the CSV (comma, line returns, double quotes) can cause problems because they can break the CSV syntax if not properly enclosed.

    Making sure your CSV is UTF-8 encoded may help, some apps require you to specifically configure them to do that.

Viewing 5 replies - 1 through 5 (of 5 total)

The topic ‘Import csv with errors’ is closed to new replies.