UTF-8 problems, again

Douglas “BearlyDoug” Hazard
(@bearlydoug)

4 years, 10 months ago
Last July, @ivijanstefan provided a fix for some characters (see this thread), however, it doesn’t go far enough.

Stored in DB:
Jennifer, my cousin’s daughter–although

Actual exported line:
Jennifer, my cousinâ€™s daughterâ€“although

What’s interesting is that it’s recommended to not use the HTML entity for the comma, rather, to use the character entity, instead. See this thread on Microsoft’s support site.

Can we get a manual fix for this, ASAP, please?

(I appreciate your time and help with this, Arshid. 🙂 )

Thanks!
- This topic was modified 4 years, 10 months ago by Douglas "BearlyDoug" Hazard.
- This topic was modified 4 years, 10 months ago by Douglas "BearlyDoug" Hazard.
- This topic was modified 4 years, 10 months ago by Douglas "BearlyDoug" Hazard.
- This topic was modified 4 years, 10 months ago by Douglas "BearlyDoug" Hazard.

Viewing 8 replies - 1 through 8 (of 8 total)

Plugin Author Arshid
(@arshidkv12)

4 years, 10 months ago

Please upload to Google drive and check it.

Thread Starter Douglas “BearlyDoug” Hazard
(@bearlydoug)

4 years, 10 months ago

Upload what? I didn’t provide any fixes. I’m asking that this be fixed. 🙂

Plugin Author Arshid
(@arshidkv12)

4 years, 10 months ago

Please upload CSV to google drive and check it.

Thread Starter Douglas “BearlyDoug” Hazard
(@bearlydoug)

4 years, 10 months ago

I cannot do that, because that would violate our privacy policy.

I did, however, quote a part of one of the lines, directly from the database, and from the CSV above.

Plugin Author Arshid
(@arshidkv12)

4 years, 10 months ago

Create a test form from localhost and try it.
Thread Starter Douglas “BearlyDoug” Hazard
(@bearlydoug)

4 years, 10 months ago
Some additional information. The file is being exported as a CSV in UTF-8 format:
```
root@lb17 17:41:41 
 > /data/www # file -i cfdb7-2021-08-11.csv 
cfdb7-2021-08-11.csv: text/plain; charset=utf-8
```
HOWEVER, if you open directly after downloading it, it opens it up in a non UTF-8 format. My coworker was kind enough to provide a workaround for this:
```
Instructions are:
1.	Open new excel file
2.	From the Data Tab -> Click “From Text/CSV”
3.	Point to the file downloaded
4.	On the Next Page -> file origin to 65001-Unicode (UTF-8)
5.	Click load
```
You might need to do some additional digging into this to see if you can force Excel to automatically open in UTF-8 format.

FYI, my excel version is “Microsoft® Excel® for Microsoft 365 MSO (16.0.14228.20216) 64-bit”
Thread Starter Douglas “BearlyDoug” Hazard
(@bearlydoug)

4 years, 10 months ago
UPDATE! I have a workaround! Two file edits, and this will allow CSV files to be directly opened within Microsoft Excel without having to use the special instructions above.

FILE: inc/admin-subpage.php (lines 462-464)
SEARCH FOR:
```
        echo "&emsp;<a href='".esc_html($_SERVER['REQUEST_URI'])."&csv=true&nonce=".$nonce."' style='float:right; margin:0;' class='button'>";
        _e( 'Export CSV', 'contact-form-cfdb7' );
        echo '</a>';
```
ADD BEFORE:
```
/***
 * @BearlyDoug (on ww.wp.xz.cn) Edit: Export as Excel CSV file...
 */
	echo "<a href='".esc_html($_SERVER['REQUEST_URI'])."&csv=true&excel=true&nonce=".$nonce."' style='float:right; margin: 0 0 0 7px;' class='button'>";
	_e( 'Export as Excel CSV', 'contact-form-cfdb7' );
	echo '</a>';
```
FILE: inc/export-csv.php (lines 122-123)
SEARCH FOR:
```
                    foreach ($resultTmp as $key => $value):
                        $matches = array();
```
ADD AFTER:
```
			/***
			 * @BearlyDoug (on ww.wp.xz.cn) Edit: Export as Excel CSV file...
			 */
			if(isset($_REQUEST['excel']) && $_REQUEST['excel'] == "true") {
				$newValue = mb_convert_encoding($value, "cp1252", "UTF-8");
				$value = $newValue;
			}
```
There’s probably a more efficient way to handle this… possibly by simply changing the charset to cp1252 (AKA Windows-1252). I’ll test that functionality now, and will report back in a new reply.

Hope this helps!
- This reply was modified 4 years, 10 months ago by Douglas "BearlyDoug" Hazard.
- This reply was modified 4 years, 10 months ago by Douglas "BearlyDoug" Hazard.
- This reply was modified 4 years, 10 months ago by Douglas "BearlyDoug" Hazard.
Thread Starter Douglas “BearlyDoug” Hazard
(@bearlydoug)

4 years, 10 months ago
Okay, I’ve tried changing
```
header("Content-Encoding: UTF-8"); 
header("Content-Type: text/csv; charset=UTF-8");
```
to cp1252, CP1252, windows-1252 and Windows-1252 with no luck.

I’ll have to defer to you how best to implement my suggested change in a more efficient manner, @arshidkv12. 🙂

Viewing 8 replies - 1 through 8 (of 8 total)

The topic ‘UTF-8 problems, again’ is closed to new replies.