Update: If anyone else has issues.
The issue seems to be with CloudFlare, and using their option of Rocket Load. Turning it off, seems to have fixed the issue.
Update: I found the fix, if anyone else has this issue.
Add the following into a CSS file (like CodeSnippets), and get the color you want. Note, this issue will only occur when you turn it into a static site.
/* Dark mode override /
.wp-dark-mode-active .wp-block-social-links .wp-block-social-link.wp-social-link .wp-block-social-link-anchor svg {
fill: #d6d0c6 !important; / Change to your desired dark mode icon color */
}