A well-designed website icon or favicon should work on both light and dark backgrounds and provide sufficient contrast.
Ultimately, yes, the image needs to have sufficient contrast against both light and dark backgrounds. However, even make.wordpress.org uses a transparent PNG with a dark logo and no background.
A lack of contrast, either dark against dark or light against light, would make the link quite unusable on smaller screens. A similar link in the post editor for older WordPress versions could have been invisible. [62625] allowed some of the Site Title to show next to the image, so the toolbar link should not be completely invisible right now, but it's still not easily recognizable. (#66064 should fix the text showing.)