So clicking 'into' the game explorer.exe, prompted explorer.exe to try to have its border hidden, but that just hid the taskbar icons. So I think there was something happening where once WindowedBorderlessGaming was launched, even if I closed it after, the attempt to make explorer.exe itself borderless left its effects. Once I removed it and restarted, the issue was gone. I found I accidentally added 'explorer.exe' as one of the 'games' it manages. Which allows borderless windowed for games that don't natively have it.