Sorry for the late response.
I looked at your posted file and noticed the HDRI environments are pretty large, 16,000mm.
If I understand your problem correctly, try making the HDRI environment smaller, 1200mm for more control. See attached render, Adjusted HDR, and KSP file.