I presume you are running a 64-bit build of Spike2 and have installed 64-bit Microsoft DirectShow compatible codecs?
The fact that the open Multimedia button is enabled just means that there is a video file with a matching name in the right place, so Spike2 will attempt to open it. To open the file we build a network of DirectShow modules to connect the file to a splitter and then to the video window and DirectShow figures out which codec is required to make the connection.
When I have this kind of problem I use a great program called GraphStudio Next, which you can download in both 32-bit and 64-bit versions.https://www.videohelp.com/software/GraphStudio
You should use the version that matches the build of Spike2 you are using... I'm presuming that you are using a recent version of Spike2. GraphStudio Next is a bit technical, but as you sound familiar with H264/5 this is probably no problem for you.
If you can get your file to play using this program we should be able to get it to work in Spike2...
To emulate what Spike2 does, open the file in GraphStudio next and attempt to connect it to the Enhanced renderer... If you need clues as to what to connect, you can use Spike2 to replay an H264 file that does work and then use the "Copy the current frame as a bitmap" button, but with the CTrl key held down. This will copy the structure of the 'graph' used to replay video to the clipboard. In my case I get:
File: E:\Users\Greg\Documents\Bugs\Video\Testmp4goto-1.mp4 MP4 File
Mpeg4s Decoder DMO
Enhanced Video Renderer
Video: FPS 30, 640x480
The MP4 demultiplexor we use appears in the GraphStudio list as "GDCL Mpeg-4 Demultiplexor". It is possible that this is incompatible with your H265 format file. If you cannot connect using this device we have a bit of a problem...
If you can find a combination of filters on your machine that does work in Graph Studio it is possible that we can add an option to the video replay to allow you to specify which demultiplexor to use...
The other possibility is to use your H265 codec with S2Video to record and compress. You would need to have a 32-bit build of the H265 compressor installed for this to work as S2Video is a 32-bit program.