Here's a KeyShot re-creation of a test I put together a couple of years ago in Nuke, using camera projection mapping.
The "SurrattHouse" image was a photo I found online. I did a camera match in Nuke and recreated a very simple model of the shape of the house. Using Nuke's camera projection node, I was able to get a decent mapping of the source image onto my model. In Nuke, I could just duplicate the texture projection camera and animate the copy as my view camera. The texture stayed mapped to the house, while the view camera could move. Obviously, you can only move within a limited range of motion before the illusion is destroyed, since you'll see stretched pixels and missing information on the sides. But, with a bit of retouching work in parts of the texture map image - and careful choreography of the view camera, you can get a very convincing scene!
And, no, you can't do this with planar mapping. You need mapping of the pixels using the original camera's focal length, angle of view, distance from the objects, etc. - a recreation of the physical layout.
The only thing missing in KeyShot is the ability to separate the mapping camera from the viewing camera.
