Does anyone have any experience in dealing with copyright of images pulled from a game for the purpose of a game guide book or walkthrough-type of document / site? I have seen countless posts about copyright and sprites pulled from games... But nothing about using those same images for a game guide... Anyone?

It is simple. You don't own the rights to them, so you don't use them. Most companies don't crack down on walk through website so much, but they still could if they wanted to.