I can sympathize with the sentiment, but you have to ask yourself why the industry is in decay. Part of the reason D&D and other tabletops have declined in popularity is because other things have come along -- back in the day, if you wanted to play a fantasy-based role-playing game with deep plots, tabletops were your only real option. Today, MMOs like WoW provide those same themes, role-play, similar social interaction, and the accessibility of playing whenever you choose without having to herd 4+ friends together at the same place and time. Modern interactive games also provide more immediate gratification. As a video gamer who recently gave D&D a good shot, one of the things that struck me was the glacial pace with which the game moves -- with 5 other players, a single turn takes an hour, 50 minutes of which I'm essentially idle and disengaged. In an MMO or any other game I'm always actively doing something. All of this is to say nothing of all the other competing styles and genres of games available today, and other kinds of entertainment that have never been more accessible and immediately available.
I think the more classic experience does still appeal to some people -- the industry may be a smaller part of the overall entertainment pie today, but I'd hazard a guess that its probably as large or larger than its ever been in total numbers today. Large publishers, indeed large companies of any kind, tend to ignore the small slivers, which makes them a sort of "cottage industry" as far as the wider gaming industry is concerned, even though the sliver may be entirely viable in its own right.
What I'm getting at is that you may very well be onto something, but its naive to think that everything is horseshit just because its mainstream -- horseshit doesn't sell like mainstream products do, they may not appeal to you, you may be desperate for something different, but that's clearly not what the mainstream audience wants. Is that lamentable? Probably, on some level. Anyhow, if the impetus behind your business plan is that the mainstream is crap, and there's a conspiracy (or unwillingness) by the mainstream publishers to keep table-top-style games from returning to the mainstream, I think you'll be disappointed with that thesis.
Personally, though I don't much like tabletop games, I think there's actually a market to be had in creating an online platform for these kinds of games -- that is, one which allows people across large distances to play together without being too difficult to use and to author content and rulesets for. That's of course different than what you seem to describe.
But regarding whether you need AAA graphics, real-time 3D, or other chrome, I say no. What you need is art that looks professional for what it is, and which has style. People know production value when they see it, regardless of what form it takes. Its that property of the visuals (and audio) that says to people "this game is worth my time" -- shoddy production values say "not even the author thinks this game is worth his time". Plenty of successful games have "simple" graphics, be they in 2D or 3D, and nearly all of those have obviously high production value. Ultimately it comes down to time and resources, and simpler graphics with higher production value are more appealing than complex graphics with low production value.