This seems to be a pretty difficult topic, does anyone have any views and how best to implement an anti-cheat system, analyzing dll's seems to be one way, although I haven't really looked into how this is done, but there are always plenty of ways to hack.

One way to make cheating as difficult as it gets is to simulate most of the game on your server. The client will only send input commands, and the server will only return "opcodes" that tell the game what to display. Depends on how much computation power your servers have. I'm assuming you are looking into some kind of server-based game, for games that are playing via LAN things get more diffucult, since the game-logic has to be executed on the hosts machine, which effectively makes cheat-protection very hard to accomplish.

Many other people will give answers on specific issues as well as general ones, but a few points to consider:

Who are you deploying to? Your target determines you vector of attack, as well as the purpose of your system. For example, a transaction system will be a much more rewarding target than the chat system on most game servers, but nearly always more difficult to compromise.

The less you trust from the client, the less you have to worry about. In limiting the client's control over the game to a restricted set of inputs, you only have to verify the integrity of those inputs. The client is always going to cheat, if the client wants to cheat. There is no fundamental way of determining if they are cheating, in addition to incurring some usability issues. Best ask yourself why, not how!

The obvious solution is to move logic server side. Personally, I feel I wouldn't be exploring my full potential as a game developer if I opted for that solution, because I am a huge fan of taking advantage of the hardware gaming enthusiasts have at their disposal - be it in their desktop, or smartphone / mobile device. The more strain I can put on a client, the more clients I can have on a server. Not that this is necessarily the end goal of creating all multiplayer games (massive games).

One strategy that I have yet to see employed is the gaming equivalent of deep packet inspection - essentially analyzing the data coming from clients to make sure it 'adds up'. Implementation is specific to each game, and the forms of cheating that could take place. From analyzing the screen, to player's actions, and gauging the feasibility of everything detected - on the server, or in the case of a P2P game on other clients.

But this may be way outside the scope of possibility for most. I'm just a fanatical crackpot, most of the time.

Packet inspection is good, and is done on professional games, but depending on the game, won't solve everything. Aim hacks in an FPS wouldn't be stopped, as just about any facing is valid on any given turn. Wallhacks also wouldn't be stopped, as they are entirely client side. I am really curious about the latest BattleField cheats, as they look like something that should be easily detected server side. Hacks that kill everyone on the opposing team, one shot kills with any weapon, all seem like things stopped by making sure the packets make sense, or that the information the player is sending is more minimal. (eg Don't send, "I hit player Q with Weapon X for 9800 damage", send "I fired Weapon X in this direction")

Game design can overcome some of those issues. World of Tanks, for example, the maximum turn rate of the gun is very slow, so aim hacking wouldn't buy the user all that much of an advantage. Unless it's detecting how fast the object is moving and aims ahead for the player, that would give it a small advantage. Also due to the plethora of models in the game, each with different weak points, that makes it tougher to write a generic all purpose aim hack. (But not impossible)

Wallhacking would also be more difficult in World of Tanks, as the server doesn't send information to the client on units that are not visible to the player's team. And when they are visible to the team, then the players are allowed to see their outlines through walls. There are downsides however, in that in a laggy environment, a player may get shot at by a unit that it could normally see, but the server hasn't yet been able to tell it where that player is.

Perfect anti-cheating is impossible. Games that are highly competitive and rely on client-side controls (like aimbots) will be failures online, unless other means of separating cheaters behaviorally is done. For example, if you have leaderboards and matchmaking that tend to match cheaters with other cheaters, most players will have a fun experience (but the leaderboards will still be off.)

If you really care, you could look into solutions like Punkbuster or Valve Anti-Cheat, but there's an ever-escalating war between them and the cheaters (kind of like between DVD publishers and DVD rippers.) The repertoire includes anything from DLL scans, to screen shot uploads, to input statistics, to network sniffing, to who-knows-what. The problem is, you'll start hurting real customers -- what if someone using an eye-controlled joystick is playing your game, and the DLL used by that joystick is not recognized by the anti-cheat? Suddenly, the paraplegic game player is considered a cheater.

Build a fun game that people will like playing, and make sure you match players who enjoy playing each other together. Put rules enforcement of important stuff (like progress, in-game money, etc) on the server, and engage with the community to figure out what really needs fixing after that.

Games that are highly competitive and rely on client-side controls (like aimbots)

What would a game be that didn't have client side controls?

League of Legends. Everything, even the pathfinding is done by the server a second time. It also doesn't have any cheat detection apart from checking the camera position.

The problem they are facing lies more in the fact that people dodge games, which is why they ban player that leave games too often.

What would a game be that didn't have client side controls?

I left some implicit context out:

Games that are highly competitive and rely on client-side controls (like aimbots)

Games that are highly competetive in how they react to precision and immediacy of client input, and thus are sensitive to tools like aimbots, ...

In Chess or Poker, it doesn't matter how fast/slow/precisely you click the game window. In Mech Warrior Online, it's less important than in Unreal Tournament. MWO can still suffer a bit from aimbots ("cored every time") but not as much as UT.

...analyzing dll's seems to be one way...

Careful with this though! Every user might not have the same version of some common DLLs like DirectX-related DLLs.
Even further, some applications modify DLLs intentionally for good reasons.

Do you use Steam? Shift+Tab brings up the in-game Steam overlay. This is done via hooking into or "hacking" a DirectX dll.

Does your game support non-XBox controllers? Many consumers, myself included, use standardized non-XBox controllers that nobody seems to support anymore. So we have to use an controller emulator (which is legal) to get our standardized "regular" controllers (like my Logitech Dual Shock 2 - a popular model) to pretend to be an XBox controller. I (and many others) use x360ce for this purpose, which pretends to be an XBox controller by "hacking" the Direct Input DLL, replacing it with x360ce's own artificial DLL. This is legit and not too uncommon.

Auto-banning users because they plugged in the wrong kind of controller is bad for business.

Cheating will occur regardless. What I'd do is [in order of importance]: A) try to minimize the effect on legit players of cheaters and griefers, so legit players' fun isn't ruined, B) have alot of detection tests, but only ban for "guaranteed" cheaters, not "maybe" cheaters, and C) try to make cheating give as little benefit as possible.