Maybe you could have a 'concentration' bar of some sort. Umm. In Rainbow six Vegas, when you get shot or (I think) if there are bullets flying around you, your vision gets blurry. You can still aim and shoot, but it's nearly impossible to see/hit anything when this happens (if it wasn't for the fact that the cursor turns red while aiming at an enemy... ..). If you are desperate (like when enemies are closing in on you) you can try to fight anyway, but usually it's better to just sit tight behind some cover and wait until you've regained your focus.

Maybe you could do something similar and/or involving player skill. For example, when the player wants to cast a fireball, some combination of say 3-5 keys would need to be pressed if at full concentration, but if there are explosions all around him and arrows flying everywhere, he would instead have to correctly enter a combination of 10-15 keys, and/or or do it faster, and if he fails, it backfires on him. If you don't want to involve player skill, you could vary the casting time depending on concentration/whatever. Maybe it's the 'magical stability' of the area that matters. In a calm area, you can do almost anything, but in a battlefield with lots of spells being cast, magic becomes increasingly more unstable, and therefore harder/riskier/whateverier to use..

[font="arial, verdana, tahoma, sans-serif"]I like this idea and I am trying to choose between two different systems to prototype. Note: created spells can be hot-keyed.

Creating Spells[/font]
Spell Creation Menu: The player would choose the element Ignis[Latin for fire, glow, lightning] There would be opposing sliders for several factors: Controlled / Chaotic Nature, Compact / Dispersed, Area of Effect. So:

Ignis + controlled + compact + large AoE = a large glow of light illuminates the area.
Ignis + chaotic + compact + small AoE = a fireball forms in his hand.

Pros: Gives player more choice of their ideal spell
Cons: Lots of programming + still clustered inventory of spells (unless there is a delete spell function)

Modifier Spell Creation
The number keys each respond to an element that you have learned. Pressing any key with an element assigned enters a 'spell slinging mode' (i.e. you first had a sword equipped a now you have a spell equipped). The mouse and control button will modify how the element is cast. So:

Ignis + left mouse (Controlled) and control held (Compact) = orb of light;
Ignis + right mouse (Chaotic) and control held(Compact) = fireball.

Pros: On the fly control at any moment
Cons: Complex; in the heat of battle, player may become confused and forget combinations.

Risk vs Reward
Controlled magic = less risk, but little to no damage.
Chaotic magic = more risk, more power.
The element may also change how a spell affects your body.

Example: Ignis + large amount of chaos
[Success] You create a fireball, but any weapons you try to carry will melt for a while.
[Failure] Entire body is charred.

Example: A Sanies[corruption, poison, venom] + large amount of chaos
[Success]Cloud that poisons enemies, but hand is poisoned and unusable for a while.
[Failure] Cough up blood, slow movement, blurred vision

To balance the relative power and usefulness of spells "in the small", you can tweak the warm up delay between starting the spellcasting and the first effects and the cooldown delay between finishing casting a spell and starting to cast it again.
For example, a fireball that would be a first choice attack in many situations (relevant for most enemies and battlegrounds, high damage/mana cost ratio, high DPS, etc.) could become more specialized if a long warmup restricted it to ambush or sniping situations, excluding quick reactions, or it could be reduced to a partial answer with a long cooldown requiring the use of something else while waiting.

Don't have time for an in-depth POV I'm afraid. But the approach I'm taking in my console-style RPG is to remove the separation between normal physical attacks and magical ones. Hitting somebody with a sword/casting a fireball at them does a different type of damage, but at the same cost. Perhaps in your game a generic energy bar may be suitable or some other system. The important thing is by treating them as a part of the same system, you remove a lot of the balancing problems. In my system, a simple sword strike will do 10HP damage against one type of opponent, whilst fireball magic will be ineffective. But then you come up against someone with armour on and the sword strike is ineffective where the fireball is not.

And this raises problems such as "But there's only one way to swing a sword, and things like AoE to consider with magics". To which my response would be balance it with different non-magical skills. Dragon Age did something similar I think in terms of the actual skills your characters could use. Your tank had everything from powerful sword strikes to weaker area of effect ones, which is exactly the kind of thing you typically do with magic. So that would be my solution, to simply treat them as the same system.

Try reading The name of the wind, there is an interesting magic system, wich looks almost like science. People can establish links between objects, and what happens to one object, happens to the other, but you have to use the energy for both; for exmple you have two bars of metal linked, you lift one but it weights more. The links have an efficiency rate, so the bar you're lifting weight not 2 but 3 times its weight. The efficiency depends on the similarity of the objects and the skill of the "archanist". In a similar way, you see an orc with a metal axe, so you take a small piece of metal, link it to the axe, and then cast the piece of metal into a bonfire, the axe melt and burns the orc hand. It's a bit like voodoo.

There was also a concept that sometimes it was easier to use non-magic techniques; for example there was a rune that could stick two bricks together, but they would join so fast, they would crash at each other and you have to put a lot more runes to balance it so it's so camplicated that's easier to use mortar. In a game you can have a wizard to use a very powerful explosive spell to destroy a city or fortified position, or yuo could use a nuke.

I may consider those ideas, but right now I'm on the fence between two systems.

Should I have a unique create-a-spell menu or should I let the player equip an element and be able to manipulate it during battle?