To sum it up and take into account explaining it to the player.

The game should tell the player that:

- 100 ships battle group is best

- 101-200 ships battle group will get some small/reasonable penalties, depending on the number of ships (dimishing returns)

- 201+ ships battle group is bad and will get big penalties

I mean, the player needs to know it. It must be in some tooltip/whatever.

So, we have two sets of penalties, depending which thereshold we exceeded.

SMALL PENALTIES:

- dimishing returns of more ships (lower firerate, lower accuracy, some ships skipping a turn)

BIG PENALTIES (cummulative with the small ones):

- higher proportional casualties (so 300 ships group losing more units against 50 ships enemy than 200 units group), it's important so a huge stack is not safer; maybe some sort of area effect weapons (where first 100 shipos are immune to it)?

- friendly fire (like all ships above 200 have a 10% to target a friendly ship instead)

Do you have other ideas for these penalties and how exactly these could work? Or other comments?

Needing BOTH space-superiority and ground-combat superiority to take a planet only makes bigger stacks superior,
since if the fleet fails at one of the two, the other is useless as well.