Doubt there is room for more than current setup, it's already practically limited by slot and energy constraints. Maybe one other layer could be limiting weapons they can fit into a ship (like a small vessel can't have Death Star Cannon)
I thought of that, though the problem is, if I don't want to get stuck saying why each and every ship can and can't use this or that, I need a system for this.
I could use slots of different sizes, or the ship total mass, or even make it so that a player can merge 2 'weapon slots' for bigger weapons, but all of these seem converge in a direction where most players would just go for the bigger badder fatter weapons and ignore smaller ones, no matter how useful they can be. And given how space warfare works, point-defense won't be much use if you can have a large array of long-range cannons to take down all incoming threats.
I'd really like to limit weapons they can fit on a given design, but I need a mechanic that works for this.
You sound like you're going the right way. Playtesting will help expose if there appears to be a dominant strategy. If so, you would need to tweak the design of the weapons, hulls or enemies to try create an even balance.
The thing is, I'm perfectly fine with some weapons being altogether more powerful than others. Let's face it, a gatling gun wouldn't do much harm against a cruiser, and the only reason you're bringing that is if you're worried you might encounter a carrier and need to kill these annoying fighters in a reliable way. In essence, the player wants to willingly sacrifice firepower potential to avoid an encounter he might not be able to handle.
Trying to make a gatling gun stand against mass drivers would be crazy. The only thing it is better at is construction costs.
If you have some A.I., you could also develop a headless simulator and simulate lots of fights with various loadouts. Leave that running for some time to gather some statistics, you might be able to find such issues earlier, allowing you to focus more of your playtesting time on tasks you cannot automate like evaluating how usable and fun your game is.
I already have just that :) Great minds think alike I guess? The main reason I built this system is because I anticipate that a lot of these weapons would show their true strengths in combined arms tactics: a long-range missile frigate with no armor isn't much of a threat, but with any form of tank or harasser in front to delay the enemy, its missiles take their toll before long.
For the most part what you describe seems good: different weapons provide different tactical advantages, and are thus useful in different circumstances.
If anything, I might suggest building on that: have defensive tools (shields, chaff, anti-missile guns, etc.), or sensors that affect the unit's visibility range or what it can detect, or components that affect the unit's movement capabilities (speed upgrades, components that allow passage through certain types of obstacle, etc.), all of which take up hardpoints just as the standard weapons do. Thus a weapon-bedecked Deathstar might be great in a simple brawl, but a quick, sensor-enhanced ship with a single long-range weapon might nevertheless be able to kite it to death, and a shielded one might be able to outlast it. Some components might interact with the environment: if, for example, your game is set in space, you might have occasional ion clouds that wreak havoc with ships; a shielding unit might allow safer passage, while a ramscoop might allow harvesting in order to use the cloud as a resource.
I'm currently satisfied with how I handle 'everything else' aside from weapons actually, and I think that my approach to that will truly generate tactical and strategic depth. This is handled as a separate entity than weapon system slots however, thus I can't really use this recommendation. Toying around with Gratuitous Space Battles (and several actual 4X games) I've come to realize that most people will try to put as many 'guns' as possible, no matter what the ship building rules are, so I've decided to remove friction for them to find exactly what the max is, and focus more on the 'what' they equip. My current challenge now however is to further prevent the 'what' from becoming too common. I feel that playtesting could reveal a design flaw too deep to fix later and want to put an insurance mechanic in there now (and maybe remove it later if I feel it is hardcore) as a contingency to that.
My reasoning is that it's easier to remove something that was put in knowingly that it may be temporary, than it is to add something that was not thought ahead of time to fix a situation.