UPDATE: Sorry mods, I wanted to post this in the Gameplay section, not sure how to move or delete this post. Sorry

I'm currently working on creating spaceship controls inspired by Elite Dangerous, although I'm aiming for something a bit less complex. The main idea is to allow players to control their ship from a first-person perspective, just like they're sitting in the cockpit. The ultimate goal is to enable movement and navigation in a zero-gravity 3D space.

At the moment, I'm using Euler angles for rotations. However, considering that spaceships can rotate in various ways, I'm contemplating whether it would be a good idea to implement Quaternions instead. They might offer a better solution for handling the ship's rotations, given their flexibility.

So far, I've managed to get the ship moving forward and backward. However, I'm facing some challenges in devising effective methods for steering the ship around in 3D space. Any advice or tips you can share would be greatly appreciated. Here are the control functions I've developed thus far:

This is my control code:

  void updateShipControls(Game::Entity::Ship &ship, Time &time, GLFWwindow *window) {
  
    float dt = (float) time.getDeltaTime();

    int upKeyState = glfwGetKey(window, GLFW_KEY_UP);
    int downKeyState = glfwGetKey(window, GLFW_KEY_DOWN);
    int leftKeyState = glfwGetKey(window, GLFW_KEY_LEFT);
    int rightKeyState = glfwGetKey(window, GLFW_KEY_RIGHT);

    if (upKeyState == GLFW_PRESS) {
      ship.thrust -= 0.1f;
    }
    if (downKeyState == GLFW_PRESS) {
      ship.thrust += 0.1f;
    }
  
    ship.acceleration = ship.transform.forward * ship.thrust;
    ship.velocity += ship.acceleration * dt;
    float speed = glm::length(ship.velocity);
    if (speed > ship.maxSpeed) {
      ship.velocity = glm::normalize(ship.velocity) * ship.maxSpeed;
      speed = ship.maxSpeed;
    }
  }

This is the struct I'm using for the ship:

    struct Ship {
      float thrust;
      vec3 velocity;
      vec3 acceleration;
      quat orientation;

      float maxSpeed;

      float mass;
      float health;
      
      float fuel;

      Transform transform;      
    };

And this is the Transform struct, just in case somebody wants to see it:

    struct Transform {
      vec3 position = vec3(0.0f);
      vec3 scale = vec3(1.0f);
      vec3 rotation = vec3(0.0f);

      vec3 forward = vec3(0.0f, 0.0f, 1.0f);
      vec3 up = vec3(0.0f, 1.0f, 0.0f);
      vec3 right = vec3(1.0f, 0.0f, 0.0f);

      vec3 worldUp = vec3(0.0f, 1.0f, 0.0f);

      mat4 model = mat4(1.0f);
    };

Your “Transform" is actually part of Ship: a bunch of interesting vec3 values, not a transform in the common sense of the word. Fields forward, up, right and rotation are redundant with orientation, while worldUp makes no sense whatsoever (your reference frame is constant and global, not a property of a spaceship) and scale doesn't seem relevant (you might scale instances of a stock 3D model to adapt it to different uses, like round, oblate and prolate asteroids sharing the same mesh, not the position or shape of an object).

You need three control axes (presumably three button pairs or joystick axes) for pitch, roll and yaw, not just left - right. Moreover, the ship doesn't necessarily thrust forward: it is only a common special case.

I'm going to suggest you implement pitch, roll and yaw. I did a jet pack controller. It's somewhat different since you only really have to worry about yaw, because you want your character to aways be vertical (outside the animation), but the idea is the same. You have angular accelerations for each of these, up to a max angular rate of change. You can allow a player to change the max rate of change with a slider up to a point.

Now when you press a key, for instance turn right, you accelerate up to the max yaw rate, and when you lift off the key you accelerate down to zero. I don't think you want to try to make a player do that by hand as it will be very frustrating, but you can if want. However, it will make achieving a zero rate of change very hard and it's not really any more realistic since a computer in a ship could do the same thing as I just implemented.

Your acceleration forward will be just that, and your speed can be unlimited (constraints of the game not withstanding). In addition, you can also add docking controls which would let you slide right and left, up and down and backwards and forwards at a constant rate. These would work like what I mentioned above. You press they and it accelerates up to a predefine speed (again which can be controlled by a slider somewhere). You lift off the key and it goes back down to zero, zero being relative to the speed at which you started.

Keep in mind for orbits there are some odd things. For instance, if you want to catch another ship in an orbit you actually have to slow down. This seems counter intuitive but if you think about it makes sense. Slowing down will drop you to a lower orbit which is faster, and then when you catch your target you will increase speed which brings you back up to a higher slower orbit. If you calculate gravity on every time step it should do this for you, so it's not really that hard to implement.