i implemented my Game UI in openGL using Orthographic projection, I have windows that varies from fullscreens to pop-ups to panels which renders on top of a 3D game.

what I did was I had a working dimension where 2D arts will be based on that, the working dimension we decided is HD 1920x1080p. Currently, if the screen size or aspect ration is diferent from our working width,

we just resize the entire screen and UI, since it is rendered in an FBO, we just resize the FBO based on the current screen size. Unfortunately some of our fullscreen UIs looked stretched.

One example is we have a screen that is full screen and has a huge circle in the middle, when run in mobile with different aspect ratio (its a mobile game engine by the way), it looks stretched.

I read somewhere that rendering the 3D game viewport should be separate from 2D to prevent this. But how to handle this situation?

Should the UI maintain its size/dimension if the screen is different? say if our working size is HD, and the screens is greater than that, then the UI will no longer be displayed as fullscreen?

Or should I render the screen background in fullscreen still but the content element should be maintained including positions of UI elements like buttons? On my example above is the huge circle in the middle of our UI in a full screen window.

Whats the best approach on this? or are there any resources to read online about this?

most of the game engine making book including game engine tutorials online are focused on the 3D stuff and does not include the UI part of it.

your help will be greatly appreciated.

Please show us the code that you use to initialize the projection-model-view matrix.

It would be useful to see a picture of UI you have. A few large simple UI elements generally don't have as noticeable scaling issues as highly detailed text-heavy interfaces.

In my experience complex UIs show up the worse of scaling with blurry icons, text etc. even if aspect ratio issues are ignored. This is much more noticeable with small text/icons and/or on displays with a lower pixel density. Rendering to the native resolution will avoid this, instead you scale the size of the elements, fonts, etc. before rendering.

Also often UI elements will be “docked”, e.g. centre, top-left, top-right, etc. This avoids most aspect ratio issues, as changing the aspect ratio changes the spacing between these elements, but individually they stay 1:1 and with the same layout. This would solve your “circle” issue, a circle in the centre will always be a circle, it might have a different radius in pixels, and the relative space above or too the sides might change.

When picking the scale you want a single scale for both horizontal and vertical. Given a different aspect ratio, you might pick the smallest. say 1920x1080 is your native, but the phone is 2960x1440. The two scale factors are about 1.54 and 1.33. So take 1.33 as your scale, and have extra spacing in the other direction.

For full screen background images and such you might do the opposite using the larger scale then render it centred, on different aspect ratios part of the image will be cutoff, for backgrounds this is usually perfectly fine and preferable to having black bars (letterbox) or such.

The other question is just generally what scale do you want to use. Simple game UIs will often scale to fit the screen, but something more complex with a lot of text, maybe scrolling/pages might be more advanced and instead scale based on DPI or such, then dynamically lay itself out to fit the screen. This will let larger phones/tablets display “more stuff”, but very high DPI phones won't display too small to be readable.