I think it might be better to use a different 2D engine, since this one is unsupported and seems buggy..

Migrating all my code to another engine would be close to impossible for this project unfortunately...

What pixel sizes should I make the window to avoid the problem? Im unsure on how to progress.

Just setting it to much larger doesnt help (which seems to me would mean there is enough pixels to not need to press the screen together, thus avoiding it skipping lines? But doesnt work either).

Another clue (maybe): there is always 10 skipped/compressed lines. If i set the window to 200x200 or 400x400 or 1600x900 doesnt matter: in windowed mode there is always 10 lines of mixing/compressed pixels as a result of this shrinking/fitting into the client area.

Both my clipping area (viewport) and client rect is 1600x900 but still the same problem in windowed mode... Is it possible to disable the windows feature that it squeezes it into the client rect? Just clip away any pixels outside? Maybe that could do it.

Have you tried not adjusting the sizes you pass to System_SetState?  As was mentioned earlier, it seems like the engine accounts for window styles, since the whole point is to abstract the OS dependent code away.  Try just passing in the exact size you want your viewport to be, so if you want 800x600, pass in 800x600 directly, no adjustment.  It seems like there's still some discrepancy between your client area, the render target, and/or your viewports/projection calculations.  This all need to be set correctly and equal to prevent the types of aliasing issue you are seeing.

@xycsoscyx Yes I originally used the same code (setting screen width and height) for fullscreen and windowed mode. I think the windowed mode started being a problem when I started compiling in win 10 (win 7 before) so maybe I need to do something manually now that was automatically taken care of before. 

1. If I adjust the setState so that "client area" is exactly 1600x900, and my clipping area is the same, why do I still get this problem? Maybe render target? (I've never used that so need to find out if it's even possible to change it)

2. When I add lots of pixels in both width and height of the window (e.g. 100x100 extra pixels), why does it still need to squeeze it? It should now have plenty of pixels...

3. Is it possible to disable the "feature" that the OS squeezes at all? Instead allowing some pixels to be cut (if these dimensions are not exactly the same according to the OS, which seems to be the problem here).

I really appreciate (and need) the help!

That sounds like an engine problem really. Windows 10 has different window border sizes than previous Window's but the engine should take care of that for you really. Could be that they hard coded the border sizes.

You have the source code:

http://kvakvs.github.io/hge/

So it should be easy to fix things to do with window sizing, they already have

Quote

fix for window borders on Vista/7

The relevant code is in

https://github.com/kvakvs/hge/blob/master/src/core/system.cpp

In function System_Initiate():

    // Create window
    // Thanks to Bryan for window rectangle fix in later Windows systems 7,8
    // http://i-am-bryan.com/webs/tutorials/hge/fix-hge-window-sizing-bug/
    rectW.left      =   GetSystemMetrics(SM_CXSCREEN)/2 - nScreenWidth/2;
    rectW.top       =   GetSystemMetrics(SM_CYSCREEN)/2 - nScreenHeight/2;
    rectW.right     =   rectW.left + nScreenWidth;
    rectW.bottom    =   rectW.top  + nScreenHeight;
    styleW          =
        WS_POPUP|WS_CAPTION|WS_SYSMENU|WS_MINIMIZEBOX|WS_VISIBLE; //WS_OVERLAPPED | WS_SYSMENU | WS_MINIMIZEBOX;

    // Fix for styled windows in Windows versions newer than XP
    AdjustWindowRect(&rectW,styleW,false);

    //Fullscreen
    rectFS.left     =   0;
    rectFS.top      =   0;
    rectFS.right    =   nScreenWidth;
    rectFS.bottom   =   nScreenHeight;
    styleFS         =   WS_POPUP|WS_VISIBLE; //WS_POPUP

@lawnjelly Thanks!

But what do I need to fix? I can find out the client area with spy and I already modify it to be exactly 1600x900. What in the above code is "incorrect" for win 10? (and can't I just "disable" this squeezing behaviour by the OS, it seems to just cause problems, I'd rather it just cuts away some pixels than squeeze).

I defer to the other guys and google on this, I have heard it is a bit of a black art getting window size info from windows. Although if that was the code you are using you posted on the first page of the thread, then it looks like you are calling AdjustWindowRect twice for a start (once in your code, once in their System_Initiate()), whether this is your problem or not I don't know.

If you compile the source to the engine you might be able to step into it from your code and debug to find out what is going on.

Hm... It seems that I have the same problem, but only with certain resolutions. I'll also try to find a solution :))

@Vladimir128 It seems to be border-calculation that is wrong in System_Initiate() for windows 10. But i cannot recompile the engine so Im stuck...