Right now I'm attempting to build an engine for rendering a tile based map with an isometric projection. Rendering the map seems to be working now but at the moment, I'm frustrated trying to figure out how to select a tile. For normal 2D this is simple, but I can't seem to figure out how this needs to be done. Since an isometric tile is a squished diamond shape, you need to find out if the click was actually on the tile, or one of the adjacent tiles. I've tried a bunch of methods such as checking if the click was in the polygon, then seeing which quadrant the click was in if the test failed. I've also tried checking if the click was in the polygon, then if the click was outside to check if the click was inside any of the eight adjacent tiles. I really don't know where to go with this so I'll post my code here.

Any suggestions would be appreciated.

Thanks.

package com.isoTest;

import com.badlogic.gdx.Gdx;
import com.badlogic.gdx.graphics.Texture;
import com.badlogic.gdx.graphics.g2d.SpriteBatch;
import com.badlogic.gdx.math.Vector2;
import java.awt.Point;
import java.awt.Polygon;

public class Map {

    
    private int[][] data;
    private Texture img, img2;
    private int TILE_HEIGHT = 32;
    private int TILE_WIDTH = 64;
    private int TILE_HEIGHT_HALF = TILE_HEIGHT/2;
    private int TILE_WIDTH_HALF = TILE_WIDTH/2;
    private int MAP_WIDTH;
    private int MAP_HEIGHT;
    private int X_OFFSET;
    
    public Map() {
        img = new Texture(Gdx.files.internal("isograss1.png"));
        img2 = new Texture(Gdx.files.internal("redcube.png"));
        data = new int[][]{{1,1,1,1,1,1,1,1,0,0,1,1,1,1,1,1,1,1},
                          {1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1},
                          {1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1},
                          {1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1},
                          {1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1},
                          {1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1},         
                          {1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1},
                          {1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1},
                          {1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1},
                          {1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1}};
        
 
        
       
            MAP_WIDTH = data[0].length*TILE_WIDTH;
            MAP_HEIGHT = (data[0].length*TILE_HEIGHT/2)+(16*data.length);
            X_OFFSET = ((data.length-1)*TILE_HEIGHT);
    }
    
    public Vector2 cartToIso(Vector2 init){
        Vector2 iso = new Vector2();
        iso.x = (((init.x - init.y) * TILE_WIDTH/2))+X_OFFSET;
        iso.y = MAP_HEIGHT-TILE_HEIGHT-((init.x + init.y) * TILE_HEIGHT/2);  
        return iso;  
    }
    
    public Vector2 cartToIso(int x, int y){
        Vector2 iso = new Vector2();
        iso.x = (((x - y) * TILE_WIDTH/2))+X_OFFSET;
        iso.y = MAP_HEIGHT-TILE_HEIGHT-((x + y) * TILE_HEIGHT/2); 
        return iso;  
    }
    
    public Vector2 isoToCart(Vector2 init){
        Vector2 cart = new Vector2();
        init.x = init.x-X_OFFSET;
        init.y = init.y-MAP_HEIGHT;
        cart.x = (init.x / (TILE_WIDTH/2) + init.y / (TILE_HEIGHT/2)) /2;
        cart.y = (init.y / (TILE_HEIGHT/2) -(init.x / (TILE_WIDTH/2))) /2;
        return cart;
    }
    
    public Vector2 isoToCart(int x, int y){
        Vector2 cart = new Vector2();
        x = x-X_OFFSET;
        y = y-MAP_HEIGHT;
        cart.x = (x / (TILE_WIDTH/2) + y / (TILE_HEIGHT/2)) /2;
        cart.y = (y / (TILE_HEIGHT/2) -(x / (TILE_WIDTH/2))) /2;
        return cart;
    }
    
    public void render(SpriteBatch batch){
        for(int i = 0; i < data.length; i++){
            for(int j = 0; j < data[i].length;j++){
                if(data[i][j] == 0){
                    Vector2 coords = cartToIso(j, i);
                    batch.draw(img,coords.x,coords.y);
                }else if(data[i][j] == 1){
                    Vector2 coords = cartToIso(j, i);
                    batch.draw(img2,coords.x,coords.y);
                }
            }
        }
        
    }
    /**
     * @param clickX mouse position x
     * @param clickY mouse position y
     */
    public void getTileAt(int clickX, int clickY){
        Vector2 coords = isoToCart(clickX,clickY);
        System.out.println("Isometric coordinates: "+coords.x+","+coords.y);
        
        //get actual tile for reference
        Vector2 tileCoords = cartToIso(coords);
        int centerX = (int)tileCoords.x + (TILE_WIDTH/2);
        int centerY = (int)tileCoords.y + (TILE_HEIGHT/2);
        
        
        Point p = new Point(clickX,Gdx.graphics.getHeight()-clickY);
        Point p1 = new Point(centerX,centerY+TILE_HEIGHT_HALF);
        Point p2 = new Point(centerX-TILE_WIDTH_HALF,centerY);
        Point p3 = new Point(centerX,centerY-TILE_HEIGHT_HALF);
        Point p4 = new Point(centerX+TILE_WIDTH_HALF,centerY);
        Polygon diamond = new Polygon();
        diamond.addPoint(p1.x,p1.y);
        diamond.addPoint(p2.x,p2.y);
        diamond.addPoint(p3.x,p3.y);
        diamond.addPoint(p4.x,p4.y);
        
        
       System.out.println("Point at: " + p1.x+","+p1.y);
       System.out.println("Point at: " + p2.x+","+p2.y);
       System.out.println("Point at: " + p3.x+","+p3.y);
       System.out.println("Point at: " + p4.x+","+p4.y);
       
       System.out.println(diamond.contains(p));
       if(!diamond.contains(p)){
            if(clickX>centerX){
                    if(clickY>centerY){
                        coords.y--;
                    }else if(clickY < centerY){
                        coords.x++;
                    }
                }else if(clickX < centerX){
                     if(clickY>centerY){
                        coords.x--;
                    }else if(clickY < centerY){
                        coords.y++;
                    }
                }
           

           System.out.println("Correction made!");
           System.out.println("Isometric coordinates: "+coords.x+","+coords.y);
       }
       
       System.out.println("Click: "+clickX+","+(Gdx.graphics.getHeight()-clickY));
 

        if(data[(int)coords.y][(int)coords.x] == 0){
            data[(int)coords.y][(int)coords.x] = 1;
        }else{
            data[(int)coords.y][(int)coords.x] = 0;
        }
    }

    
    

    public int getMAP_WIDTH() {
        return MAP_WIDTH;
    }

    public void setMAP_WIDTH(int MAP_WIDTH) {
        this.MAP_WIDTH = MAP_WIDTH;
    }

    public int getMAP_HEIGHT() {
        return MAP_HEIGHT;
    }

    public void setMAP_HEIGHT(int MAP_HEIGHT) {
        this.MAP_HEIGHT = MAP_HEIGHT;
    } 
}

You need 2 things:

• Convert your mouse (screen) position into cartesian map position
• Convert the cartesian position into the iso position and get the tile

If your map starts at 0,0 screen position, you can use your cartToIso giving the mouse position (screen position), which is equal to the cartesian map position, and you should get the related tile position.

If your map has scrolling you need to add the relative offset to the mouse position in order to trasform it to the cartesian map position.

Have you considered a mouse map?

Mouse maps are the easy way to do hexagon or isometric (or any shape) mouse picking without any complex math.

You basically divide the screen into rectangles (offset by the camera), figure out what rectangle the mouse is in for your tile x,y, then index into an image like the above one (kept on the CPU side of things, not stored in the videocard, because it's not actually drawn).

You get the pixel color, and then make a slight adjustment:(based on how your map staggering is laid out)

if(color == blue) TileY -= 1;
if(color == red) TileY -= 1, TileX += 1;
if(color == green) TileY += 1;
if(color == yellow) TileY += 1, TileX += 1;
if(color == black) do nothing;

...snip...

I have never heard of this! Very clever!

I first learned about mouse maps from an article here at GameDev.net many years ago, but have successfully used them in small projects.

The first first article (1999), and the idea expanded by someone here for use with tile elevations (2003). I should've linked to them in the original post.

Have you considered a mouse map?

Mouse maps are the easy way to do hexagon or isometric (or any shape) mouse picking without any complex math.

You basically divide the screen into rectangles (offset by the camera), figure out what rectangle the mouse is in for your tile x,y, then index into an image like the above one (kept on the CPU side of things, not stored in the videocard, because it's not actually drawn).

You get the pixel color, and then make a slight adjustment:(based on how your map staggering is laid out)

if(color == blue) TileY -= 1;
if(color == red) TileY -= 1, TileX += 1;
if(color == green) TileY += 1;
if(color == yellow) TileY += 1, TileX += 1;
if(color == black) do nothing;

I had honestly never heard of a mouse map up until this post but it seems like a really good solution. I'm just not sure how you're supposed to implement the actual mouse map. Do I create an image with the different colors? How do I check what color the click was registered on? The rest of it seems pretty straightforward.

At the moment what I was thinking was creating some sort of mouseMap.png or similar then getting where the mouse was clicked on the mousemap. All you'd have to do at that point is check the color. I'm just unsure of how exactly you would do that. Any suggestions?

Okay, so I tried to implement this solution and this is what I came up with.

    public void gta(int clickX, int clickY){
        Pixmap mouseMap = new Pixmap(Gdx.files.internal("mouseMap.png"));
        Color red = new Color(255,0,0,0);
        Color green = new Color(0,255,0,0);
        Color blue = new Color(0,84,166,0);
        Color yellow = new Color(255,242,0,0);
        Color black = new Color(0,0,0,0);

        Vector2 coords = isoToCart(clickX,clickY);
 
        int mouseMapX = clickX%TILE_WIDTH;
        int mouseMapY = clickY%TILE_HEIGHT;
        
        int val = mouseMap.getPixel(mouseMapX, mouseMapY);
        Color color = new Color(val);
        
        if(color.equals(red)){
            coords.y++;
        }else if(color.equals(green)){
            coords.x--;
        }else if(color.equals(blue)){
            coords.x++;
        }else if(color.equals(yellow)){
            coords.y--;
        }
         
        if(data[(int)coords.y][(int)coords.x] == 0){
            data[(int)coords.y][(int)coords.x] = 1;
        }else{
            data[(int)coords.y][(int)coords.x] = 0;
        }
    }

It still seems to be quite a bit off. I'm not sure what's going on with this code. I'll keep at it but I'm going to leave this here to see if anyone has any ideas.

Unrelated issue: Typically you shouldn't load images every frame, you should load images once and then store them someplace where you can repeatedly use them.

I'm just not sure how you're supposed to implement the actual mouse map. Do I create an image with the different colors?

Yes. Make sure it lines up to the shape and resolution of your isometric tiles (some people use different isometric angles and measurements).

How do I check what color the click was registered on?

PosInTile = (PosInWorld / SizeOfMouseMap)

The mousemap size should be the size of the regular flat isometric tiles.

Don't forget that you're looking at the position in the world, and not necessarily the position of the mouse on the screen. You have to compensate for the camera, if you have a world that scrolls.

All you'd have to do at that point is check the color. I'm just unsure of how exactly you would do that.

That depends entirely on your API and language, though it looks like you already figured that out.

if(color.equals(red))
{
    coords.y++;
} else if(color.equals(green)){
    coords.x--;
} else if(color.equals(blue)){
    coords.x++;
} else if(color.equals(yellow)){
    coords.y--;
}

You aren't just going up or down and left or right, you are going diagonally. Your ++ and -- to the coords should be affecting both X and Y in two of those cases, depending on the layout of your isometric map.

My guess is:

'red' is y++
'green' is y--
'blue' is x++ AND y++
'yellow' is x++ AND y--

But you have to figure that out since I don't know your map layout.

Also, I don't know what your 'isoToCart()' function is doing. It looks like you are trying to mix your old solution with the new solution, and the two conflicting solutions are stepping on each other and giving you extra wrong results.

I first learned about mouse maps from an article here at GameDev.net many years ago, but have successfully used them in small projects.
The first first article (1999), and the idea expanded by someone here for use with tile elevations (2003). I should've linked to them in the original post.

Thanks again! I have a deep love for tile-based games and am always looking for clever ways of doing it.

I just made hexagonal tile picking today. I didn't understand how to do the mouse map, so I just did the math and it's working great. First figure out which tile the cursor would be in, ignoring the corners (the light area in the attached image). It should be mostly accurate at this point, with the exception of the corners, so make sure this is working right before messing with the corners.

Then I check if it's to the left of the left line, or to the right of the right line (or just left or right side of the isometric tile). If it's on one of those sides, I check if it's above or below the middle line, then you just have to check it against the angle of that line to see if it's in one of the corners.

With a hexagon i know the angle of that line is 30 degrees off vertical, so I was able to just check the lines against the 2:1 ratio of a 30 degree angle. You can do the same thing with an isometric tile, just figuring out the angle and what the ratio should be. Then if it's over or under that ratio, you know if it should be in your tile or in the next one.

*Edit, oops, I had the wrong dimensions for the light area of the isometric tile. Fixed it