You're going to want to interpolate between the 4 nearest heightmap positions. There are two ways to do this.
The first way is to just bilinear interpolate using the objects fractional XY position, interpolating (edit:) the heights of both p1--p2 and p3---p4 along the X axis (if p1-p4 are laid out in a 'Z' shape). Then interpolate along the Y axis using the object's fractional position between the top edge and bottom edge of the terrain grid square, interpolating the two height values resulting from the first X axis lerps.
The second way is better suited if your terrain grid squares are being rendered as a pair of triangles, which means you only want to interpolate between the 3 points that the entity is above, instead of all 4. All you have to do is determine which triangle's right-angle the entity is closest to, and then interpolate between the height of the points of that triangle.
If you use the first method, and you are drawing two triangles per heightmap square, you could get entities that interpolate above/below the ground a bit, but it's much simpler to do.