Howdy,

So ive nearly finished my image libary for loading and converting textures, but im wonderingabout a more effecient way to get surface data.

These are the 2 storage methods that come to mind:

std::vector<std::vector<Byte> > mData;
Byte* mData;

Vector Pros:
-Can access any surface/mip easily:

Uint32 Image::GetSurfaceIndex(Uint32 face, Uint32 mipLevel)const
{
	return (face * (mFileInfo.mArraySize - 1)) + mipLevel;
}

-No manually memory clean up etc.

Cons:
-Cannot load Blob data in single call, must run loop such as:

for (size_t f = 0; f < mFileInfo.mArraySize; f++)
{
  Uint32 w = mFileInfo.mWidth;
  Uint32 h = mFileInfo.mHeight;

  for (size_t i = 0; i < mFileInfo.mMipCount; i++)
  {
    numBytes = GetSurfaceByteCount(w, h, mFileInfo.mSurfaceFormat);
    std::vector<Byte> cpyData = std::vector<Byte>(data, data + numBytes);
    mData[GetSurfaceIndex(f, i)] = cpyData;
  }
  if (src + numBytes > end)
  {
    Reset();
    return;
  }

  src += numBytes;

  w = SF::Max(w >> 1, 1);
  h = SF::Max(h >> 1, 1);
}

Byte array Pros:
-Loading is simple:

mData = new Byte[mFileInfo.mByteCount];
memcpy(mData, ptr, mFileInfo.mByteCount);

Cons:

-Getting ptr to start of a surface becomes inefficient, especilaly with mips as it requires a loop.

Byte* Image::GetPixelPtr(Uint32 face, Uint32 mipLevel, Uint32 x, Uint32 y)
{
	if (face > mFileInfo.mArraySize) { return nullptr; }
	if (mipLevel > mFileInfo.mMipCount) { return nullptr; }
  
  	int w  = mFileInfo.mWidth;
    int h = mFileInfo.mHeight;
  	int mipBytes = 0;
  	for(int i = 1; i < mipLevel, i++)
    {
      w = SF::Max(w >> 1,1 );
      h = SF::Max(h >> 1, 1);
      mipBytes += GetSurfacebytes(w, h, mFileInfo.mSurfaceFormat);
    }

	return &mData[GetSurfaceOffset(face) + mipBytes + ((y * w) + x) * mPixelByteSize];
}

Note that GetSurfaceOffset() calculates the offset to the start of a surface, includeing the offset of the previous surfaces mips.

Does anyone know of a better way to store surface arrays with there mips in a single blob thats effecient to access?

Thanks

I didn't dig into the details, but I'm wondering if you have a reason for not considering an RAII wrapper (e.g. std::vector) for the second option ('Byte* mData').

Actually, I guess I'm a little confused, because you have:

mData = new Byte[mFileInfo.mByteCount];

But also:

if (mData.size() == 0) { return nullptr; }

What is the type of mData? You mention no manual memory cleanup as a pro for the first approach, which suggests you have a raw array in mind for the second approach, but the use of 'mData.size()' seems to conflict with that.

Apologies if I'm misunderstanding or misreading your example code.

Whoops, sorry i fixed that, i mostly copy and pasted the vector version and re-wrote it on game dev for the examples. I have been messing with 2 versions one that is a raw memory blob, and one that uses a vector, i go both to a stage where they work fine, but a raw array would be prefferable if i can get rid of that mip offset loop.

Other issues aside, why not use std::vector for the second approach as well?

Just now, Zakwayda said:

Other issues aside, why not use std::vector for the second approach as well?

Do you mean pre-fill a vector with pointers into the array when i initialize the image? if so yes that would work. So ill go do that now ha.

13 minutes ago, Jman2 said:

Do you mean pre-fill a vector with pointers into the array when i initialize the image? if so yes that would work. So ill go do that now ha.

For the moment at least I'm only referring to how the raw memory is managed.

In your original post you mentioned no manual memory cleanup as an advantage of the first approach. Further, you have this:

mData = new Byte[mFileInfo.mByteCount];

Which suggests that what you have in mind for the second approach is a raw array requiring manual cleanup.

The only question I'm asking (at the moment at least) is why not use e.g. std::vector for the second approach as well. Everything else could be functionally the same, so it's not a matter of changing the overall approach. It's just that there doesn't seem to be anything about the second approach that would prevent you from using RAII.

Another way to put it is that the issue of memory management appears to be orthogonal to how the data is stored functionally, so I'm not sure why you wouldn't use RAII for both approaches.

Apologies if that's still unclear.

Ah right so its me not being clear, I have Texture data it can have n number of face and each face can have n number of mips.

First approach:
Was to use a std::vector<std::vector<Byte> > note that the vector contains a vector of bytes. This means each surface and each mip gets its own vector. So i can quickly jump into it using some arithmatic:

return (face * (mFileInfo.mArraySize - 1)) + mipLevel;

The problem with a vector of vectors, is i have to load up a raw blob then extract each surface and set each vector individually, its not that bad but its still a double loop that doesnt need to be.

Second Approach:

Just store a single Byte* blob; this means i can load from a binary file in one huge chunk, the problem is accessing individual surface or mip information quickly. I would need to skip a byteOffset into the array to get the starting pointer of the surface, this means doing some math to calculate the size of each surface and each surfaces mip chain to skip to get to the surface i need + the mipByte offset into the individual mip chain for that surface. That requires a loop to accumulate the bytes for the level which sucks.

Clarity Note:

All of this is contained in a class that does it all for you, it has release methods and cleans its self up regardless of the method used. The focous is mainly just on the effecient access to individual surfaces levels.

21 minutes ago, Jman2 said:

Second Approach:

Just store a single Byte* blob; this means i can load from a binary file in one huge chunk, the problem is accessing individual surface or mip information quickly.

The only point I'm trying to make is that you can use std::vector for the second approach as well - you don't have to use a raw array or clean up the memory manually yourself.

Ah right sorry, thats me misunderstanding.