Advertisement

HELP, working with HUGE image files!

Started by May 15, 2006 05:18 PM
14 comments, last by JakeM 18 years, 9 months ago
Quote:
Original post by JBourrie
Make sure you have lots of hard drive space open when working with Photoshop.


Sometimes if your computer crashes when you are using Photoshop it'll throw a huge temp file onto your hard drive that will eat up space. I wouldn't know where to search for them on a Windows machine, but I recall that I used to completely fill up my hard drive (at that time 10GB of free space) after 3 or 4 crashes. Following what Joe says, try looking for those if you find that your HD is unexpectedly filled up.

Scott
Newfound Room -- Open your mind to open content.
Quote:
I wouldn't know where to search for them on a Windows machine, but I recall that I used to completely fill up my hard drive (at that time 10GB of free space) after 3 or 4 crashes.


Back in the NT days it was pagefile.sys, but I don't know if that exists anymore.

Check out my new game Smash and Dash at:

http://www.smashanddashgame.com/

Advertisement
Quote:
Original post by entropy_metric

My current estimates are my smallest files are 15 megs for a compressed JPEG file. Really complex graphs can go as high as 50 megs.

My current research involves graphing social networks from data mined data.


The resolutions for these jpegs are insanley huge.


My current CPU is a dual core 3.2 ghz. Ram is 512 megs.


I'm starting to think I might have to code custom software for viewing these files. I need to be able to anaylze these social networks in totality so splitting them into chunks is not a very good option. Anymore ideas?


To repeat the very first reply made in this thread (my own), stating the size of a JPEG is useless to anyone involved in image manipulation. Understand that JPEG is a lossy, compressed file format that can be scaled by a huge degree. 15MB could mean a near lossless JPEG of a high resolution digital camera photo (still easily manipulated in Photoshop on even a low end system like yours) or it could mean a very lossy 1 million x 1 million pixel image.

There is only one piece of remotely useful advice that can be provided based on scientic terms like "insanley huge" is that your computer is badly designed: 512MB of memory is entry level - people using Photoshop or any other media manipulation program typically start at 1GB, I use 2GB. You are also stuck with a Pentium D, which attempts to squeeze two processor cores through a single archaic front side bus to the north bridge. When dealing with image data of even trivial size the CPU cache is rendered useless, meaning all image data is getting streamed through that tiny bus, making an already slow, badly designed CPU run even slower because it's spending all its time waiting on memory fetches.

In order to help you better use your images, make a piece of custom software, or better format the images in the first place, you need to provide some quantitative information:

How big are the images, as measured in pixels? For example an image you might use for wallpaper on your desktop might be 1280x1024 (where the image is 1280 pixels wide, and 1024 pixels high). In Photoshop use Image->Image Size and see the width and height measured in pixels (it will also show the image size in "M", which is for how many million pixels there are)

What kind of images are they? Photographs? 3D renders? 2D renders using a small color palette (not many different colors, like a bar graph)? 2D renders with lots of gradiants?

Why are they being saved as JPEGs? What program is producing them?
Q. How big are the images, as measured in pixels?

A. depends on the data being graphed, its different each time. But the smallest one generated is 24437 x 26937.


Q. What kind of images are they?

A. The images are being produced by the program graphviz. Their simple 2d images. With different colors.

www.graphviz.org

Q. Why are they being saved as JPEGs?

A. Because graphviz only outputs in .jpeg only.


Seems like the only thing I can really do to improve performance is get more ram. Couple of questions though. Would there be a better format to view my images in? If I decide to code a custom program, what exactly can I do in my program that will make viewing these images faster. I heard earlier someone suggesting a chunking example.

Here is an example of the image loaded in the gimp.

graph

graph

graph




I use IrfanView as my image viewer. I've had to work with very large (20+ megs) JP2000 and SID files, and it handles them fine (takes a few seconds to load). It can also do very basic editing.
-------------www.robg3d.com
I think what may be killing your computer are the dimensions of the image itself.
I don't believe the actual file size has anything to do with it.

Think about what a paint program has to do to display such an image. It probably has
to create a window the size of your image and draw it. If this was a 'dumb' paint
program, then it would probably try to create a 24-bit or 32-bit device context, so
the memory alone to create such a window would be 24437 x 26937 x 3 = 1.8 GB or
2.4 GB if 32-bit. And if the paint program has some kind of undo system, it may
try to create another copy of the image, so that could potentially double the
memory usage.

This topic is closed to new replies.

Advertisement