I've written a few network apps before but one of the problems I never had lots of time to solve was creating a network application that is able to send/recv as fast as the user's connection allows(i.e. 2 - 5 KB/s for dial-up, but 30 - 40 KB/s for broadband). Obviously this doesn't normally apply to games as you shoot for a steady stream of packet bandwidth and stick to it, but for things such as file transfer I would like to get the most out of a user's connection as possible. My theory was that since file transfer, atleast how I've done it, works on a "send a piece, acknowledgement, send another piece" basis; if the user had a faster connection then it would naturally scale up. However this doesn't seem to be the case. What would be a good way of making sure you were using a user's network connection as effeciently as possible?

If you require synchronous acknowledgement before sending the next packet, then you will be bound by the latency of the connection.

Look at TCP, which has a sliding window, where the receiving end can open the window with acks -- this allows you to overlap sending more data with acknowledgement. There is also sender-side management, look for the congestion avoidance algorithm and the silly windows avoidance mechanism. Also, fast-start vs slow-start would be applicable.

Or just use TCP, if the most bandwidth throughput is what you want (at the expense of latency in the face of packet drops).

I was planning to use TCP anyway since this is for a FTP application and FTP uses TCP anyway.

If you are streaming the file via TCP, what would be the best way to send it? Create an arbitrary send buffer(say, 1 - 2K), send the first stream of data, wait a few milliseconds(say, 20ms) and send the next piece, repeat till done while checking to make sure send is sending the entire byte size? That solution doesn't seem like it would scale based on connections very well.

The best way to send using TCP:

1) create two buffers for loading from disk, each of which is at least 64k (ideally bigger if you're going for LAN speeds).
2) fill them both with data
3) whenever a socket shows that it has more space available (say, by returning from select()), send whatever you can from the current buffer (a single call to send() will do this)
4) when the current buffer is empty, move to the next buffer, and put the empty buffer in a disk load queue

The kernel will do buffering of your sending, and will manage the send window and bandwidth for you. Just dump as much as you can into the socket.

That makes sense, but why the 2 buffers? Just to always have data ready to send instead of taking a couple of cycles to re-populate a single buffer?