I am sending a file from one socket to another, and there are a lot of packets that are now in transit and in the socket send buffer. What I want to do is to abort the file transfer immediately (once the user clicks "Abort", the file transfer is aborted without any delay).

If I closed the connection gracefully (by sending a FIN packet), then the FIN packet will only get processed by the other socket when the packets in transit and in the send buffer has been fully received by the other socket (which could take a long time).

I think that the only way to do this is to close the connection ungracefully (by sending an RST packet). Am I correct or is there another way?

Yes, that is what "gracefully" means. Another way could be to keep a separate connection with another socket over which you send control information ("file transfer started", "file transfer complete", "file transfer aborted" and so on).

Also, ungracefully closing a TCP connection is not inherently bad, it's only bad in that you may lose buffered data or data in transit that, from the application's perspective, has already been sent (unless the application implements its own acknowledgment protocol on top of TCP). If you're aborting a transfer, you presumably no longer care if the file arrives intact, so it's okay. Just make sure the remote can distinguish between a completed transfer and a prematurely aborted transfer, and make sure the socket is not used for anything else.

Another way could be to keep a separate connection with another socket over which you send control information ("file transfer started", "file transfer complete", "file transfer aborted" and so on).

Let's say I did that, now I am sending a file through the first socket, and after a while I decided to abort the file transfer, so I send the string "file transfer aborted" through the second socket and then this string is received by the other end, what should the other end do now?

Let's say I did that, now I am sending a file through the first socket, and after a while I decided to abort the file transfer, so I send the string "file transfer aborted" through the second socket and then this string is received by the other end, what should the other end do now?

The second end can then "gracefully" close his socket and let the OS clean up in the background while it moves on to other stuff since it now knows the transfer is aborted. That's if you really want to close it gracefully; like I said, if you need to tear down a connection NOW and don't care what happens to the data already committed to the TCP stream, then a hard shutdown is acceptable (to be fair, such situations are rare; when transferring large files it is common to transfer them block by block with, depending on your requirements, periodic status and integrity checking, which allows you to send control information over the same socket with minimal latency).

The second end can then "gracefully" close his socket

I assume you mean it should close the socket that is receiving the file.

When the socket wants to gracefully close the connection, it would send a FIN packet to the other end, now the other end will also send its own FIN packet, which like I said earlier will only get processed by the other socket when the packets in transit and in the send buffer has been fully received by the other socket, which doesn't solve the problem (unless I misinterpreted what you mean).

Note that I think I will go with the RST approach, however I'm curious to understand this other approach.

I'm not sure what level you're working at.

When you say "send a FIN packet" and "send a RST" packet, you talk in terms of the implementation.

If you control the implementation, then you also control what the user has buffered. Thus, you can throw away what the user has buffered, and only keep the bit that is currently "on the wire," which is the size of the remote accept window.

If you want the connection to appear to have terminated gracefully, at the TCP level, then that's what you have to do.

However, some higher-level application protocol will probably be confused. For example, if the protocol says "here comes file X" and then data starts streaming, and then data stops streaming -- should the remote end assume that it has all of X? If there is a length field in the protocol, the remote end will know it didn't receive all of it, so the "graceful" shutdown at the TCP level still ends up being a "forceful/unexpected" shutdown at the application level.

So, if the reason you want to "immediately" stop the transfer is that you changed your mind -- you don't want to send that 20 GB file over your cell phone data plan -- then the best thing to do is to remove all the data that's on the wire as well as all the data that the user has buffered, which means tearing down the connection state, which means sending a RST, and responding with RST if you hear anything more from the other end.

Now -- this is from the point of view of the implementation. When you're writing application software, you typically don't have control at that level, but instead use some particular socket API. In Berkeley sockets (most common,) compare the actions of shutdown() with the actions of close(). They may translate to particular TCP packets on the wire, but there is no guarantee -- there is only the specific defined semantics of those API calls that the application makes.

I'm not sure what level you're working at.

I am working at the application level, that is, I am using sockets and not controlling the TCP packets themselves (which can only be done in raw sockets I suppose).

When you say "send a FIN packet" and "send a RST" packet, you talk in terms of the implementation.

When I say "send a RST", this means that I want to close the connection ungracefully, which means that when I close the socket, the socket will remove all of its data (send buffer, receive buffer, etc.) and sends an RST packet to the other end telling it that it is no longer there.

But when I say "send a FIN packet", this means that I want to close the connection gracefully, which means that when I close the socket, all of the data in transit and in the send buffer will be sent to the other end (followed by a FIN packet), and all of the data from the other end in transit and in its send buffer will be sent (followed by a FIN packet also). Which as I have said earlier is a bad idea if I want to abort a file transfer immediately, because it could take a long time for the pending data to arrive before the connection is closed.

So, if the reason you want to "immediately" stop the transfer is that you changed your mind -- you don't want to send that 20 GB file over your cell phone data plan -- then the best thing to do is to remove all the data that's on the wire as well as all the data that the user has buffered, which means tearing down the connection state, which means sending a RST, and responding with RST if you hear anything more from the other end.

Yes, this is the only approach that makes sense.

BTW, I am working under Windows.

On Windows, calling closesocket() will be the fastest way for you to close the connection.

FWIW: to have your socket to send an RST on socket close - SO_LINGER is your friend.

Documentation for Winsock closesocket() is here:

https://msdn.microsoft.com/en-us/library/windows/desktop/ms737582(v=vs.85).aspx

Especially this bit:

If the l_onoff member of the linger structure is nonzero and l_linger member is zero, closesocket is not blocked even if queued data has not yet been sent or acknowledged. This is called a hard or abortive close,

Other TCP implementations may do different things -- read your OS man pages!