Hi. I am working on a project that includes downloading data from webservers. There is a problem. Some webservers rejects the program request for data. In one example, the webserver responded with an error that the request was ignored because the client (program) is trying to access the data from another server. In terms of web browsers, this is like clicking on a link at www.google.com to download data from www.gamespot.com. To my understanding, the server is incorrect because the program is communicating directly to the server just as a web browser does. I would like to know how do web server such as the example know where the client originally found the link to the data? In the example above, is there any HTTP 1.1 code that I need to add to the request so that the server to process the request? Thanks, Kuphryn

1. Use HTTP 1.0 so you dont have to deal with chunking.
2. What is the header you are sending to the server?

Thanks.

The program can process incoming data. That is not a problem. One problem is with the server wanting to send the data.

I do not have access to the server code, which is on my home workstation. I can post it this evening.

Kuphryn

Okay. I have tried both solutions, but none worked. Here is an example of the request.




Again, the request above does not work. Here is one website I tested use to test the request.

http://visualparadox.com/wallpapers/coloroncall.htm
http://visualparadox.com/images/no-linking-allowed/coloroncall800.jpg

Kuphryn

That's not a well-formed GET request, and you are explicitly not accepting JPEG. You probably want something like this instead:


GET /images/no-linking-allowed/coloroncall800.jpg HTTP/1.1
Host: visualparadox.com
Referer: http://visualparadox.com/wallpapers/coloroncall.htm
Accept: text/plain, text/html, image/jpeg
Accept-Language: en-us
Accept-Encoding: gzip
Connection: Keep-Alive


I assume you are prepared to handle gzip encoded and chunked responses from the server, because that's what your HTTP/1.1 request implies.

EDIT: Forgot to add the jpeg MIME type.

[edited by - spock on January 14, 2003 11:00:27 AM]

I can certainly remove the server domain and add the jpg to Accept parameter. I will update this thread afterward.

What do you mean gzip? I thought all requests needs to include a form of encoding.

What do you mean chunks? All data transfered via Winsock are "chunks." For example, the server will send a 1mb file in chunks.

Kuphryn

kuphryn, you need to take a closer look at the HTTP 1.1 spec. The chunking is a transfer encoding and your client must support it in order to speak HTTP 1.1. For simple requests it's usually much easier to just do HTTP 1.0, like pawn69 suggests.

By including Accept-Encoding: gzip in your request, you specify that you want and is prepared to decode a gzip compressed response (this is called a content encoding).

If all you want to do is download that JPEG and feed the server a Referer: header, a request such as this should suffice:


GET /images/no-linking-allowed/coloroncall800.jpg HTTP/1.0
Host: visualparadox.com
Referer: http://visualparadox.com/wallpapers/coloroncall.htm


[edited by - spock on January 14, 2003 4:03:32 PM]

The client communicates to the servers via HTTP 1.1 without a problem.

What does chunk have to do with the server not allowing the client access to protected files?

Kuphryn

quote:

---

Original post by kuphryn
What does chunk have to do with the server not allowing the client access to protected files?

---

Absolutely nothing. It''s usually overkill to implement an HTTP 1.1 client just to make a few simple requests to download data, that''s all. If you make HTTP 1.1 requests you must be able to handle the things HTTP 1.1 servers will send you in response, and that includes chunked data.

Nothing is an overkill when you consider software design and implement as an ecstasy!

Kuphryn