HTTPRequest with redirect

JBurger · Post by **JBurger** » Tue Mar 09, 2004 3:21 pm

Maybe someone can help be figure this out, all I'm trying to do is save a status page that I receive after I login.

The web page is here
http://64.174.67.108/AquaNotes/index.as ... tankID%3D1

I'm using the following code. But every time it runs, I save the login page, not the page the login redirects you to.

Let>PostData = txtUserName=guest&txtPassword=guest&cmdLogin=+Login+
HTTPRequest>http://http://64.174.67.108/AquaNotes/i ... MLResponse

Here are the headers I captured showing what the browser does when you enter and submit this info in manually.

** Log Reset ***

+++GET 35+++
POST /AquaNotes/index.asp?comebackto=/AquaNotes/status.asp?tankID%3D1 HTTP/1.1
Host: 64.174.67.108
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.6) Gecko/20040206 Firefox/0.8
Accept: application/x-shockwave-flash,text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,video/x-mng,image/png,image/jpeg,image/gif;q=0.2,*/*;q=0.1
Accept-Language: en-us,en;q=0.5
Accept-Encoding: gzip, deflate
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
Keep-Alive: 300
Referer: http://64.174.67.108/AquaNotes/index.as ... tankID%3D1
Cookie: ASPSESSIONIDQCSDDCQB=FHNFPNGCGLDFGEOIMFFDMBCO
Content-Type: application/x-www-form-urlencoded
Content-Length: 52
Connection: keep-alive
Posting 52 bytes...
txtUserName=guest
txtPassword=guest
cmdLogin=+Login+
Continue ignored...

+++RESP 35+++
HTTP/1.1 302 Object moved
Server: Microsoft-IIS/5.0
Date: Tue, 09 Mar 2004 14:53:24 GMT
Location: /AquaNotes/status.asp?tankID=1
Content-Length: 121
Content-Type: text/html
+++CLOSE 35+++

+++GET 36+++
GET /AquaNotes/status.asp?tankID=1 HTTP/1.1
Host: 64.174.67.108
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.6) Gecko/20040206 Firefox/0.8
Accept: application/x-shockwave-flash,text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,video/x-mng,image/png,image/jpeg,image/gif;q=0.2,*/*;q=0.1
Accept-Language: en-us,en;q=0.5
Accept-Encoding: gzip, deflate
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
Keep-Alive: 300
Referer: http://64.174.67.108/AquaNotes/status.asp?tankID=1
Cookie: ASPSESSIONIDQCSDDCQB=FHNFPNGCGLDFGEOIMFFDMBCO
Connection: keep-alive

+++RESP 36+++
HTTP/1.1 200 OK
Server: Microsoft-IIS/5.0
Date: Tue, 09 Mar 2004 14:53:24 GMT
Content-Type: text/html
Transfer-Encoding: chunked

+++CLOSE 36+++

Any idea how to get this to work?

Thanks much!
-Joe

Bob Hansen · Post by **Bob Hansen** » Wed Mar 10, 2004 12:07 am

Does removing the extra http:// solve the problem

HTTPRequest>http://http://64.174.67.108/AquaNotes/index.as ... MLResponse

JBurger · Post by **JBurger** » Wed Mar 10, 2004 12:16 am

No. That's me being a bonehead with cut and past. Sorry about that. The real code doesn't have the double http.

-Joe

Bob Hansen · Post by **Bob Hansen** » Wed Mar 10, 2004 12:39 am

Too bad, now I gotta work.....hmmm what can we see now?

1. The Let>PostData line has spaces around the "=". This is no good.
Let>PostData = txtUserName=guest&txtPassword=guest&cmdLogin=+Login+

2. The HTTPRequest line may need %PostData% vs. PostData
HTTPRequest>http://64.174.67.108/AquaNotes/index.as ... l.txt,POST,PostData,HTMLResponse

3. Does the folder exist for the local filename? (It may be created if not, I don't remember right now, so thought I would suggest it).
HTTPRequest>http://64.174.67.108/AquaNotes/index.as ... tankID%3D1,C:\macros\savedhtml.txt,POST,PostData,HTMLResponse

4. I don't think you want to include anything for the Login button, The only fields for data entry are txtUserName and txtPassword. Stop the PostData after entering the password.
Let>PostData = txtUserName=guest&txtPassword=guest&cmdLogin=+Login+

I suspect that the #1 and #4 issues may be the cause. Also be sure to remove all trailing spaces in the script.

JBurger · Post by **JBurger** » Wed Mar 10, 2004 12:54 am

#1 I tried it both ways, this is the last one I tried. Both showed no difference.

#4 was added as well after, not having it in didn't work. With and without it produced the same results.

I'll try #3 and see if it changes anything.

Bob Hansen · Post by **Bob Hansen** » Wed Mar 10, 2004 1:30 am

1. I finally ran the code with my suggestions 1,2,4 and got bad result.

2. I ran in Editor, single step with Show Watch List.. AHA! Got ERROR message that could not save to c:\macros\....... (I guess that answers that question for me. The destinaion folder will NOT be created.

3. I created macros folder and was able to save the results. Opened results and it was a login failure page.

4. Went to the page and logged in manually. Got a prompt to store password, which would have to be planned for if this was manual, but don't think this would affect HTTP Request.. Past that box and I got an error page with the following message:

Microsoft VBScript runtime error '800a000d'

Type mismatch: 'currentStatus'

?, line 0

This would make me think that the MacroScheduler code is probably good, since I cannot log in manually.

JBurger · Post by **JBurger** » Wed Mar 10, 2004 1:43 am

You should be able to log in. If you are blocking cookies it will error out. Other than that it should work manually.

Bob Hansen · Post by **Bob Hansen** » Wed Mar 10, 2004 2:29 am

I did just get into the page now. It looks like this page is doing a periodic refresh. I wonder if that affected me getting in (only tried it once earlier). Could this refreshing have anything to do with returning the file?

I also looked at the source code on the login and the "table" page. I am wondering if this is not coming back because it is an "asp" vs. an "htm" page?

Also notice that the login page has a line to return back to itself:

comebackto=/AquaNotes/status.asp?tankID%3D1

I am just rambling right now with no good ideas. The ones I mention above are the meaningless mots of madness. I can usually do better than that.

I am out of good ideas at the moment. Will probably need to wait for someone else to solve this for you, or pehaps one of us will have a brainstorm tonight.

JBurger · Post by **JBurger** » Wed Mar 10, 2004 2:25 pm

I didn't write the actual ASP app. I think what it does is use the combo of a cookie and the password. You submit credentials to the same page as the login. If it validates you get redirected to another page and the cookie lets you in. The refresh only occurs after you are on the status page.

I'm not sure if it's the fact that it's asp or cookies. But I had been trying for some time to get it to work before I posted here.

Lumumba · Post by **Lumumba** » Wed Mar 10, 2004 3:44 pm

That seems to be a session ID issue. Please check Curl,

it's a command line tool for transferring files with URL syntax, supporting FTP, FTPS, HTTP, HTTPS, GOPHER, TELNET, DICT, FILE and LDAP. Curl supports HTTPS certificates, HTTP POST, HTTP PUT, FTP uploading, kerberos, HTTP form based upload, proxies, cookies, user+password authentication, file transfer resume, http proxy tunneling and a busload of other useful tricks. Open source & freeware.

Platform: Windows/Linux/True64/Solaris/IRIX/HP-UX ...

Curl - the Webpage

Guest · Post by **Guest** » Wed Mar 10, 2004 6:46 pm

CURL worked. That's so much for that link.

It seemed to be the sessionid cookie. After I set up curl to work with cookies, the correct page was able to be saved to a file.

Thanks again.

Bob Hansen · Post by **Bob Hansen** » Wed Mar 10, 2004 7:17 pm

I just downloaded for trial.

What did you use for cURL syntax and command line?

Guest · Post by **Guest** » Wed Mar 10, 2004 7:26 pm

syntax

curl -o htmlfile.txt -d "txtUserName=guest&txtPassword=guest&cmdLogin=+Login+" -b cookies.txt -c cookies.txt http://64.174.67.108/AquaNotes/index.as ... p?tankID=1 -o htmlfile2.txt -b cookies.txt -c cookies.txt http://64.174.67.108/AquaNotes/status.asp?tankID=1

JBurger · Post by **JBurger** » Wed Mar 10, 2004 7:28 pm

Sorry I posted as guest. The board keeps losing my logon