Sunday, September 20, 2009

W3 Simple Proxy/Anonymizer with Custom User-Agent

A while ago I decided to hit the XHTML Validate link to see what the W3 XHTML Validator said was wrong with a web page. Of course, there were tons of things wrong (I don't know why people ever put the link on their page because they are never compliant with the w3 standards). Anyways, I noticed that it gives you the option to view the source code of the page it's supposed to be validating and thought you could use this as a proxy to view html web pages. Also, the w3 validator gives you the option of specifying the user-agent header that will be sent to the server, which could come in handy. They also seem to have a mechanism in place to keep you from inserting additional headers into the HTTP Request sent to the server, although the mechanisms for the uri param and the user-agent param are different.

Here's a sample url http://translate.google.com/translate?hl=en&sl=es&tl=en&u=http://validator.w3.org/check%3Furi%3Dhttp://gnarlysec.blogspot.com%26charset%3D(detect%2Bautomatically)%26doctype%3DInline%26ss%3D1%26group%3D0%26user-agent%3DW3C_Validator/1.654. The source at the bottom of the page would still need to be parsed out, but that's the basic idea. Also note the url param "user-agent".

1 comment: