IE extensions: Manipulating any request or viewing any response

Summary

A common requirement for an IE extension (BHO or toolbar) is for it to be able to manipulate any request that is made by the browser.  Another common requirement is for it to be able to view any response that the browser recieves.  For example, perhaps you need to add a header to every request that is sent.  Or you wish to view httponly cookies.  Unfortunately, this is no easy task.

PassThroughAPP

A common mechanism that is in use and which has been discussed in the newsgroups is to implement your own Asynchronous Pluggable Protocol (APP) handler.  APPs are how URLMON actually retrieves data from a URL.  And the default URL moniker implementation is what IE uses to retrieve data.  So if you can implement your own APP handler, and replace the default handler for HTTP or HTTPS requests, then you would have full control over all of the requests IE makes.

Of course if you did this you would have to implement all of the logic that exists in the default handler.  Well, instead of re-implementing all of that code, the common approach is to create the default handler first, and then pass the requests through to the default handler after you're done reading or manipulating the data.  The mechanism by which you establish your handler as the default handler is using an API call:  IInternetSession::RegisterNameSpace.

There is a fair amount of work to implementing such a handler, but luckily (for those of you who wish to use this approach), a sample was created by Igor Tandetnik that shows how to do it, and it is fittingly called the PassThroughAPP.

Problems with PassThroughAPP

But there is a big problem when using the PassThroughAPP approach in an IE extension:  The namespace registration is not chained, and there is no "GetRegisteredNameSpace" call available, so it cannot be chained.  So only the last caller of RegisterNameSpace will be able to have their APP handler invoked.  So multiple extensions using this mechanism cannot work.

The V-Table Thunk Solution

Until an API is made available from MSFT, and it is made backwards compatible, we will have to use a different approach if we want to handle all requests & responses made in IE.  Another approach would be to use a V-Table thunk to replace the default implementation.  This could be done in a way that chains the procedure entries that are replaced to the old ones, thus multiple extensions could be installed and all be called.

It turns out that Google Gears has implemented this mechanism, and their code is open source, if you wish to see a working version.  Their implementation is a bit heavyweight because it replaces entire interfaces, but for most users only the Start and StartEx methods need to be replaced. 

Here is a pseudo-code description of the simplified V-Table Thunk solution I am using:

• Create the CLSID_HttpProtocol object directly from URLMON.DLL.
• QI for the IInternetProtocolRoot interface.
• Store the original address of the Start method index that can be found in the V-Table (Start is at index 3, right after QI, AddRef, Release). You will need this address to pass the calls through. Also store the address of that V-Table entry itself itself so the patch can be "unpatched" later.
• Replace the address of the Start method in the V-Table with your own method.
• Do the same for the StartEx method of the IInternetProtocolEx interface (actually StartEx is what is called on IE7+, not Start).
• At this point, your StartEx method will be called whenever a download is requested.  Note that your StartEx method is not a member function, but if you make it stdcall, then it will look just like the standard "StartEx" method, except the first argument will be pThis (which in this case is IInternetProtocolRoot).
• Implement your StartEx method to call the original method, and voila!  Your intercepter is in place and your version of StartEx will be called for every HTTP download (and you can manipulate it or pass it through as you desire).

Handling IHttpNegotiate Callbacks 

Just as in the PassThroughAPP case, you can get your own IHttpNegotiate interface called by creating your own COM object that implements IInternetProtocolSink, IServiceProvider, and IHttpNegotiate.  Use this object's IInternetProtocolSink interface when you call the original StartEx method, and this object will be called back for the IInternetProtocolSink methods and the default APP implementation will QueryService it for IHttpNegotiate. 

And away you go.

I hope this helps.