Using Procmon and Wireshark to solve website access problems

Two tools I couldn't easily do my job without are Procmon from Sysinternals (written by Mark Russinovich, a Microsoft Fellow and published author) and Wireshark.  Indeed, when I worked for a local IT support company some years ago Procmon was often my go-to tool.  As a network engineer Wireshark's also indispensable.  In this blog post I'll discuss how you can use these two tools to solve problems accessing websites that host API endpoints called from locally installed applications.

Firsty, a bit about these tools.  I won't include screenshots at this stage as they follow as part of the example later.

Procmon

Procmon, or process monitor to use its full name, shows you every system call performed by every process running on a Windows system.  It's useful for identifying what files an application touches and whether that access was successful.  Network access is also shown in the tool.

When Windows Vista came out, and introduced the world to UAC, it was necessary to adjust permissions in order to allow old software to work.  Developers had often assumed excessive permissions (as I mention in this post) which UAC blocked.  In a former job I'd directed one colleague to Procmon so often that he'd come to me asking for help and pretty much answer his own question with "Procmon" - he was usually right.

Wireshark

Knowing what packets are crossing your network card can be really useful for troubleshooting and Wireshark is a network protocol analyser.  A protocol analyser shows you the packets and explains their content so you can see what's really being sent.  It's a bit more complicated than that, and I'm not really qualified to explain exactly how it works, so if you're interested in its inner workings please have a look at its webpage (linked in the intro).

The scenario

A colleague in our software support team, we'll call him "David", was having problems with a payment application on one of his servers.  The application was unable to access the Internet and was giving the following error when he triggered the test function:

HTTP 407: Proxy authentication error.

I'll be honest, the cause of the problem was pretty clear cut - the application was trying to use our web proxy (which incidentally is being removed and everything's being migrated off of it) and it wasn't permitted to do so.  David checked the application configuration and confirmed it was not configured to use the proxy (although it clearly was) so sought the vendor's advice.  Maybe it was using the Internet Explorer proxy settings instead?  The vendor's response:

No, it doesn't use the Internet Explorer proxy settings.

Things got a bit heated at this point (I'd also misunderstood an email from David as finger pointing, so was quite annoyed) as the error is clearly showing the application is trying to use the proxy, and the vendor's having none of it.  At this point I pulled out the aforementioned tools.

First, prove the proxy is in use

Clearly it's important to verify the facts, despite the fact I trusted the error being presented.  First I install Wireshark on the application server and run it.  Meanwhile David instructs the application to connect to the payment gateway.  By adding a filter (the green box at the top of the screenshot) I've set Wireshark to only show me traffic involving a particular IP.  For example, to only see traffic to 172.16.1.2 I could set a filter of ip.addr == 172.16.1.2.

I've anonymised the screenshot to hide our internal IPs, but IP addresses starting 172 are our proxy.  For clarity I've annotated the screenshot to highlight the error the application was receiving - you'll recognise part of it from the original error message.  As the user isn't authenticated the proxy returns with HTTP 407 Proxy Authentication Required.

We've proven the proxy is in use, but maybe that's something else, rather than the payment application.  It's unlikely, given the address on the CONNECT line was the payment gateway's, but for due diligence we need to confirm it's the payment application making that connection.  Enter Procmon.

Identify the process with Procmon

Given Procmon logs every system call and every action of every process, for best results you only want it to capture what's happening from just before you start your test until just after.  Even then you could be looking at millions of lines in the log.  Once you've got some logs to work with (so here David had to attempt a connection again) you can filter them.  I filtered the operation to those starting "TCP" (including TCP connect, send and receive) and searched for the proxy's IP.  Again, pardon the anonymised screenshot but it's fairly clear to see what's happening:

Looking at the path column we've got proof the application (identified in the process name column) did indeed interact with the proxy.  For example:

TPayments.servers.local:52845 -> 172.16.1.2:8881

But the application's not configured to use the proxy, right?

Check the application config

The Payment Management Server.exe application has a config file (Payment Management Server.exe.config) which is just XML.  A quick read of it confirmed what David and the supplier had said - it's not configured to use the proxy.  There's no value for proxy address:

<add key="Proxy Address" value="" />
<add key="Bypass Proxy On Local" value="true" />
<add key="Web Proxy User Authentication" value="" />

The configuration certainly didn't say to use the proxy.  To rule out the machine account's proxy configuration I checked netsh winhttp show proxy which returned Direct access (no proxy server).  So we're back, again, to Internet Explorer's (IE) proxy settings - the only proxy settings left.

Reviewing IE's proxy settings

The Payment Management Server.exe service runs with its own account, which at some point had been logged in to Windows interactively (giving the user a desktop).  Logging in interactively also meant the account received proxy settings from Group Policy.  Searching the registry for the proxy's IP showed nothing, but searching for the proxy's DNS name found settings for the user in IE's settings.  (It was quicker, and more thorough, to search the registry for the proxy details, hence taking this route.)

Just to prove the settings were for the service's account I ran a check against Active Directory to compare the SID (the obscured number that starts S-1-5-21-3 at the bottom of the screenshot, above) and voila!  It was the correct account.  Removing the IE proxy settings lead to success.

I then told the supplier he needed to "update his documentation", as the IE settings clearly were used.  The response?  "Excellent!  Well done".  Not what I'd expect a supplier's support team to come back with, but then I wouldn't expect to have to do their troubleshooting for them either (why else do we pay them?!).  Sadly this happens all too often and a month or so later I was having to do exactly the same thing for another supplier.

Hopefully this post gives you a rough idea of how you can use some free tools to check out what's going on.

Banner image an excerpt from the Procmon logs.