We deployed some new features to a web application and I got word that some were not working. Its a new application so each problem has a pretty steep learning curve for me. The QA team said that no error messages were shown and it worked on some computers but not others. This initially sounded like a browser problem so I was not involved from the start since my focus is on the servers.
The dev team found using fiddler is that there was an HTTP error of 500. This was happening when trying to call a validation service (.svc) file. To get more out of this message we had to turn on debugging in the web.config file. Then with fiddler I was able to inspect the inner exception message which was the title of this blog.
There are two IIS7 servers that are network load balanced. I had no idea why IIS wouldn’t allow this simple HTTP call to a web service. Some of the first things we did were:
Make sure .NET 3.5 was installed [check]
Make sure .NET 4.0 was installed [check]
Check the application pool identities and settings [check]
recycle app pools [check]
reboot server [check]
None of these worked. These settings were the same on both servers but I decided to do some testing on each server individually. This narrowed down the problem to the second server. All of the files in the web app are replicated using a mesh topology. But the Temporary files in the .NET folders are not so I checked security on those next. That also was not a problem.
I turned on failed request tracing which is a really cool feature in IIS7. It spit out some .xml log files for the 500 error I was getting and showed me that ISAPI was at fault. However, I have done these traces before and had the same result. I think ISAPI is at a higher level than I needed to see to get a good error.
Searching of this 405 error reported some people having trouble with handler mappings. But that didn’t make sense for me because those are stored in the web.config file for each site, and that was the same on the working server and broken server. I started then looking at server level IIS7 settings. The ISAPI restrictions were the same so that wasn’t it.
At this point I decided to go nuclear and uninstall .NET 4.0, IIS7, application server role and .NET 3.5. In 2008 R2 .NET 3.5 is a feature that you can choose to install. I noticed when trying to uninstall that, there are actually some sub features to .NET 3.5. As it turns out, WCF Activation is a very important feature to have installed on a server that uses HTTP service calls.
Once I saw this I re-installed everything that I blew away and sure enough, the code worked just fine. What really gets me is that in all the googling I did there wasn’t a link between 405 and WCF Activiation. Hopefully now there is :]