Microsoft Jungle: ASP.NET Process Model and Request Processing Architecture

ASP.NET 2.0 Explained

Topics Covered

ASP.NET overview
Overview of IIS 6.0
ASP.NET process model
ASP.NET Pipeline and ASP.NET internals
Brief overview of IIS 7 and the changes in ASP.NET pipeline

What is ASP.NET

ASP.NET is a Microsoft .NET platform that provides services, a programming model and infrastructure to develop distributed applications on top of the HTTP protocol. ASP.NET framework takes advantage of HTTP commands and policies to set up client to server communication and cooperation.

HTTP is a stateless protocol, which means two successive requests across the same session have no knowledge of each other. It is important to understand ASP.NET HTTP runtime that extends the overall capabilities of the hosting web server.

ASP.NET has a process model which is the sequence of operations needed to process a request. ASP.NET is a request processing engine which takes a request and passes it through its internal pipeline to send the response back to the server. Although, ASP.NET runtime can be hosted in any application outside of IIS but we will be discussing it in the context of IIS.

IIS 6.0 provides four Internet services: the World Wide Web Publishing Service (WWW service) for hosting Internet and intranet content; the File Transfer Protocol (FTP) service for hosting sites where users can upload and download files; the Network News Transfer Protocol (NNTP) service for hosting discussion groups; and the Simple Mail Transfer Protocol (SMTP) service for sending and receiving e-mail messages. IIS hosts the ASP.NET runtime environment with ISAP extension dll (aspnet_isapi.dll)

ISAPI (Internet Server API) extensions are dlls that are called to handle a request from IIS. ISAPI consists of two components
1) Extensions and
2) Filters
Which are developed using ISAPI. Clients can access ISAPI extensions in the same way they access a static HTML page or an ISAPI extension can be called indirectly by mapping a file to a particular ISAPI extension in IIS. ISAPI filters are used to modify or enhance the functionality provided by IIS.ISAPI filters intercept specific events before the server itself handles them. ISAPI filters can be programmed to examine and modify request/response stream. ISAPI filter is used to modify custom authentication or to automatically redirect requests based on http headers sent by the client, however ISAPI filters cannot be run out of process. ISAPI filters are typically used in Encryption, Logging, Authentication and Data Compression.

ASP.NET provider model is an infrastructural feature that improves an application’s architecture by enabling developers and architects to operate under the hood of some system components. It provides extensive customization of the application’s run-time environment and code reusability.

Overview of IIS 6.0

Security

Secure by default – don’t install it unless you need it
Secure by design – if you do install it, install in a lock down mode, you still have binaries but don’t execute it unless you needed it
Secure in deployment- you only get the feature you need it.

Reliability

Fault tolerant architecture – one application can’t penalize others
Health monitoring – w3wp.exe has the capability to monitor web service which is recycling
Queuing - optimum use of threads

Scalability

scale-up/scale-out
kernel-mode caching
integrated application platform

System

XML based configuration
command line administration
remote administration

Figure 1 – Architecture of IIS 6.0 worker process isolation mode

Figure 1 is copied and pasted from http://technet2.microsoft.com/windowsserver/en/library/09324e32-e7d1-4ec7-8448-9ea3c45cb6a31033.mspx?mfr=true

We will briefly look at IIS 6.0 architecture because this will be helpful to understand the ASP.NET infrastructure and the request pipeline. Figure 1 shows the architecture of IIS 6.0 worker process isolation mode. IIS 6.0 worker process isolation mode is a fault tolerant architecture, what that means is web applications are separated by worker process boundaries that prevents one application from affecting another since every process has a protected memory. IIS 6.0 also provides application isolation with application pools.

An application pool is group of web applications in a worker process. Application pool provides the isolation with .NET Application Domain which is a lightweight process boundary without the cost of inter process communication and .NET runtime manages the memory of an App Domain thus providing the memory isolation. Multiple applications hosted in a worker process get the isolation boundary with each web application running in its own app domain.

Since an AppDomain belongs to a single process but you can host multiple app domains in a worker process, .NET provides a great solution to host multiple applications in an isolated domain without the overhead of running more than one process.

However, Application Domain is not a secure boundary since ASP.NET applications run with full trust by default. Applications running with full trust can execute native code and bypass .NET runtime security checks. Applications running with full trust can execute methods in other app domains and may even call private methods using .net reflection.

For security reasons, multiple applications may be isolated with multiple worker processes.

ASP.NET Process Model
When a client requests an ASP.NET resource, IIS handles the request and passes it to ASP.NET ISAPI extension for further processing. The request processing goes several pre defined steps which again depends on the version of IIS. The sequence of operations performed to process an ASP.NET request is called ASP.NET process model. Since the request processing depends on the version of IIS, we will look at different IIS process model and the request pipeline.

IIS 5.0 Process Model

Figure 2 below shows the IIS 5.0 process model. IIS 5.x is the default process model on Windows 2000/XP.

HTTP request first goes through TCPIP driver in kernel mode, which is part of the windows networking subsystem.
HTTP request is queued at inetinfo.exe in user mode which is listening on TCP port for incoming http requests.
Inetinfo.exe hosts the ISAPI extension dll
If a request is for an ASP.NET resource, request will be forwarded to aspnet_isapi.dll(asp.net isapi exntesion dll)
Inetinfo.exe will communicate the asp.net request to aspnet_wp.exe(worker process)
Inter process communication occurs over the named pipe
Once it goes to the worker process, it follows the ASP.NET request pipeline which will be discussed later after we look at IIS 6.0 process model.

It also supports multiple app domain in a worker process (aspnet_wp.exe) with each web application (IIS virtual directory) running in a single App Domain.

Figure 2 – IIS 5.0 Process Model

IIS 6.0 worker process isolation Mode

Figure 3 – IIS 6.0 worker process isolation mode architecture

Figure 3.1 - Web Administration Service initializes HTTP.SYS and manages the lifetime and the recycling of worker process

IIS 6.0 worker process isolation mode is the default process model on windows server 2003. Figure 3 above shows the IIS 6.0 worker process isolation mode architecture. Noticeable differences are the application pools, worker processes and http.sys driver. IIS 6.0 introduces http.sys kernel driver which sits on top of TCPIP driver and it allows incoming http requests to be queued at kernel level.

Figure 4 below shows the asp.net request pipeline.

Figure - 4 ASP.NET Request Pipeline in IIS 6.0 worker process isolation mode

We will review the IIS 6.0 request processing and queuing mechanism in IIS core components HTTP protocol stack (HTTP.sys) and a worker process. Please refer to Figure 3 for IIS6.0 worker process isolation mode process model.

HTTP protocol stack (HTTP.sys)
HTTP protocol stack is a HTTP listener implemented as a kernel mode device driver. HTTP.sys is an IIS 6.0 core component, which is part of the networking subsystem of the windows operating system.

HTTP.sys role in request processing

Every http request arrives at HTTP.sys
HTTP.sys determines if the request is valid. If the request is not valid, it sends a code for an invalid request back to the client
If the request is valid, HTTP.sys checks to see if the request is for static content (HTML) because static content can be served immediately.
If the request is for dynamic content, HTTP.sys checks to see if the response is cached in kernel mode.
If the response is in the cache, HTTP.sys returns the response immediately
If the response is not cached, HTTP.sys determines the correct request queue and places the request in that queue
If the queue has no worker processes assigned to it, HTTP.sys signals the WWW service to start one.
The worker process pulls the request from the queue and processes the request, evaluating the URL to determine the type of request (ASP, ISAPI or CGI).
The worker process sends the respond back to HTTP.sys.
HTTP.sys sends the response back to the client and logs the request if configured to do so.

IIS Pipeline

Figure 6 represents the key components of IIS 6.0, the left hand side of the diagram is IIS Pipeline. As shown in the diagram the first thing is ISAPI filter which allows you to do to certain things such as URL Mapping, Authenticate request, logging and above that we have certain states built in the pipeline including Authentication, Custom Errors, Logging.

1) When IIS receives a request, the request goes through the ISAPI filters and several states until a handler is determined.

2) Handler will examine the extension of the requested resource by looking up the script map. If it is a static content or if there is no mapping to a file extension it will be treated as a static content. Static content will be directly handled by IIS without using an ISAPI extension.

3) If the extension of the requested resource is mapped to an ISAPI extension, IIS handler will let ISAPI extension handles the request.

4) When .NET framework is installed on a server, it maps .aspx, .asmx and several other extensions to aspnet_isapi.dll. When a request for ASP.NET resource arrives, IIS handler will load the aspnet_isapi.dll and request is sent to the ISAPI extension through Extension Control Block data structure. aspx page will go through the asp.net request pipeline as shown in the right hand side of the diagram in figure 6.

ASP.NET Request Pipeline

Various steps are involved before ASP.NET runtime inside a worker process starts processing an incoming HTTP request. When a request is assigned to ASP.NET ISAPI extension dll, it goes through several managed objects which is called HTTP pipeline. Since the request flows through a deterministic path for processing, it is called the request processing pipeline. We will look at the HTTP runtime pipeline but before that lets look at what happens before request goes through HTTP pipeline.

IIS6.0 in worker process isolation mode allows to group web applications in application pools. As shown in the figure below any web site, web or virtual directory can be assigned to an application pool. Each asp.net is housed within its own managed application domain. IIS 6.0 supports up to 2,000 application pools in worker process isolation mode, where each application pool can be configured separately.

HTTP request and AppDomain internals

AppDomain provides each application a runtime environment with its own memory isolation without the cost of a separate windows process. AppDomain is like a process because it acts as the container for code and data. .NET runtime uses an AppDomain to isolate code inside of a secure boundary similar to protected memory space used by a process with much less overhead than a process. An AppDomain belongs to a single process but a process can host multiple domains.

Figure 5 – Worker Process and App Domain

Let’s look at AppDomain creation and role of the ASP.NET core objects in processing a request.
When server receives the very first request for an ASP.NET resource, System.Web.Hosting.AppManagerAppDomainFactory class creates application domains for a Web-application manager and this is also the entry point into .NET runtime. ApplicationManager class initializes, activates and manages the lifetime of ASP.NET applications. Figure 10 below shows the class and the method from .NET reflector (you can download Lutz Roeder’s .NET Reflector tool from http://www.aisto.com/roeder/dotnet/ ). First it creates the undocumented ISAPIApplicationHost object which uses other undocumented class and unmanaged COM interfaces to set up the communication from ISAPI extension to ASP.NET. One other class used to create ISAPIApplicationHost object worth mentioning is ProcessHost, it implements IProcessHost, IAdphManager, IPphManager interfaces exposed via COM to interact with ISAPI extension. These interfaces exposes methods via unmanaged COM to start application, start app domain protocol listener channel, set up pipeline runtime and start process protocol listener channel.

Figure 7 Application domain creation in ASP.NET

AppDomain is created immediately after ISAPIApplicationHost object and returns an instance of ISAPIRuntime object. ISAPIRuntime class exposes the IISAPIRuntime interface through COM to process a request. IISAPIRuntime interface implements ProcessRequest method which has the ECB (Extension Control Block) structure as [in] parameter. ASP.NET receives reference to ECB structure to retrieve the current request.

When a worker process is started, the required authentication level for COM and DCOM security is passed as command line parameters. COM and DCOM security is important to .NET framework because the interface to the CLR is a COM object. The worker process, therefore loads the CLR up as it were a COM object.

When an ASPX request hits IIS, the web server assigns a token based on the authentication model – anonymous, windows, basic or digest. This token is passed along to the worker process. The request is picked up by a thread within the worker process. This thread inherits the identity token from the IIS thread that originally picked the incoming request up.

ASP.NET internals and the HTTP pipeline

Once an AppDomain is created successfully, it calls ISAPIRuntime.ProcessRequest() is called, code snippets from .NET Reflector is shown below.

Figure 8

ISAPIRuntime.ProcessRequest() method receives a reference to ISAPI ECB data structure which is passed to ISAPIWorkerRequest.CreateWorkerRequest() to create an instance of HttpWorkerRequest class the current request. HttpWorkerRequest contains all the information about the current request including the request URL, the headers and so on. Once the HttpWorkerRequest object is created, it is passed into the static ProcessRequest method of the HttpRuntime class which is executed in the AppDomain of the web application.

HttpWorkerRequest wr = ISAPIWorkerRequest.CreateWorkerRequest(ecb, iWRType);

HTTP Pipeline

HTTP pipeline is basically handling the HTTP request over a chain of managed objects. Figure 9 shows the UML communication diagram for HTTP pipeline.

HTTP pipeline is activated with HttpWorkerRequest object containing all the information about a current request
HttpRuntime.ProcessRequest is called from ISAPIRuntime.ProcessRequest which initializes a number of internal ASP.NET core objects for request processing.
HttpRuntime creates an instance of HttpContext class, which represents the context for the current request and fills it up with any HTTP information specific to a request. ProcessRequest creates an instance of the HttpWriter class which is a TextWriter that actually buffers any text programmatically sent out by the code in the page. HttpContext object creates and provides access to other .Net core objects as shown in UML diagram (Figure – 9)
HttpRuntime class uses HttpApplicationFactory to either find or create an instance of HttpApplication class by calling the static GetApplicationInstance method. From the pipeline’s perspective, a virtual directory under IIS is an application. Before creating HttpApplication instance, HttpApplicationFactory ensures that ApplicationOnStart event is fired as shown in the code snippet below taken from .NET reflector

internal class HttpApplicationFactory
{
    private void EnsureAppStartCalled(HttpContext context)
    {
        if (!this._appOnStartCalled)
        {
           lock (this)
           {
               if (!this._appOnStartCalled)
               {

                    using (HttpContextWrapper wrapper = new HttpContextWrapper(context))
                    {
                      WebBaseEvent.RaiseSystemEvent(this, 0x3e9);
                      this.FireApplicationOnStart(context);
                    }
                    this._appOnStartCalled = true;
                }
            }
          }
      }
}

If the website (virtual directory) has a Global.asax file, ASP.NET creates an instance of the class defined in this file instead of an HttpApplication object. The Global.asax file is an optional file that contains event handlers at application level and session level events raised by ASP.NET or HTTP modules. At runtime, Global.asax is parsed and compiled into a dynamically generated class derived (if not already done so) from the HttpApplication base class.

An HttpApplication holds a collection of HTTP module objects (which implements IHttpModule interface). HTTP modules are filters that can examine and modify the contents of HTTP request and response messages as they pass through the pipeline. HTTP modules are similar to ISAPI filters in that they examine every incoming request, take actions based on request and examine the outbound response and take actions if required. HTTP modules provide pluggable architecture so that custom modules can be easily added or removed to or from an ASP.NET application by modifying the web.config file. You can write custom HTTP modules by implementing the IHttpModule interface with ease because HTTP module is written in managed code and is fully integrated with the lifecycle of an ASP.NET application. Typical use of custom http modules includes Custom Header, Custom Logging and Custom Authentication.

Figure 9 – Aggregation association UML Diagram for HttpContext object and other ASP.NET core objects

Once Http modules are created, HttpApplication class takes over the request processing and determines the appropriate HTTP handler for the current request. The current request is processed by calling either BeignProcessRequest or ProcessRequest depending on whether the http handler is Async or not. For example, .aspx page is handled by System.Web.UI.PageHandlerFactory. WebServiceHandlerFactory class creates an instance of an IHttpHandler implementation for handling Web service requests.
The HttpApplication object uses an HTTP handler factory to either find or create an HTTP handler object. HTTP handlers are endpoints for HTTP communication that process the requests and generate the responses. HTTP handler factories implement IHttpHandlerFactory interface and HTTP handlers implement the IHttpHandler interface. HttpApplicationFactory and HttpHandlerFactory classes pool HttpApplication objects and the HTTP handler objects for efficiency. An HttpApplication, its modules and its handler will only be used to process one request at a time.

Figure 10 - HTTP Pipeline Class Diagram to illustrate the realization of IHttpHandler, IHttpModule, IHttpHandlerFactory interfaces

Figure 11 HTTP Pipeline UML Communication Diagram

Once the handler is created, its ProcessRequest method is called with the current instance of HttpContext class which has access to Request, Response, HttpServerUtility and other request specific data. Once ProcessRequest method returns, the request is complete

Brief overview of IIS 7 and the ASP.NET integrated pipeline

IIS 7 builds on top of IIS 6 with the following features

Security

Customized, componentized type web services, iis 6.0 footprint is in the small set of modules (core component), if something related with digest authentication need to be updated with the patch resulting in updating the server because footprint was in a small set of core components. IIS 7 has a foot print in larger set of modules reducing the attack surface
Reduced management of patches
URLScan built in functionality– block certain type of requests so that it will not be processed in the pipeline

Extensibility

Brand new Win32 API to extend IIS functionality. It seems isapi is not really used by Microsoft. Sharing is caring, according to Microsoft the new extended api is the same as what used by Microsoft.
Integrated support for httpmodules (asp.net)

Configuration

Distributed delegate configuration so that developer doesn’t have to be the admin to change the configuration
Rich Extensibility
Integrated configuration for web platform, asp.net and iis configuration live together.

System Management

Improved Brand new IIS manager, such as specialized reports can be easily done as snap in
AppCmd.exe command line administrator
http & https remote administration

Diagnostics

Brand new state API, you can query what requests are running, state of the site, failed request tracing (includes also successful request tracing)
Easy to setup & use failed request tracing
IIS & ASP.NET integrated diagnostics

In IIS7, the ASP.NET request processing pipeline is integrated with IIS pipeline, essentially providing a wrapper over it instead of plugging into it.

A request arriving for any content type is processed by IIS, with both native IIS modules and ASP.NET modules being able to provide request processing in all stages.