Fielding_apachecon2000

HTTP and Apache Roy T. Fielding eBuilt, Inc. The Apache Software Foundation fielding@apache.org Overview HTTP Basics Apache httpd architecture • • • • • • Connection management (listening) Request Reading Finding a representation Conditional request evaluation Response Writing Connection management (closing) Future Improvements waka/2.0 HTTP Basics Web architecture • Layered-client-server • Uniform “resource” interface – Simple, but powerful – Semantics are visible to intermediaries • HTTP extends interface across network HTTP Message Syntax GET /Test/hello.html HTTP/1.1\r\n Host: kiwi.ics.uci.edu:8080\r\n User-Agent: GET/7 libwww-perl/5.40\r\n \r\n HTTP/1.1 200 OK\r\n Date: Thu, 09 Mar 2000 15:40:09 GMT\r\n Server: Apache/1.3.12\r\n Content-Type: text/html\r\n Content-Language: en\r\n Transfer-Encoding: chunked\r\n Etag: “a797cd-465af”\r\n Cache-control: max-age=3600\r\n Vary: Accept-Language\r\n \r\n 4090\r\n … Request Envelope GET /Test/hello.html HTTP/1.1\r\n Host: kiwi.ics.uci.edu:8080\r\n User-Agent: GET/7 libwww-perl/5.40\r\n \r\n Request Control Data GET /Test/hello.html HTTP/1.1\r\n Host: kiwi.ics.uci.edu:8080\r\n User-Agent: GET/7 libwww-perl/5.40\r\n \r\n Request Metadata GET /Test/hello.html HTTP/1.1\r\n Host: kiwi.ics.uci.edu:8080\r\n User-Agent: GET/7 libwww-perl/5.40\r\n \r\n Request Metadata POST /nasdaq/trade.php HTTP/1.1\r\n Host: www.datek.com\r\n User-Agent: poseur/1.0 libwww/3.1\r\n Content-type: www/x-urlencoded\r\n Content-length: 58\r\n \r\n Ticker=LNUX&quantity=140&action=sell &type=limit&amt=298.00 Request Data POST /nasdaq/trade.php HTTP/1.1\r\n Host: www.datek.com\r\n User-Agent: poseur/1.0 libwww/3.1\r\n Content-type: www/x-urlencoded\r\n Content-length: 58\r\n \r\n Ticker=LNUX&quantity=140&action=sell &type=limit&amt=298.00 Request Data GET /Test/hello.html HTTP/1.1\r\n Host: kiwi.ics.uci.edu:8080\r\n User-Agent: GET/7 libwww-perl/5.40\r\n \r\n Response Envelope GET /Test/hello.html HTTP/1.1\r\n Host: kiwi.ics.uci.edu:8080\r\n User-Agent: GET/7 libwww-perl/5.40\r\n \r\n HTTP/1.1 200 OK\r\n Date: Thu, 09 Mar 2000 15:40:09 GMT\r\n Server: Apache/1.3.12\r\n Content-Type: text/html\r\n Content-Language: en\r\n Transfer-Encoding: chunked\r\n Etag: “a797cd-465af”\r\n Cache-control: max-age=3600\r\n Vary: Accept-Language\r\n \r\n 4090\r\n … Response Control Data GET /Test/hello.html HTTP/1.1\r\n Host: kiwi.ics.uci.edu:8080\r\n User-Agent: GET/7 libwww-perl/5.40\r\n \r\n HTTP/1.1 200 OK\r\n Date: Thu, 09 Mar 2000 15:40:09 GMT\r\n Server: Apache/1.3.12\r\n Content-Type: text/html\r\n Content-Language: en\r\n Transfer-Encoding: chunked\r\n Etag: “a797cd-465af”\r\n Cache-control: max-age=3600\r\n Vary: Accept-Language\r\n \r\n 4090\r\n … Response Metadata GET /Test/hello.html HTTP/1.1\r\n Host: kiwi.ics.uci.edu:8080\r\n User-Agent: GET/7 libwww-perl/5.40\r\n \r\n HTTP/1.1 200 OK\r\n Date: Thu, 09 Mar 2000 15:40:09 GMT\r\n Server: Apache/1.3.12\r\n Content-Type: text/html\r\n Content-Language: en\r\n Transfer-Encoding: chunked\r\n Etag: “a797cd-465af”\r\n Cache-control: max-age=3600\r\n Vary: Accept-Language\r\n \r\n 4090\r\n … Response Data GET /Test/hello.html HTTP/1.1\r\n Host: kiwi.ics.uci.edu:8080\r\n User-Agent: GET/7 libwww-perl/5.40\r\n \r\n HTTP/1.1 200 OK\r\n Date: Thu, 09 Mar 2000 15:40:09 GMT\r\n Server: Apache/1.3.12\r\n Content-Type: text/html\r\n Content-Language: en\r\n Transfer-Encoding: chunked\r\n Etag: “a797cd-465af”\r\n Cache-control: max-age=3600\r\n Vary: Accept-Language\r\n \r\n 4090\r\n … Overview HTTP Basics Apache httpd architecture • • • • • • Connection management (listening) Request Reading Finding a representation Conditional request evaluation Response Writing Connection management (closing) Future Improvements waka/2.0 httpd Initialization httpd core • apache-2.0/src/main/*.c Initiate program • http_main.c Read configuration files • http_config.c, http_core.c Start logging • http_log.c Start server listeners (MPM) • apache-2.0/src/modules/mpm/* Connection management Listening and Accept • mpm/prefork/prefork.c • mpm/mpmt_pthread.c •… Connection set-up • src/main/http_connection.c • Modular hooks for protocol independence • ap_process_http_connection() – Calls ap_read_request() and we are finally at the level of HTTP Request Reading src/main/http_protocol.c GET /Test/hello.html HTTP/1.1\r\n Host: kiwi.ics.uci.edu:8080\r\n User-Agent: GET/7 libwww-perl/5.40\r\n \r\n Request Line • read_request_line(r) Header fields • get_mime_headers(r) Host determination • http_vhost.c Request Processing src/main/http_request.c Access control • Directory walk • Location walk • File walk Handler selection • Field pre-processing • Content Handler Request Body src/main/http_protocol.c Declaration • ap_setup_client_block(r, policy) Test • ap_should_client_block(r) Read until done • ap_get_client_block(r, …) Shortcut • ap_discard_request_body(r) Finding a representation Each content handler is responsible for associating the URI with a resource and applying the method, returning a representation as the response Redirection Negotiation Subrequests Errors Conditional request evaluation  Conditional requests modify method processing depending on the current state of the resource representation • Last-Modified • Etag  Cache validation • If-Modified-Since • If-Nonematch  Prerequisites • • • • If-Match If-Unmodified-Since Range-If If Response Writing HTTP/1.1 200 OK\r\n Date: Thu, 09 Mar 2000 15:40:09 GMT\r\n Server: Apache/1.3.12\r\n Content-Type: text/html\r\n Content-Language: en\r\n Transfer-Encoding: chunked\r\n Etag: “a797cd-465af”\r\n Cache-control: max-age=3600\r\n Vary: Accept-Language\r\n \r\n 4090\r\n … Status code and reason phrase Header fields Body Response Writing Status code and reason phrase • r->status • r->status_line || status_lines[idx] Header fields • r->headers_out • r->err_headers_out Headers set by modules • mod_expires.c • mod_headers.c – Header set Author “John Q. Public” Send Body Body • File • Buffer • IO-layer Transfer-encoding • Length delimited • Chunked • Connection terminated Connection management Persistent connections • The big hairy if • ap_set_keepalive(r) Connection tear-down • Lingering close Overview HTTP Basics Apache httpd architecture • • • • • • Connection management (listening) Request Reading Finding a representation Conditional request evaluation Response Writing Connection management (closing) Future Improvements waka/2.0 Future Improvements Configurable protocol elements • Methods, status codes, field names Tokenized field manipulation • Field name -> token • Field value -> linked list Streaming IO Layers • Stacked protocol manipulation • Separate streams for data and metadata – Token streams • Bucket brigades – Strings, Files, Cache handles waka/2.0  A replacement for HTTP/1.x • Emphasizes the REST architectural style • Ideas from Spero’s HTTP-ng and W3C HTTP-NG  Packet streams • Control, meta-metadata, metadata, data • Prioritized bandwidth usage  Self-descriptive • • • • All syntax an extensible standard All cache rules in response Simple quality of service indicators Transport independent  Tokenized • Optimized for memory transfer Questions? Where to go for more information • http://www.ics.uci.edu/pub/ietf/http/ • http://www.ics.uci.edu/~fielding/talks/ Places to see: • • • • • Front Door Developer Notes PR Database Apache Week O’Reilly Net www.apache.org dev.apache.org bugs.apache.org www.apacheweek.com www.oreilly.net/apache/