12-24-05. Here's an idea of the rough performance for httpserver compared to mathopd. The computer is Mac OS X 10.2.8, 333 MHz iMac, Seagate 40 GB ATA hard drive.
cd cd www/joseph/unix/connect alias tictoc='ps -ax | grep -e httpserver -e mathopd -e lighttpd' tictoc for a in 1701 8081 1702; do # httpserver, mathopd, lighttpd echo port $a time sed "s|localhost|localhost:$a|" < r1 | \ rp 30000 | ./connect1 :$a | wc -c done tictoc
399 ?? S 0:00.43 /usr/local/sbin/mathopd -f /home/myerskid/.14mathopdrc 401 ?? S 0:32.77 /usr/local/sbin/mathopd -f /home/jmyers/.14mathopdrc 22841 std S 0:21.67 ./httpserver 22953 std R+ 0:00.01 grep -e httpserver -e mathopd 1701 57390000 real 0m22.844s user 0m0.180s sys 0m1.980s 8081 58260000 real 0m35.624s user 0m0.130s sys 0m1.930s 399 ?? S 0:00.43 /usr/local/sbin/mathopd -f /home/myerskid/.14mathopdrc 401 ?? R 1:04.78 /usr/local/sbin/mathopd -f /home/jmyers/.14mathopdrc 22841 std S 0:41.04 ./httpserver 22964 std R+ 0:00.01 grep -e httpserver -e mathopd
We can see that the processing time consumed by these 30,000 requests was 41.04 - 32.77 = 8.27 seconds for httpserver--that is 3,628 responses per second of processing. The total phsyical rate was 22.844 seconds for all processes, for an end-to-end delivery speed of 1,313 responses per second.
(Total httpserver output was 57,390,000 bytes, so there was a throughput achieved of 2.512 MB/sec.)
For mathopd, the exact same test performed under the same conditions (not even logging was enabled for mathopd, to give it the maximum performance) resulted in a time difference of 64.78 - 32.77 = 32.01 seconds for mathopd. The mathopd configuration file is here.
Note that httpserver does not cache even a single file, so the results are not artificially inflated. It is the operating system's job to provide cached file data through the open() system call. Other functions shouldn't have been invented. Millions of caching systems doing it their own way create nothing but invalid data. A few performance numbers are boosted, but provide no benefit to real-world requests which do not usually consist of 30,000 similar requests.
As is usual with thoughtless designs, lighttp failed in almost everything: configuration was a mess, installation was a pain. The server lighttpd always wanted to do things its own way. Even when N was reduced to 3,000 (a tenth as much work!), lighttpd gave up with these errors:
2005-12-29 10:10:32: (connections.c.984) http-header larger then 64k -> disconnected 0 2005-12-29 10:10:32: (request.c.471) overlong request line -> 400
(Long lines are wrapped.)
We'll start with something simple like a input file requesting file1; file1 contains a request for file2, which is returned to the first client; file2 is empty, so the loop terminates there. (But this is too laborious; let's eliminate the return back to the first client.)
So we have:
cat r1 GET /index.html HTTP/1.1 Host: localhost cat r2 GET /joseph/unix/connect/r1 HTTP/1.1 Host: localhost ./connect1 :1701 < r2 | ./connect1 :1701 [400 bad request] [output of index.html]
We get a 400 bad request before index.html, because we forgot that connect sends back all of the output. So the HTTP headers of the first response are forwarded to the server. Some HTTP servers would disconnect at this point, and no more output could be obtained. But httpserver simply gives a message that indicates the bad request. (A server either has to close the connection, or give some response for every request, since otherwise requests and responses are not one-to-one, and the client will not know which response goes with which request in a series of requests in the pipeline.)
So now we'll do this:
./connect1 :1701 < r2 | tail -3 | ./connect1 :1701
This is kind of worthless, though.
tictoc; time rp 30000 < ./connect1 :1701 | wc -c; tictoc PID TT STAT TIME COMMAND 42548 ?? Ss 0:00.01 www.codelib.net/c/httpserver/httpserver -d .:1701 236670000 real 0m7.093s user 0m6.323s sys 0m0.778s PID TT STAT TIME COMMAND 42548 ?? Ss 0:00.01 www.codelib.net/c/httpserver/httpserver -d .:1701
The processing time for these requests (inside of httpserver) was less than 0.01 seconds, which would be a rate of 3,000,000 requests processed per second. The total physical rate was 7.093 seconds (on a somewhat busy server running 381 other processes at this time), so the end-to-end delivery speed was 4229.5 requests-and-responses per second. Total out put was 236670000, so the throughput was 33.366 MB/sec. (The index.html in this test was a larger file.)
tictoc; time rp 10000 < r1 | ./connect1 :1701 | wc -c; tictoc PID TTY STAT TIME COMMAND 3817 ? Ss 0:00 www.codelib.net/c/httpserver/httpserver -d .:1701 6000000 real 0m0.489s user 0m0.004s sys 0m0.074s PID TTY STAT TIME COMMAND 3817 ? Ss 0:00 www.codelib.net/c/httpserver/httpserver -d .:1701
So we get 20,449 physical requests and responses per second, on a 1.1 GHz single-processor machine. This is on CentOS 4.2.
I really need to improve my benchmarker.
ls -l httpserver /usr/local/sbin/mathopd -rwxr-xr-x 1 root staff 88024 Sep 6 2004 /usr/local/sbin/mathopd -rwxr-xr-x 1 jmyers staff 21276 Dec 24 12:55 httpserver
The server mathopd is one of the smallest, most lightweight, and efficient servers. Therefore, it is a significant accomplishment that httpserver has been reduced below 88 KB; in fact, its binary size is under 21 KB and less than a fourth the size of mathopd. (1-16-05. On FreeBSD 5.4, the binary size of httpserver is 14856, less than 15 KB.)
The compile date reflects the fact that new mime types from mime.types were compiled into the httpserver program on 12-24-05.
The code has not changed since March 25, 2005. There have been no functional changes since 2004. The server planning and design started in 2002.