sendfileをどうしましょう nginx vs apache2 vs sinatra+unicorn vs node - ニート=>ネトゲ屋=>ブラック=>電話屋

ファイルを送信するときには許すのならアプリケーションで送信するよりもnginxかapache2で送信する方が良いっぽい。当たり前だけど。
ローカルの各サーバーへリクエストを投げて試してみた。request対象ファイルは4.2Mのmp3ファイル。

環境
varnish => apache2(port:81)
varnish => nginx(port:8080)
varnish => node(port:23001)
varnish => nginx(port:8080) =>unixソケット sinatra+unicorn

生sinatra-unicornは運用上ありえないと思うので除外した。
生nodeもありえないかもですが。

nginx           77.7 req/s Net I/O: 312120.8 KB/s
apache2       55.7 req/s Net I/O: 223899.3 KB/s
varnish         43.7 req/s Net I/O: 175443.2 KB/s
node            16.6 req/s Net I/O:   66824.2 KB/s
sinatra+unicorn 8.0  req/s Net I/O:  32260.7 KB/s

sinatra-unicornとnginxとの差は10倍弱ある。sinatraはrailsよりもルーティングが速い(とおもうのですが)のにこの差になる。varnishに乗っかればnode/sinatra-unicornの速度は
varnishはアプリケーションをwebサーバーでの静的ファイルの送信レベルまで持ち上げてくれそう。
nodeはsinatra-unicornの2倍のパフォーマンスが出るので部分で使い分けができると良い。ともかくnginxが速い。

簡素なベンチ

nginx 77.7 req/s (12.9 ms/req) Net I/O: 312120.8 KB/s (2556.9*10^6 bps)

nginxはpassenger-install-nginx-moduleで入ったもの
$ httperf --hog --client=0/1 --server=192.168.110.7 --port=8080 --uri=/resource/music/iTunesMac/Vocaloid/impacts/02.mp3 --send-buffer=4096 --recv-buffer=16384 --num-conns=100 --num-calls=10
httperf --hog --client=0/1 --server=192.168.110.7 --port=8080 --uri=/resource/music/iTunesMac/Vocaloid/impacts/02.mp3 --send-buffer=4096 --recv-buffer=16384 --num-conns=100 --num-calls=10
httperf: warning: open file limit > FD_SETSIZE; limiting max. # of open files to FD_SETSIZE
Maximum connect burst length: 1

Total: connections 100 requests 1000 replies 1000 test-duration 12.870 s

Connection rate: 7.8 conn/s (128.7 ms/conn, <=1 concurrent connections)
Connection time [ms]: min 123.1 avg 128.7 max 257.2 median 124.5 stddev 19.2
Connection time [ms]: connect 0.2
Connection length [replies/conn]: 10.000

Request rate: 77.7 req/s (12.9 ms/req)
Request size [B]: 114.0

Reply rate [replies/s]: min 75.4 avg 77.0 max 78.6 stddev 2.3 (2 samples)
Reply time [ms]: response 0.6 transfer 12.3
Reply size [B]: header 220.0 content 4113118.0 footer 0.0 (total 4113338.0)
Reply status: 1xx=0 2xx=1000 3xx=0 4xx=0 5xx=0

CPU time [s]: user 0.78 system 8.78 (user 6.0% system 68.2% total 74.2%)
Net I/O: 312120.8 KB/s (2556.9*10^6 bps)

Errors: total 0 client-timo 0 socket-timo 0 connrefused 0 connreset 0
Errors: fd-unavail 0 addrunavail 0 ftab-full 0 other 0

apache2 55.7 req/s Net I/O: 223899.3 KB/s (1834.2*10^6 bps)

//apacheはapt-getで入った状態のもの
$ httperf --hog --client=0/1 --server=192.168.110.7 --port=81 --uri=/resource/music/iTunesMac/Vocaloid/impacts/02.mp3 --send-buffer=4096 --recv-buffer=16384 --num-conns=100 --num-calls=10
httperf --hog --client=0/1 --server=192.168.110.7 --port=81 --uri=/resource/music/iTunesMac/Vocaloid/impacts/02.mp3 --send-buffer=4096 --recv-buffer=16384 --num-conns=100 --num-calls=10
httperf: warning: open file limit > FD_SETSIZE; limiting max. # of open files to FD_SETSIZE
Maximum connect burst length: 1

Total: connections 100 requests 1000 replies 1000 test-duration 17.941 s

Connection rate: 5.6 conn/s (179.4 ms/conn, <=1 concurrent connections)
Connection time [ms]: min 127.3 avg 179.4 max 332.4 median 183.5 stddev 43.6
Connection time [ms]: connect 0.3
Connection length [replies/conn]: 10.000

Request rate: 55.7 req/s (17.9 ms/req)
Request size [B]: 114.0

Reply rate [replies/s]: min 45.6 avg 52.0 max 61.0 stddev 8.0 (3 samples)
Reply time [ms]: response 1.0 transfer 16.9
Reply size [B]: header 244.0 content 4113118.0 footer 0.0 (total 4113362.0)
Reply status: 1xx=0 2xx=1000 3xx=0 4xx=0 5xx=0

CPU time [s]: user 0.71 system 9.08 (user 4.0% system 50.6% total 54.6%)
Net I/O: 223899.3 KB/s (1834.2*10^6 bps)

Errors: total 0 client-timo 0 socket-timo 0 connrefused 0 connreset 0
Errors: fd-unavail 0 addrunavail 0 ftab-full 0 other 0

varnish 43.7 req/s Net I/O: 175443.2 KB/s

$ httperf --hog --client=0/1 --server=192.168.110.7 --port=80 --uri=/resource/music/iTunesMac/Vocaloid/impacts/02.mp3 --send-buffer=4096 --recv-buffer=16384 --num-conns=100 --num-calls=10
httperf --hog --client=0/1 --server=192.168.110.7 --port=80 --uri=/resource/music/iTunesMac/Vocaloid/impacts/02.mp3 --send-buffer=4096 --recv-buffer=16384 --num-conns=100 --num-calls=10
httperf: warning: open file limit > FD_SETSIZE; limiting max. # of open files to FD_SETSIZE
Maximum connect burst length: 1

Total: connections 100 requests 1000 replies 1000 test-duration 22.897 s

Connection rate: 4.4 conn/s (229.0 ms/conn, <=1 concurrent connections)
Connection time [ms]: min 146.6 avg 229.0 max 315.7 median 260.5 stddev 57.9
Connection time [ms]: connect 0.5
Connection length [replies/conn]: 10.000

Request rate: 43.7 req/s (22.9 ms/req)
Request size [B]: 114.0

Reply rate [replies/s]: min 37.4 avg 44.6 max 64.4 stddev 13.2 (4 samples)
Reply time [ms]: response 0.9 transfer 22.0
Reply size [B]: header 304.0 content 4113118.0 footer 0.0 (total 4113422.0)
Reply status: 1xx=0 2xx=1000 3xx=0 4xx=0 5xx=0

CPU time [s]: user 0.85 system 9.16 (user 3.7% system 40.0% total 43.7%)
Net I/O: 175443.2 KB/s (1437.2*10^6 bps)

Errors: total 0 client-timo 0 socket-timo 0 connrefused 0 connreset 0
Errors: fd-unavail 0 addrunavail 0 ftab-full 0 other 0

node 16.6 req/s Net I/O: 66824.2 KB/s

$ httperf --hog --client=0/1 --server=192.168.110.7 --port=23001 --uri=/stream/02.mp3 --send-buffer=4096 --recv-buffer=16384 --num-conns=100 --num-calls=10
httperf --hog --client=0/1 --server=192.168.110.7 --port=23001 --uri=/stream/02.mp3 --send-buffer=4096 --recv-buffer=16384 --num-conns=100 --num-calls=10
httperf: warning: open file limit > FD_SETSIZE; limiting max. # of open files to FD_SETSIZE
Maximum connect burst length: 1

Total: connections 100 requests 1000 replies 1000 test-duration 60.114 s

Connection rate: 1.7 conn/s (601.1 ms/conn, <=1 concurrent connections)
Connection time [ms]: min 535.4 avg 601.1 max 749.1 median 610.5 stddev 39.2
Connection time [ms]: connect 1.0
Connection length [replies/conn]: 10.000

Request rate: 16.6 req/s (60.1 ms/req)
Request size [B]: 79.0

Reply rate [replies/s]: min 15.0 avg 16.6 max 17.6 stddev 0.7 (12 samples)
Reply time [ms]: response 2.9 transfer 57.1
Reply size [B]: header 287.0 content 4113118.0 footer 0.0 (total 4113405.0)
Reply status: 1xx=0 2xx=1000 3xx=0 4xx=0 5xx=0

CPU time [s]: user 6.21 system 23.51 (user 10.3% system 39.1% total 49.4%)
Net I/O: 66824.2 KB/s (547.4*10^6 bps)

Errors: total 0 client-timo 0 socket-timo 0 connrefused 0 connreset 0
Errors: fd-unavail 0 addrunavail 0 ftab-full 0 other 0

sinatra+unicorn(workker4) 8.0 req/s Net I/O: 32260.7 KB/s

$ httperf --hog --client=0/1 --server=192.168.110.7 --port=8080 --uri=/musicdb_dev/test/02mp3 --send-buffer=4096 --recv-buffer=16384 --num-conns=100 --num-calls=10
httperf --hog --client=0/1 --server=192.168.110.7 --port=8080 --uri=/musicdb_dev/test/02mp3 --send-buffer=4096 --recv-buffer=16384 --num-conns=100 --num-calls=10
httperf: warning: open file limit > FD_SETSIZE; limiting max. # of open files to FD_SETSIZE
Maximum connect burst length: 1

Total: connections 100 requests 1000 replies 1000 test-duration 124.520 s

Connection rate: 0.8 conn/s (1245.2 ms/conn, <=1 concurrent connections)
Connection time [ms]: min 1069.7 avg 1245.2 max 2046.2 median 1234.5 stddev 108.1
Connection time [ms]: connect 0.3
Connection length [replies/conn]: 10.000

Request rate: 8.0 req/s (124.5 ms/req)
Request size [B]: 88.0

Reply rate [replies/s]: min 6.6 avg 8.0 max 8.4 stddev 0.3 (24 samples)
Reply time [ms]: response 15.8 transfer 108.7
Reply size [B]: header 316.0 content 4113118.0 footer 0.0 (total 4113434.0)
Reply status: 1xx=0 2xx=1000 3xx=0 4xx=0 5xx=0

CPU time [s]: user 10.98 system 38.95 (user 8.8% system 31.3% total 40.1%)
Net I/O: 32260.7 KB/s (264.3*10^6 bps)

Errors: total 0 client-timo 0 socket-timo 0 connrefused 0 connreset 0
Errors: fd-unavail 0 addrunavail 0 ftab-full 0 other 0

nodeとsinatraのソース

app.get("/stream/02.mp3",function(req,res){
    res.sendfile("/var/www/resource/music/iTunesMac/Vocaloid/impacts/02.mp3");
  });

get "musicdb_/test/02mp3" do
  send_file("/var/smb/sdb1/music/iTunesMac/Vocaloid/impacts/02.mp3")
end