互联网，请记住我

用ab来做性能测评好像不公平吧?

08月 21, 2007 on 11:47 pm | In php性能, php技术 |

我的师兄,从某种程度上说也是我php方面的师父,和他的同事做了测评,分别是针对php4,php5的性能对比和function,class的性能对比的。
这里是师兄的对比结果
师兄的同事做的测评
但是我不认为用ab来做出的测试是合理的.理由:用ab来测试时,结果好像总似是在跳舞一样.做为证据,我将我的notebook依文本方式重启,用ab来测lighttpd的表现.
具体环境:
Haier H40S Notebook
RAM:1.5G
CPU:CY 1.6GHz
硬盘:60G,(具体型号什么的不清楚,列个60G表示不是SCSI硬盘)
OS:Fedora Core 7.0(Moonshine)
web server:lighttpd 1.4.15
测试工具:ab
X环境:未运行X server
命令:ab -n 10000 -c 50 http://localhost:8181/
第一份结果:

Benchmarking localhost (be patient)

Server Software: lighttpd/1.4.15
Server Hostname: localhost
Server Port: 8181

Document Path: /
Document Length: 345 bytes

Concurrency Level: 50
Time taken for tests: 1.786830 seconds
Complete requests: 10000
Failed requests: 0
Write errors: 0
Non-2xx responses: 10000
Total transferred: 4980000 bytes
HTML transferred: 3450000 bytes
Requests per second: 5596.50 [#/sec] (mean)
Time per request: 8.934 [ms] (mean)
Time per request: 0.179 [ms] (mean, across all concurrent requests)
Transfer rate: 2721.58 [Kbytes/sec] received

Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 3 2.6 4 8
Processing: 1 4 0.7 5 6
Waiting: 0 1 1.7 2 5
Total: 5 8 2.2 8 13
WARNING: The median and mean for the processing time are not within a normal deviation
These results are probably not that reliable.

Percentage of the requests served within a certain time (ms)
50% 8
66% 10
75% 10
80% 11
90% 11
95% 12
98% 12
99% 12
100% 13 (longest request)

这里的结果是Requests per second: 5596.50 [#/sec] (mean)(插一句,lighttpd的性能真是好,用apache从来没上过3000)
第二份ab -n 10000 -c 50的结果:

Benchmarking localhost (be patient)

Server Software: lighttpd/1.4.15
Server Hostname: localhost
Server Port: 8181

Document Path: /
Document Length: 345 bytes

Concurrency Level: 50
Time taken for tests: 1.313719 seconds
Complete requests: 10000
Failed requests: 0
Write errors: 0
Non-2xx responses: 10011
Total transferred: 4985478 bytes
HTML transferred: 3453795 bytes
Requests per second: 7611.98 [#/sec] (mean)
Time per request: 6.569 [ms] (mean)
Time per request: 0.131 [ms] (mean, across all concurrent requests)
Transfer rate: 3705.51 [Kbytes/sec] received

Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 1 1.0 1 5
Processing: 1 5 2.0 5 32
Waiting: 0 3 1.8 3 30
Total: 1 6 2.1 6 35

Percentage of the requests served within a certain time (ms)
50% 6
66% 6
75% 6
80% 6
90% 7
95% 8
98% 10
99% 11
100% 35 (longest request)

结果表明RPS是Requests per second: 7611.98 [#/sec] (mean).
第三份结果:

Benchmarking localhost (be patient)

Server Software: lighttpd/1.4.15
Server Hostname: localhost
Server Port: 8181

Document Path: /
Document Length: 345 bytes

Concurrency Level: 50
Time taken for tests: 1.300247 seconds
Complete requests: 10000
Failed requests: 0
Write errors: 0
Non-2xx responses: 10009
Total transferred: 4984482 bytes
HTML transferred: 3453105 bytes
Requests per second: 7690.85 [#/sec] (mean)
Time per request: 6.501 [ms] (mean)
Time per request: 0.130 [ms] (mean, across all concurrent requests)
Transfer rate: 3743.14 [Kbytes/sec] received

Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 1 1.0 1 5
Processing: 1 4 1.4 5 11
Waiting: 0 2 1.1 3 8
Total: 1 6 1.2 6 14

Percentage of the requests served within a certain time (ms)
50% 6
66% 6
75% 6
80% 6
90% 7
95% 9
98% 10
99% 11
100% 14 (longest request)

RPS结果是Requests per second: 7690.85 [#/sec] (mean).
第四份结果是:

Benchmarking localhost (be patient)

Server Software: lighttpd/1.4.15
Server Hostname: localhost
Server Port: 8181

Document Path: /
Document Length: 345 bytes

Concurrency Level: 50
Time taken for tests: 1.315784 seconds
Complete requests: 10000
Failed requests: 0
Write errors: 0
Non-2xx responses: 10033
Total transferred: 4996434 bytes
HTML transferred: 3461385 bytes
Requests per second: 7600.03 [#/sec] (mean)
Time per request: 6.579 [ms] (mean)
Time per request: 0.132 [ms] (mean, across all concurrent requests)
Transfer rate: 3708.06 [Kbytes/sec] received

Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 1 1.2 1 8
Processing: 1 4 1.4 5 11
Waiting: 0 2 1.1 3 8
Total: 1 6 1.4 6 15

Percentage of the requests served within a certain time (ms)
50% 6
66% 6
75% 6
80% 6
90% 7
95% 9
98% 11
99% 12
100% 15 (longest request)

结果是Requests per second: 7600.03 [#/sec] (mean).
第五份结果是:

Benchmarking localhost (be patient)

Server Software: lighttpd/1.4.15
Server Hostname: localhost
Server Port: 8181

Document Path: /
Document Length: 345 bytes

Concurrency Level: 50
Time taken for tests: 1.812989 seconds
Complete requests: 10000
Failed requests: 0
Write errors: 0
Non-2xx responses: 10000
Total transferred: 4980000 bytes
HTML transferred: 3450000 bytes
Requests per second: 5515.75 [#/sec] (mean)
Time per request: 9.065 [ms] (mean)
Time per request: 0.181 [ms] (mean, across all concurrent requests)
Transfer rate: 2682.31 [Kbytes/sec] received

Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 3 2.7 4 9
Processing: 1 4 0.8 5 6
Waiting: 0 1 1.7 2 5
Total: 5 8 2.3 9 13
WARNING: The median and mean for the processing time are not within a normal deviation
These results are probably not that reliable.

Percentage of the requests served within a certain time (ms)
50% 9
66% 10
75% 10
80% 11
90% 12
95% 12
98% 12
99% 12
100% 13 (longest request)

RPS结果是Requests per second: 5515.75 [#/sec] (mean);

可以看到,同是ab -n 10000 -c 50 http://localhost:8181/,结果大不一样,不仅分为7600RPS和5500RPS两档,而各档中具体数据也不一样.这个结果可以理解,OS中总是会有各种各样的消息要处理,影响具体数据结果也很正常.
因此,谢华亮(sir?mm?)的测试用ab的结果来进行php4,php5的性能比较是不太对的.至少目前看来,我没有从他的文章中看到他进行了多次ab测评后取结果的平均值.但是他列出来这样一个结果:

这大致可以说明,php5.2确实是比php4.4.2要快.但是从1197.17>1176.06得出php5的类跑得比functiong还快,我不这么认为.只是21个request的差别,太小了,ab测试的随便一个风吹草吹就行了(而且说class比function要快,严重地伤害了我的感情,哈哈)…
嗯,先整这么多.有空好好研究一下,如何对web程序进行性能测评.
另外,有空见到这两位兄台,好好争论一番.