Multiple stacks

[motin]

Fixes #33 and includes fix for #34
Simplifies the fix for #8

Update: This PR also includes more robust results parsing, a php-based framework list, as well as fixes for #36 and #38 - all required to properly support multiple stacks.

[kenjis]

I can't accept this change. It exclude shutdown process of the framework.

[motin]

And why do you have to remove $_SERVER['DOCUMENT_ROOT']? I hope the code exactly the same for all frameworks.

The docker stacks are not served in the sub-directory php-framework-benchmark (since it does not make sense to do so when running fully dedicated stacks for the benchmarks), thus relying on the document root + a sub-directory is not working in both cases. Dirname + __FILE__ always works, but I can refactor to use an environment variable instead.

Regarding putting require libs/output_data.php in the actual controller method. I can't accept this change. It exclude shutdown process of the framework. And why do you need to echo?

Many frameworks required the response to be returned in the actual controller method and not echoed after the framework has been shutdown. To be able to return the value without using output buffering, I changed the printf to sprintf. This also makes it ready to benchmark event-driven frameworks such as http://reactphp.org/

I traced this back to #38
In the process, I also came across #36

[motin]

The latest commits also fixes #36 and #38

[kenjis]

The docker stacks are not served in the sub-directory php-framework-benchmark (since it does not make sense to do so when running fully dedicated stacks for the benchmarks)

The docker stacks should have the same sub-directory. So you don't have to change the code only for the docker stacks.

[kenjis]

BEAR.Sunday should work fine on php7.

[motin]

The docker stacks should have the same sub-directory. So you don't have to change the code only for the docker stacks.

Fixed and reversed the changed code.

[motin]

Btw, interesting results already especially regarding HHVM: facebook/hhvm#6645

[kenjis]

Please remove this. I don't need it. I don't maintain the results of Docker.

[motin]

This is not primarily for you :) It is for everyone that easily wants to run their own benchmarks. Instead of having to set-up several server environments, compile php extensions and configure their own server software to run all the different framework, they can instead simply install docker toolbox and be running within minutes.

I will remove the change from this PR in hope that you will consider this approach in the future: #41

[kenjis]

Benchmark results in Docker do not matter unless you or someone deploy using it.

Is your production environment Docker on Mac OS X? If not, the benchmark results might mislead you.

[motin]

My production environment is Docker in AWS instances and it is trivial to run the benchmarks as well when dealing with Docker stacks. Let's discuss in #41

[kenjis]

Why don't you move docker-results.sh and docker-urls.sh to docker/bin/?

[kenjis]

ab command is missing.

$ docker-compose run shell
root@889a4a34285c:/repo# sh setup.sh
ab command not found.

[motin]

Why don't you move docker-results.sh and docker-urls.sh to docker/bin/?

Sure, now when there are three scripts it makes more sense. When there was only a single script, it felt odd to create a new bin-folder instead of re-using the existing one.

[motin]

ab command is missing.

Weird, it is installed at https://github.com/motin/php-framework-benchmark/blob/e720dd142933fed0ffe3247bce6874fa5d900668/docker/stack/php/shell/Dockerfile#L3

I'll add docker-compose build shell to the readme so that there is no chance of it not being included.

[kenjis]

Weird, it is installed at https://github.com/motin/php-framework-benchmark/blob/e720dd142933fed0ffe3247bce6874fa5d900668/docker/stack/php/shell/Dockerfile#L3

You're right. I don't know why. I did according to your README.
I'll try it again later.