Performance Comparison C vs. Java vs. Javascript vs. PHP vs. Python vs. Cobol vs. Dart

Original post is here: eklausmeier.goip.de

1. Introduction. This post is a continuation of a previous benchmark in Performance Comparison C vs. Java vs. Javascript vs. LuaJIT vs. PyPy vs. PHP vs. Python vs. Perl. Here we compare

1. C
2. Java
3. JavaScript
4. Python
5. Cobol
   • GnuCOBOL
   • gcobol (gcc based COBOL)
6. Dart

I tried to run IBM COBOL as well, but installation as described in Installing IBM COBOL for Linux on Arch Linux failed.

Testing machine is a Ryzen 7 5700G (Cezanne) with 16 CPUs and 4672.0698 MHz max clock. Operating system is Arch Linux 6.2.6-arch1-1. Source code for the programs is in Performance Comparison C vs. Java vs. Javascript vs. LuaJIT vs. PyPy vs. PHP vs. Python vs. Perl.

2. Results. I again use the n-queens problem, i.e., how many times can n queens be put on an n x n chess board without attacking any other queen. Task at hand is to compute from n=1 to n=13 all possible combinations. For example, for C I call time xdamcnt2 13. I ran the programs multiple times and took the lowest result. Of course, all programs produced exactly the same results -- all computations are integer computations. Expected results are:

Runtimes in real and user seconds are given in below table. Times taken with time command.

3. Special modifications in COBOL program. Simon Sobisch, in private e-mail from 13-Mar-2023 to me, made below changes to the COBOL program:

1. No variable with sign any more, i.e., variable is always greater or equal zero
2. Compiler optimization flag -fno-trunc

Modifications by Simon Sobisch:

 121c21
 2<        77 l             pic s9(8) comp-5.
 3---
 4>        77 l             pic 9(8) comp-5.
 5139,149c134,146
 6<                compute l = z - A(j)
 7<                if l = 0 then
 8<       *            exit section
 9<                    go to configOK-exit
10<                end-if
11<                if l < 0 then
12<                    compute l = 0 - l
13<                end-if
14<                if l = k - j then
15<       *            exit section
16<                    go to configOK-exit
17---
18>                evaluate true
19>                when z < A(j)
20>                    move A(j) to l
21>                    subtract z from l
22>                when z = A(j)
23>                    exit section
24>                when other
25>                    move z to l
26>                    subtract A(j) from l
27>                end-evaluate
28>                add j to l
29>                if l = k then
30>                    exit section

4. Conclusions.

1. C is by far the fastest. Faster than Java by a factor of 1.5. Almost two times faster than Dart as its other closest competitor.
2. Java is roughly 20% faster than Javascript.
3. JavaScript and Dart have almost the same performance.
4. Cobol can be made faster than JavaScript and Dart but slightly slower than Java.
5. PHP 8 is two times faster than Python, but almost 25-times slower than Javascript.
6. PHP with JIT is almost three times faster than normal PHP.
7. PHP with JIT enabled is five times faster than Python, but nine times slower than Javascript.

5. Dart program. Dart is a programming language by Google and used for Flutter.

 1int abs(int x) { return  ((x >= 0) ? x : -x); }
 2
 3/* Check if k-th queen is attacked by any other prior queen.
 4   Return nonzero if configuration is OK, zero otherwise.
 5*/
 6bool configOkay (int k, List<int> a) {
 7	int z = a[k];
 8
 9	for (int j=1; j<k; ++j) {
10		int l = z - a[j];
11		if (l == 0  ||  abs(l) == k - j) return false;
12	}
13	return true;
14}
15
16
17int solve (int N, List<int> a) {  // return number of positions
18	int cnt = 0;
19	int k = a[1] = 1;
20	int N2 = N;  //(N + 1) / 2;
21	bool flag = false;
22
23	for (;;) {
24		if (configOkay(k,a)) {
25			if (k < N)  { a[++k] = 1;  continue; }
26			else ++cnt;
27		}
28		flag = false;
29		do
30			if (a[k] < N)  { a[k] += 1;  flag = true; break; }
31		while (--k > 1);
32		if (flag) continue;
33		a[1] += 1;
34		if (a[1] > N2) return cnt;
35		a[k=2] = 1;
36	}
37}
38
39
40void main (List<String> argv) {
41	int NMAX = 100;
42	List<int> a = [ for(var i=0;i<100;++i) 0 ];
43	int argc = argv.length;
44
45	print(" n-queens problem.\n"
46	"   2   4    6     8      10         12           14\n"
47	" 1 0 0 2 10 4 40 92 352 724 2680 14200 73712 365596\n"
48	);
49
50	int start = 1;
51	int end = 0;
52	if (argc >= 1)  end = int.parse(argv[0]);
53	if (end <= 0  ||  end >= NMAX)  end = 10;
54	if (argc >= 2) { start = end;  end = int.parse(argv[1]); }
55	if (end <= 0  ||  end >= NMAX)   end = 10;
56
57	for (int n=start; n<=end; ++n)
58		print(" D(${n}) = ${solve(n,a)}");
59}

Added 28-Sep-2024: Francesco Cristiano on 24-Sep-2024 wrote to me and stated that inlining the abs() function in PHP would reduce the runtime by one second. That inlining is fully justified as the C program in question also uses a macro like:

1#define abs(x)   ((x >= 0) ? x : -x)

So doing the same in PHP:

1if ($l == 0  ||  ($l >= 0 ? $l : -$l) == $k - $j) return 0;

But this change in the PHP program not only reduces the runtime by one second, it almost halves the runtime!

The original benchmark was done more than a year ago. The environment now has changed a bit: Linux is now 6.10.10, and PHP is 8.3.12. Old program under this changed environment:

 1$ time php xdamcnt2.php 13
 2array(2) {
 3  [0]=>
 4  string(12) "xdamcnt2.php"
 5  [1]=>
 6  string(2) "13"
 7}
 8 n-queens problem.
 9   2   4    6     8      10         12           14
10 1 0 0 2 10 4 40 92 352 724 2680 14200 73712 365596
11 D( 1) = 1
12 D( 2) = 0
13 D( 3) = 0
14 D( 4) = 2
15 D( 5) = 10
16 D( 6) = 4
17 D( 7) = 40
18 D( 8) = 92
19 D( 9) = 352
20 D(10) = 724
21 D(11) = 2680
22 D(12) = 14200
23 D(13) = 73712
24        real 6.71s
25        user 6.69s
26        sys 0
27        swapped 0
28        total space 0

Inlined program:

 1time php xdamcnt3.php 13
 2array(2) {
 3  [0]=>
 4  string(12) "xdamcnt3.php"
 5  [1]=>
 6  string(2) "13"
 7}
 8 n-queens problem.
 9   2   4    6     8      10         12           14
10 1 0 0 2 10 4 40 92 352 724 2680 14200 73712 365596
11 D( 1) = 1
12 D( 2) = 0
13 D( 3) = 0
14 D( 4) = 2
15 D( 5) = 10
16 D( 6) = 4
17 D( 7) = 40
18 D( 8) = 92
19 D( 9) = 352
20 D(10) = 724
21 D(11) = 2680
22 D(12) = 14200
23 D(13) = 73712
24        real 3.53s
25        user 3.50s
26        sys 0
27        swapped 0
28        total space 0

Thanks Francesco.

Added 28 Sept 2024 at 15:14, Francesco Cristiano wrote:

Hi Elmar,

Glad to have contributed to your benchmark. I have been using PHP 8-jit to implement a power-to-ammonia simulation plant (with 3 millions time-steps) all through CLI and the execution time is indeed impressive. I don't see why people keep saying that PHP is a scripting language for webserver only.

Cheers Fran

Added 02-Oct-2024, Francesco Cristiano wrote:

Hi Elmar,

I've been testing how close PHP can get to Java using your n-queen code as a benchmark, and after using some tricks to improve the PHP code I'm getting some interesting results: This is the tweaked PHP code:

 1function solve ( $N, &$a) {  // return number of positions
 2    $cnt = 0;
 3    $k = $a[1] = 1;
 4    $N2 = $N;  //(N + 1) / 2;
 5
 6    for (;;) {
 7        $flag = false;
 8
 9        //code equivalent to configOkay function
10        $configOkayFlag=true;
11        {
12
13            $z = $a[$k];
14
15            for ($j=1; $j<$k; ++$j) {
16                $l = $z - $a[$j];
17                if ($l == 0  ||  (($l >= 0) ? $l : -$l) == $k - $j) {
18                    $configOkayFlag= false;
19                    break;
20                }
21
22            }
23        }
24        // end of code equivalent to configOkay function
25
26        //if (configOkay ($k, $a)) {
27        if ( $configOkayFlag) {
28            if ($k < $N)  {
29                $a[++$k] = 1;
30                continue;
31            }
32            else
33                ++$cnt;
34        }
35
36        do
37            if ($a[$k] < $N)  {
38                $a[$k] += 1;
39                $flag = true;
40                break;
41            }
42        while (--$k > 1);
43
44        if ($flag)
45            continue;
46
47        $a[1] += 1;
48        if ($a[1] > $N2)
49            return  $cnt;
50        $k = 2 ;
51        $a[2] = 1;
52    }
53}
54$get_as_float = true;
55$start = microtime(true);
56$a= [];
57for($i=1; $i <= 13; $i++){
58
59    $result=solve($i,$a);
60
61    echo "i=$i - number of combinations: $result\n";
62
63}
64$time_elapsed_secs = microtime(true) - $start;
65echo "time_elapsed_secs:$time_elapsed_secs \n";

And the equivalent Java code is the following:

 1class bench {
 2
 3public static long solve (int N, int[] a) {  // return number of positions
 4        long cnt = 0;
 5        int k = a[1] = 1;
 6        int N2 = N;  //(N + 1) / 2;
 7        boolean flag;
 8
 9        for (;;) {
10            flag = false;
11
12            //code equivalent to configOkay function
13            boolean configOkayFlag=true;
14             {
15                int z = a[k];
16
17                for (int j=1; j<k; ++j) {
18                    int l = z - a[j];
19                    if (l == 0  || ((l >= 0) ? l : -l) == k - j) {
20                        configOkayFlag= false;
21                        break;
22                    }
23
24                }
25            }
26            // end of  code equivalent to configOkay function
27
28            //if (configOkay(k,a)) {
29            if ( configOkayFlag) {
30                if (k < N)  { a[++k] = 1;  continue; }
31                else ++cnt;
32            }
33            do
34                if (a[k] < N)  { a[k] += 1;  flag = true; break; }
35            while (--k > 1);
36            if (flag) continue;
37            a[1] += 1;
38            if (a[1] > N2) return cnt;
39            k = 2;  a[2] = 1;
40        }
41    }
42
43public static void main(String[] args){
44long startTime = System.currentTimeMillis();
45
46int[] a = new int[73713];
47
48        for(int i=1; i<=13; i++){
49            System.out.println("i="+i+ " - number of combinations: "+ bench.solve(i, a));
50        }
51        long stopTime = System.currentTimeMillis();
52        long elapsedTime = stopTime - startTime;
53
54        System.out.println("Total time: " + elapsedTime +"ms");
55}
56}

These are the results I'm getting:

c:\php8.4\php.exe -f bench.php
i=1 - number of combinations: 1
i=2 - number of combinations: 0
i=3 - number of combinations: 0
i=4 - number of combinations: 2
i=5 - number of combinations: 10
i=6 - number of combinations: 4
i=7 - number of combinations: 40
i=8 - number of combinations: 92
i=9 - number of combinations: 352
i=10 - number of combinations: 724
i=11 - number of combinations: 2680
i=12 - number of combinations: 14200
i=13 - number of combinations: 73712
time_elapsed_secs:1.577357

java bench
i=1 - number of combinations: 1
i=2 - number of combinations: 0
i=3 - number of combinations: 0
i=4 - number of combinations: 2
i=5 - number of combinations: 10
i=6 - number of combinations: 4
i=7 - number of combinations: 40
i=8 - number of combinations: 92
i=9 - number of combinations: 352
i=10 - number of combinations: 724
i=11 - number of combinations: 2680
i=12 - number of combinations: 14200
i=13 - number of combinations: 73712
Total time: 875ms

Not bad, PHP is only taking 1.8 times the execution time taken by Java.

Cheers Fran