Discussion Forum

We previously reported some benchmark data using COMSOL-CFD 5.2 on six different high-end computers, and we now report some more data.

Our benchmark results are at odds with what one would expect from the information one finds on the various COMSOL knowledgebase pages (866, 1001, 1096, etc.).

The computers we tested are listed below in order from fastest to slowest on a large COMSOL-CFD-Mixer k-w model (rotating machinery, compressible non-isothermal high-velocity flows, over 1M meshcells).

1. Two E5-2687 v3’s (Q3’14); 512 GB DDR4/1866; (each processor: 10 cores, 3.5 GHz turbo, 68 GB/s, 25MB cache); C612 chip set. SSD.
2. Two E5-2640 v3’s (Q3’14); 128 GB DDR4/1866; (each processor: 8 cores, 3.4 GHz turbo, 59 GB/s, 20MB cache); C612; 6GB/s. SSD.
3. An i7-3930K (Q4’11); 64 GB DDR3; (6 cores, 3.2 GHz turbo, 51 GB/s, 12MB cache).
4. An i7-6820HQ (Q3’15); 64 GB DDR4; (4 cores, 3.5 GHz turbo, 34 GB/s 8MB cache); SSD; a laptop.
5. An i7-5960X (Q2’16); 128 GB DDR4; (10 cores, 4 GHz turbo, 68 GB/s, 25MB cache).
6. Two E5-2697v4’s (Q1’16); 512 GB DDR4/2400; (each processor: 16 cores, 3.6 GHz turbo, 77 GB/s, 40MB cache); C612 chip set; SSD.
7. Four E5-4627v2’s (Q1’14); 512 GB DDR3/1866; (each processor: 8 cores, 3.6 GHz turbo, 77 GB/s, 16MB cache).

In the above list, each successive computer was significantly slower than the one immediately above it. Two years ago we also evaluated the NSL license (under 5.1), and all our attempts to get clusters and cloud computing to outperform the first computer listed above were unsuccessful. We also put extensive effort (with some input from COMSOL support) into trying to get the last two computers listed above to perform better, but to no avail.

What was most surprising was that computer 7 was at least 40% slower than computer 6, computer 6 was at least 40% slower than computer 5, and computer 5 was at least 40% slower than computer 4. The differences between computers 1, 2, 3, and 4 were in the 15-30% range. Note that each of the processors listed above (except the laptop) has 4 memory channels (doesn’t seem to mean much).

We recently repeated similar benchmark tests using COMSOL 5.3 on four of the computers listed above (computers 1, 4, 5, and 6). The new benchmark tests show no significant difference compared to 5.2. We understand there are some enhancements in 5.3 that will permit better performance after better optimization of the mesh and solver. We have not yet explored that.

We did compare PARDISO and MUMPS on several of these computers, and found MUMPS to be ~30% faster on computer 6. We tried various changes in the BIOS settings, tried with and without hyperthreading, tried disabling some cores, etc. None of these changes mattered much when running a single instance of COMSOL. Enabling hyperthreading did help some when running multiple instances.

We also often need to solve large complex rf problems, so we have also run some large rf problems on all of these computers. There, (with 5.2) we found the order of performance of these computers was very different, and closer to what one would expect based on commonly available computing power benchmarks. For moderate-sized CFD problems not involving rotating machinery and using the L_VEL turbulence model, the relative performance of the above computers fell somewhere between that for the k-w rotating CFD models and for the RF models.

(Very strange that COMSOL-CFD-Mixer k-w is often not able to run as fast on modern high-end computers as on old slower computers. Would really be great if COMSOL could put some serious effort into getting COMSOL-CFD-Mixer k-w to take better advantage of modern computers – and provide better guidance on computer recommendations!)

We would like to buy the fastest computer available (within reason) for our greatest current need – large complex COMSOL-CFD-Mixer problems (k-omega, frozen rotor, compressible flow, non-isothermal, transonic, high boundary pressure gradients, more than 1M meshcells, ~1000 boundaries, boundary meshes on most boundaries).

Based on our benchmark COMSOL-CFD-Mixer k-w tests on 7 computers, we are guessing that our best choice might be a computer with two Xeon gold 6146 processors with 512 GB DDR4/2666 (each processor: 12 cores, 4.2 GHz turbo, 25MB cache).

We have some other high-end software which we know can utilize more cores effectively, so for that software our best choice for a high-end computer would probably be one with four Xeon gold 6138 processors with 512 GB DDR4/2666 (each processor: 20 cores, 3.7 GHz turbo, 22MB cache). However, our previous experience with a four-socket computer (computer #7 in the above list) leads us to fear that this could be a very bad choice for large COMSOL-CFD-Mixer k-w models.

Before we consider shelling out ~$20K for a computer that might be no faster (maybe even slower) than an inexpensive computer we’ve had for more than 2 years, we would love to know if anyone out there has tried running a large COMSOL-CFD-Mixer k-w model on a machine with two or four Xeon gold-series processors, and if so, what motherboard and processors were used.

The Xeon gold-series processors are quite new, so it may be unlikely that anyone can be found who has had experience running a large COMSOL-CFD-Mixer k-w model on a machine using two or four of these processors. If anyone has experience running a large COMSOL-CFD-Mixer k-w model on a machine that may be faster than the fastest computer we’ve found (dual E5-2687v3’s), we’d love to hear about that too!

David Doty, PhD