Discussion Forum

First, if you want distributed parallel processing (DPP), then you must use the MUMPS direct solver. No other solver works in DPP. That is stated in the COMSOL documentation.

Second, if you have a single node, multiple cores can be used in shared-memory parallel processing with several solvers and also in other parts of the solution process such as meshing and finite-element assembly. There is essentially no constraint on the shared-memory mode of parallelism in COMSOL.

See my paper on the conference CD for some benchmarking of COMSOL v4-beta2 using DPP. I am still benchmarking with v4.1, but I can say it is at least as good as v4-beta2. Still limited to MUMPS, but it does work.

Danial, in your case where you only have a single node, but several cores using the same shared memory, then PARDISO will be best. It will run in parallel on all your cores. We have a similar machine and see much better than
your speedup on a problem large enough to fill the memory, but not use any disk swapping.

The latest result I have for distributed parallel processing on a multi-node cluster using both shared memory (per node) and distributed parallel processing we see a speedup of ~ 7.5 across 12 nodes and 96 cores. If we had more cores in our cluster, the speedup would be better because it has not rolled over yet. See attached plot. This cluster uses infiniband to communicate between nodes which helps tremendously. Also, across the nodes, the memory requirements are reduced significantly.

Danial, I cannot tell from the information you sent what the computer bus speed is. How many processors do you have that hold the 48 cores ? Do you have 48 physical cores ? On our linux box here, we have 6 cores on each of two processors for a total of 12 physical cores. However, the linux operating system takes advantage of the "hyperthreading" feature of Intel processors which appears to the operating system to look like 24 cores (doubles the number of cores to the operating system, even though there are only 12 physical cores). We have found that hyperthreading actually only give us about 30% of a physical core. So, the most we could theoretically gain with this system is a speedup of (12*1.3) or 15.6 not 24. It is better than nothing, but hyperthreading can be misleading if you don't know what you have in the box. Is your OS using hyperthreading ? Is the bios on the box set to enable hyperthreading ?

If you have 8 physical cores on a processor, that would be 6 processors in your box without hyperthreading.

If you have 6 cores on a processor, with 4 processors in the box and hyperthreading enabled, that would be 48 and that might explain at least part of your problem. This would be my guess. I have not seen a 48-physical-core single node before.

My best guess is the problem is too small to see the speed up. Try to get it to spend more time in the solver by filling up all the memory.

One suggestion to save some additional clock time. You might expand your solver scheme to use the segregated solver, while still using the PARDISO direct solver within each segregated step. You could first separate into two steps: 1) the u-p laminar flow (u,v,w,p) and 2) the remaining variables. This will likely take more iterations, but still save overall cpu/clock time. You may also be able to break down the 2nd step further in some cases.

Anyhow a good model, well optimised, should run for at most some seconds on a laptop, dont you think so ?

I think hyperthreading will improve the performance of COMSOL even though it is "not supported". To COMSOL, or any process for that matter running on a Linux system anyway, the OS takes care of the hyperthreading such that an n-core system appears like a 2n-core system to the users. I have found that each hyperthread core is equivalent to about 1/3 of a real core in terms of cpu speed improvement.

We run COMSOL on hyperthreaded machines routinely. For example, presently I have COMSOL running on a machine with 12 physical cores on it, but 24 total cores since it is hyperthreaded. The load factor periodically goes up to indicate that all 24 cores are being used and the turnaround time is likewise improved. I can start the COMSOL server with the "-np 24" switch enabled to use all the cores on that system.

I think hyperthreading will improve the performance of COMSOL even though it is "not supported". To COMSOL, or any process for that matter running on a Linux system anyway, the OS takes care of the hyperthreading such that an n-core system appears like a 2n-core system to the users. I have found that each hyperthread core is equivalent to about 1/3 of a real core in terms of cpu speed improvement.

We run COMSOL on hyperthreaded machines routinely. For example, presently I have COMSOL running on a machine with 12 physical cores on it, but 24 total cores since it is hyperthreaded. The load factor periodically goes up to indicate that all 24 cores are being used and the turnaround time is likewise improved. I can start the COMSOL server with the "-np 24" switch enabled to use all the cores on that system.