Discussion Closed This discussion was created more than 6 months ago and has been closed. To start a new discussion with a link back to this one, click here.
Tip: Batch calculations with Matlab + Comsol 4.0a
Posted 2010年9月9日 GMT-4 09:26 Interfacing, LiveLink for MATLAB Version 5.2a 24 Replies
Please login with a confirmed email address before reporting spam
Hi,
I just wanted to share my experience regarding batch-type (i.e., non-interactive) calculations using comsol 4.0a and the matlab livelink. I lost quite some time trying to run a comsol matlab script (.m file) on a linux cluster, and I thought some people might be interested in hearing the workaround I found.
When working with the cluster, I have two major constraints:
1. I do not have any kind of display (X) support,
2. I have to submit the calculation via a simple bash script.
In 3.5a, running a .m file in batch mode was very easy (At the very least, it was nicely documented : www.comsol.com/support/knowledgebase/980/), One would use:
comsol matlab -ml -nodesktop -ml -nosplash -mlr "myscript, exit"
In comsol 4.0a, using something along the lines of
comsol server matlab -nodesktop -nosplash -r myfile
does not lead to similar behavior. First, an xterm is spawned, and the 'nodesktop' matlab runs in this new terminal. The '-r myfile' option is ignored, and matlab does not run my .m file on startup.
To work around the fact that 'comsol server matlab' spawns an xterm, I was told by comsol support to do the following:
1. Start the comsol server
2. Start matlab, and connect to the server using mphstart
To be able to do this within one bash script, I basically had to launch the server in the background. However, I could not use
comsol server &
matlab -nodesktop -nosplash -r myfile
as the server process would stop before matlab could connect to it whenever it was launched as a background process. By using instead
comsol server </dev/null > /dev/null &
matlab -nodesktop -nosplash -r myfile
instead of simply "comsol server &" then the server process is not stopped and you can run matlab, connect to comsol server, and run your .m script, without any user-intervention nor need for X support. Also, the comsol server exits when running ModelUtil.disconnect in Matlab, so I don't leave any process on the compute node..
Hoping this might be useful for someone ;)
By the way, if anyone has a better (cleaner?) way of achieving the same thing, I'd be happy to hear it!
Charles
I just wanted to share my experience regarding batch-type (i.e., non-interactive) calculations using comsol 4.0a and the matlab livelink. I lost quite some time trying to run a comsol matlab script (.m file) on a linux cluster, and I thought some people might be interested in hearing the workaround I found.
When working with the cluster, I have two major constraints:
1. I do not have any kind of display (X) support,
2. I have to submit the calculation via a simple bash script.
In 3.5a, running a .m file in batch mode was very easy (At the very least, it was nicely documented : www.comsol.com/support/knowledgebase/980/), One would use:
comsol matlab -ml -nodesktop -ml -nosplash -mlr "myscript, exit"
In comsol 4.0a, using something along the lines of
comsol server matlab -nodesktop -nosplash -r myfile
does not lead to similar behavior. First, an xterm is spawned, and the 'nodesktop' matlab runs in this new terminal. The '-r myfile' option is ignored, and matlab does not run my .m file on startup.
To work around the fact that 'comsol server matlab' spawns an xterm, I was told by comsol support to do the following:
1. Start the comsol server
2. Start matlab, and connect to the server using mphstart
To be able to do this within one bash script, I basically had to launch the server in the background. However, I could not use
comsol server &
matlab -nodesktop -nosplash -r myfile
as the server process would stop before matlab could connect to it whenever it was launched as a background process. By using instead
comsol server </dev/null > /dev/null &
matlab -nodesktop -nosplash -r myfile
instead of simply "comsol server &" then the server process is not stopped and you can run matlab, connect to comsol server, and run your .m script, without any user-intervention nor need for X support. Also, the comsol server exits when running ModelUtil.disconnect in Matlab, so I don't leave any process on the compute node..
Hoping this might be useful for someone ;)
By the way, if anyone has a better (cleaner?) way of achieving the same thing, I'd be happy to hear it!
Charles
24 Replies Last Post 2017年4月28日 GMT-4 05:52
Please login with a confirmed email address before reporting spam
Posted:
1 decade ago
Hi
thanks for the tips, I start too to have large models I would like to run batch over the night/week-end so this will be usefull.
Hope COMSOL will make it cleaner/easier in next release
--
Good luck
Ivar
thanks for the tips, I start too to have large models I would like to run batch over the night/week-end so this will be usefull.
Hope COMSOL will make it cleaner/easier in next release
--
Good luck
Ivar
Please login with a confirmed email address before reporting spam
Posted:
1 decade ago
This is a very good tip, i just need it to solve my problem, but as i tried as you suggested.
the command
$comsol server < /dev/null > /dev/null &
does work, while after running this command, you can see there is a 'comsollauncher' process
running in the background, also consuming quite a lot of cpu, though nothing is running at all.
then i run the command
$matlab -nodesktop -nosplash -r file
it gives me an error like
'??? Undefined function or variable 'ModelUtil'.'
Do you guys know what's wrong with it?
Thanks in advance.
the command
$comsol server < /dev/null > /dev/null &
does work, while after running this command, you can see there is a 'comsollauncher' process
running in the background, also consuming quite a lot of cpu, though nothing is running at all.
then i run the command
$matlab -nodesktop -nosplash -r file
it gives me an error like
'??? Undefined function or variable 'ModelUtil'.'
Do you guys know what's wrong with it?
Thanks in advance.
Please login with a confirmed email address before reporting spam
Posted:
1 decade ago
Make sure that you have your COMSOL40a\mli directory added to the Matlab path.
Please login with a confirmed email address before reporting spam
Posted:
1 decade ago
This is a great tip!
Perhaps one step better, for people running OSX - I've saved the whole thing into it's own symbolic link.
Navigate to /usr/bin, create a file with the text editor of your choice (I used nano, for simplicity's sake). In the file, put in:
#!/bin/bash
comsol server </dev/null> /dev/null &
matlab -nodesktop -nosplash -r 'mphstart'
If it gives you trouble, you might have to use a sudo command to get it to create (as in sudo nano shortcutname).
Finally, you need to give everyone access to run this program, so back in the /usr/bin directory,
chmod +x /usr/bin/shortcutname
Perhaps one step better, for people running OSX - I've saved the whole thing into it's own symbolic link.
Navigate to /usr/bin, create a file with the text editor of your choice (I used nano, for simplicity's sake). In the file, put in:
#!/bin/bash
comsol server </dev/null> /dev/null &
matlab -nodesktop -nosplash -r 'mphstart'
If it gives you trouble, you might have to use a sudo command to get it to create (as in sudo nano shortcutname).
Finally, you need to give everyone access to run this program, so back in the /usr/bin directory,
chmod +x /usr/bin/shortcutname
Please login with a confirmed email address before reporting spam
Posted:
1 decade ago
I won't say it is a cleaner way, but it is more 'transparent' so to say for many people to start comsol's server like this:
comsol server > server.txt &
This way whatever output comsol server wants to spew will be dumped in server.txt file. And honestly, sometime during debugging it is useful to know what it has to say :)
comsol server > server.txt &
This way whatever output comsol server wants to spew will be dumped in server.txt file. And honestly, sometime during debugging it is useful to know what it has to say :)
Please login with a confirmed email address before reporting spam
Posted:
1 decade ago
Good tip, that also works for Comsol 4.1
But nevertheless: very inconvenient to be honest..
Hopefully there is a better solution in the next version.
But nevertheless: very inconvenient to be honest..
Hopefully there is a better solution in the next version.
Please login with a confirmed email address before reporting spam
Posted:
1 decade ago
Thank you very much, Charles!
I've been trying to run Comsol 4.0a in batch mode on our cluster for like a year (periodically dropping it and coming back to it again and involving our cluster and comsol tech support with no results). I've just tested what you suggested - your method works nicely! Finally!!!
Sincerely,
Alex.
I've been trying to run Comsol 4.0a in batch mode on our cluster for like a year (periodically dropping it and coming back to it again and involving our cluster and comsol tech support with no results). I've just tested what you suggested - your method works nicely! Finally!!!
Sincerely,
Alex.
Please login with a confirmed email address before reporting spam
Posted:
1 decade ago
Updated:
7 years ago
Hi Charles,
Do you have any idea of running batch process on windows?
I have two files and I want to run them one by one in the way that results of one should be used as initial conditions for the other, is there any way of doing it?
Thanks in advance for any of your help!
Do you have any idea of running batch process on windows?
I have two files and I want to run them one by one in the way that results of one should be used as initial conditions for the other, is there any way of doing it?
Thanks in advance for any of your help!
Please login with a confirmed email address before reporting spam
Posted:
1 decade ago
Hi Muhammad,
Sorry, I cannot really help you with running batch calculations under Windows. I mainly use my employer-issued Windows laptop as a glorified email client, and therefore don't have much experience with Comsol under Windows.
Regarding the second part of your question; could you not simply merge the two files into one, using the second file as a second study step? In that case it would be straightforward to setup the calculation in such a way that the results of the first calculation can be used as initial conditions for the second calculation..
Good luck,
Charles
Sorry, I cannot really help you with running batch calculations under Windows. I mainly use my employer-issued Windows laptop as a glorified email client, and therefore don't have much experience with Comsol under Windows.
Regarding the second part of your question; could you not simply merge the two files into one, using the second file as a second study step? In that case it would be straightforward to setup the calculation in such a way that the results of the first calculation can be used as initial conditions for the second calculation..
Good luck,
Charles
Please login with a confirmed email address before reporting spam
Posted:
1 decade ago
Thanks a lot Charles for your feedback!
I have defined the second study step in the same file but could not find any option in the second study to set the results of first study as initial values.(Please see the attached image for the options)
Do you mean that I need to define the whole model again?
Can you please elaborate what do you mean by setting the second file as second study step?
Thanks in advance!
I have defined the second study step in the same file but could not find any option in the second study to set the results of first study as initial values.(Please see the attached image for the options)
Do you mean that I need to define the whole model again?
Can you please elaborate what do you mean by setting the second file as second study step?
Thanks in advance!
Attachments:
Please login with a confirmed email address before reporting spam
Posted:
1 decade ago
The above worked for me on 4.2a. Useful tips, many thanks. Saved me a wasting an afternoon.
Please login with a confirmed email address before reporting spam
Posted:
1 decade ago
Hi, I hope this thread is still active.
I am having big troubles in implementing your suggestions. I prepared a run_script.sh file with:
#!/bin/sh
comsol server </dev/null > /dev/null &
bsub matlab -nodesktop -nosplash -r "addpath(genpath('/cluster/apps/comsol/4.4/mli')); mphstart; study1"
wher study 1 is the fiel that I actually need to run. Then I submit the job on the cluster I have access to with
bsub < run_script.sh
but I get an error as if I didn't start the comsol server:
Error using mphstart (line 152)
A connection to Comsol could not be established. Please check that a Comsol
Server is started prior to calling mphstart
How do I fix this?
I am having big troubles in implementing your suggestions. I prepared a run_script.sh file with:
#!/bin/sh
comsol server </dev/null > /dev/null &
bsub matlab -nodesktop -nosplash -r "addpath(genpath('/cluster/apps/comsol/4.4/mli')); mphstart; study1"
wher study 1 is the fiel that I actually need to run. Then I submit the job on the cluster I have access to with
bsub < run_script.sh
but I get an error as if I didn't start the comsol server:
Error using mphstart (line 152)
A connection to Comsol could not be established. Please check that a Comsol
Server is started prior to calling mphstart
How do I fix this?
This is a great tip!
Perhaps one step better, for people running OSX - I've saved the whole thing into it's own symbolic link.
Navigate to /usr/bin, create a file with the text editor of your choice (I used nano, for simplicity's sake). In the file, put in:
#!/bin/bash
comsol server </dev/null> /dev/null &
matlab -nodesktop -nosplash -r 'mphstart'
If it gives you trouble, you might have to use a sudo command to get it to create (as in sudo nano shortcutname).
Finally, you need to give everyone access to run this program, so back in the /usr/bin directory,
chmod +x /usr/bin/shortcutname
Please login with a confirmed email address before reporting spam
Posted:
1 decade ago
Hi Stefano
Just to be sure, did you start the comsol server at least once on the node? The first time you start it, you need to (interactively ) setup a username/password. As far as I know, this cannot be done directly using a script. Luckily, you only need to do it once..
Charles
Just to be sure, did you start the comsol server at least once on the node? The first time you start it, you need to (interactively ) setup a username/password. As far as I know, this cannot be done directly using a script. Luckily, you only need to do it once..
Charles
Please login with a confirmed email address before reporting spam
Posted:
1 decade ago
actually, problem solved. There was a mistake in my file.
Thanks for the answer anyway.
Thanks for the answer anyway.
Please login with a confirmed email address before reporting spam
Posted:
1 decade ago
Hi Stefano,
does your script also work when it is started multiple times? I'm wondering if this is possible without setting a distinct port in order to connect an instance of MATLAB with a certain Comsol server.
does your script also work when it is started multiple times? I'm wondering if this is possible without setting a distinct port in order to connect an instance of MATLAB with a certain Comsol server.
Please login with a confirmed email address before reporting spam
Posted:
1 decade ago
Hi Eli,
I haven't being launching simulations from matlab in a while, so I had to go back to old scripts :)
I simply launched the following script
#!/bin/sh
comsol server </dev/null > /dev/null &
env > $HOME/env.log
matlab -nodesktop -nosplash -r "disp('start'); addpath(genpath('/cluster/apps/comsol/4.4/mli')); mphstart; disp('done')" > mphstart.log
So, it is just opening a connection with the server and just displaying something.
I launched 2 of them and it seems to work.
I haven't being launching simulations from matlab in a while, so I had to go back to old scripts :)
I simply launched the following script
#!/bin/sh
comsol server </dev/null > /dev/null &
env > $HOME/env.log
matlab -nodesktop -nosplash -r "disp('start'); addpath(genpath('/cluster/apps/comsol/4.4/mli')); mphstart; disp('done')" > mphstart.log
So, it is just opening a connection with the server and just displaying something.
I launched 2 of them and it seems to work.
Please login with a confirmed email address before reporting spam
Posted:
1 decade ago
Hi,
thank you fopr providing this. I use something similar, but because I want to run many models in parallel, there are lots of MATLAB instances and Comsol servers running at the same time. In order to make MATLAB connect to a distinct Comsol server I need to provide a port number when starting up the server:
comsol server -port 2037 </dev/null > /dev/null &
And then connect MATLAB to this Comsol server:
matlab -nodesktop -nosplash -r " addpath(genpath('path_to_mli')); mphstart(2037); someFunction;exit;" > matlab_output.txt
I guess without the port number MATLAB would connect to some random Comsol server.
thank you fopr providing this. I use something similar, but because I want to run many models in parallel, there are lots of MATLAB instances and Comsol servers running at the same time. In order to make MATLAB connect to a distinct Comsol server I need to provide a port number when starting up the server:
comsol server -port 2037 </dev/null > /dev/null &
And then connect MATLAB to this Comsol server:
matlab -nodesktop -nosplash -r " addpath(genpath('path_to_mli')); mphstart(2037); someFunction;exit;" > matlab_output.txt
I guess without the port number MATLAB would connect to some random Comsol server.
Please login with a confirmed email address before reporting spam
Posted:
1 decade ago
If you don't specify the port number Comsol will create a server with the default port number 2036. If that's already busy It will use 2037 and so on. I am not specifying the port number and that seems to work even in batch mode.
Your method is also fine, I guess you need to specify a different port number for each job? But I think that's not difficult to implement.
Your method is also fine, I guess you need to specify a different port number for each job? But I think that's not difficult to implement.
Please login with a confirmed email address before reporting spam
Posted:
1 decade ago
If I open several instances of MATLAB and Comsol server in a sequence without giving MATLAB a distinct port to connect to, it might happen that e.g. two MATLABs connect to the same Comsol server. This would produce errors as soon as both want to run models on that server at the same time. Of course if there are only two MATLAB-Comsol pairs opened this double connect to one server is not liekly to happen.
Please login with a confirmed email address before reporting spam
Posted:
10 years ago
Hi,
I am not sure whether the post is still active. I tried the following in out linux server:
#!/bin/sh
comsol server </dev/null >/dev/null &
matlab -nodesktop -nosplash -r "disp('start');addpath(genpath('/apps/comsol4.4/mli'));mphstart;disp('done');exit;" > OUT.txt
These lines are saved in a file named run1.s and then I run the file using the command
nohup sh ./run1.s &
I receive the error: Suspended (tty output) sh ./run1.s
I tried using the following two lines in command line without using a shell script and it works fine:
comsol server </dev/null >/dev/null &
matlab -nodesktop -nosplash -r "disp('start');addpath(genpath('/apps/comsol4.4/mli'));mphstart;disp('done');exit;" > OUT.txt
Can anyone help?
I am not sure whether the post is still active. I tried the following in out linux server:
#!/bin/sh
comsol server </dev/null >/dev/null &
matlab -nodesktop -nosplash -r "disp('start');addpath(genpath('/apps/comsol4.4/mli'));mphstart;disp('done');exit;" > OUT.txt
These lines are saved in a file named run1.s and then I run the file using the command
nohup sh ./run1.s &
I receive the error: Suspended (tty output) sh ./run1.s
I tried using the following two lines in command line without using a shell script and it works fine:
comsol server </dev/null >/dev/null &
matlab -nodesktop -nosplash -r "disp('start');addpath(genpath('/apps/comsol4.4/mli'));mphstart;disp('done');exit;" > OUT.txt
Can anyone help?
Please login with a confirmed email address before reporting spam
Posted:
9 years ago
Things not working for me
I was expecting to see some messages showing the server is starting up. I waited about 3min but nothing showed up. Then I proceeded adding a wait time
$sleep 120
and the run
After matlab starts, I ran
>> addpath((genpath(<path/to/comsol44/mli>))
>>mphstart
And matlab complained with
"A connection to Comsol could not be established. Please check that a Comsol
Server is started prior to calling mphstart"
Something might have gone wrong during the server startup but how to I correct it?
I wonder what exactly this means
should I be replacing </dev/null> with some other path? I am confused
comsol server </dev/null > /dev/null &
I was expecting to see some messages showing the server is starting up. I waited about 3min but nothing showed up. Then I proceeded adding a wait time
$sleep 120
and the run
matlab -nodesktop -nosplash -r myfile
After matlab starts, I ran
>> addpath((genpath(<path/to/comsol44/mli>))
>>mphstart
And matlab complained with
"A connection to Comsol could not be established. Please check that a Comsol
Server is started prior to calling mphstart"
Something might have gone wrong during the server startup but how to I correct it?
I wonder what exactly this means
</dev/null > /dev/null
should I be replacing </dev/null> with some other path? I am confused
Please login with a confirmed email address before reporting spam
Posted:
9 years ago
Hi Camille,
'> /dev/null &' sends all output of the starting Comsol server to /dev/null which means it will not be visible to you, the process is then send to background (via '&') so that the terminal is free to proceed to the next command for starting MATLAB. A better way is it to send the output to a log-file using 'Comsol server > /dev/null 2>&1 > serverLog.txt &'.
Alternatively you may also just start Comsol server in one terminal window and then open another one and start MATLAB from there. This also enables you to monitor the output of Comsol server. It should show you when a MATLAB instance connects.
Best,
Eli
'> /dev/null &' sends all output of the starting Comsol server to /dev/null which means it will not be visible to you, the process is then send to background (via '&') so that the terminal is free to proceed to the next command for starting MATLAB. A better way is it to send the output to a log-file using 'Comsol server > /dev/null 2>&1 > serverLog.txt &'.
Alternatively you may also just start Comsol server in one terminal window and then open another one and start MATLAB from there. This also enables you to monitor the output of Comsol server. It should show you when a MATLAB instance connects.
Best,
Eli
Please login with a confirmed email address before reporting spam
Posted:
8 years ago
Thanks Charles Doiron! Works on 5.2a
Please login with a confirmed email address before reporting spam
Posted:
8 years ago
This post was extremly useful, thank you very much. However I'd like to add something, that will be probably mostly useful for students like me, who don't have superuser rights (and might not be big linux gurus either.)
A lot of times if Comsol Server is started with e.g. "comsol-52a server &", it just exits, and no chance to connect to it with MATLAB. (If the log is redirected to a txt, one can see that it waits for username and password, and it exits, as none is provided.) So what I suggest is opening up a screen, start a Comsol Server there, with e.g. "comsol-52a server", and look at the output: a lot of times the port it connects to is not 2036, but 2037, 2038, etc. You're free to detach from the screen now.
Now, open up a new screen, start MATLAB, without X forwarding enabled, it then displays its command promt. (And does not exit with an error, fortunately.) Add the "mli" library to the MATLAB path, like "addpath(''/usr/local/comsol52a/multiphysics/mli')" and now run "mphstart", but also specify the port number used: "mphstart(2037)" (if localhost:2037 was the port number displayed by Comsol Server previously.) Without passing the port number as an argument, it will just throw an error, because it always tries to connect to port 2036 by default. Now you can start any Comsol script and detach from the screen, it will keep running.
In case of other problems, it is worth opening the "mphstart.m" file in the "mli" library previously added, it lists a lot of other arguments "mphstart" is able to receive.
In case somebody can suggest a nicer solution, e.g. without using two screens, please share.
A lot of times if Comsol Server is started with e.g. "comsol-52a server &", it just exits, and no chance to connect to it with MATLAB. (If the log is redirected to a txt, one can see that it waits for username and password, and it exits, as none is provided.) So what I suggest is opening up a screen, start a Comsol Server there, with e.g. "comsol-52a server", and look at the output: a lot of times the port it connects to is not 2036, but 2037, 2038, etc. You're free to detach from the screen now.
Now, open up a new screen, start MATLAB, without X forwarding enabled, it then displays its command promt. (And does not exit with an error, fortunately.) Add the "mli" library to the MATLAB path, like "addpath(''/usr/local/comsol52a/multiphysics/mli')" and now run "mphstart", but also specify the port number used: "mphstart(2037)" (if localhost:2037 was the port number displayed by Comsol Server previously.) Without passing the port number as an argument, it will just throw an error, because it always tries to connect to port 2036 by default. Now you can start any Comsol script and detach from the screen, it will keep running.
In case of other problems, it is worth opening the "mphstart.m" file in the "mli" library previously added, it lists a lot of other arguments "mphstart" is able to receive.
In case somebody can suggest a nicer solution, e.g. without using two screens, please share.