You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
This repository was archived by the owner on Oct 23, 2020. It is now read-only.
Repository navigation
This repository was archived by the owner on Oct 23, 2020. It is now read-only.
This is what I see. I am running from commit 2763249 on mark-petersen/ocean/smoothBLD_Luke_Phil_Todd. Identical runs with split explicit do not have this problem. I expect to see the same thing on MPAS-Dev/ocean/develop, but have not checked.
gr1112.localdomain> mpirun -n 576 ocean_model_smoothBLD_Luke_Phil_Todd
App launch reported: 16 (out of 16) daemons - 540 (out of 576) procs
Operating system error: Operating system error: Cannot allocate memory
Allocation would exceed memory limit
Operating system error: Cannot allocate memory
Allocation would exceed memory limit
Operating system error: Cannot allocate memory
Allocation would exceed memory limit
Operating system error: Cannot allocate memory
Allocation would exceed memory limit
Operating system error: Cannot allocate memory
Allocation would exceed memory limit
Operating system error: Cannot allocate memory
Allocation would exceed memory limit
Cannot allocate memory
Allocation would exceed memory limit
Operating system error: Operating system error: Cannot allocate memory
Allocation would exceed memory limit
Cannot allocate memory
Allocation would exceed memory limit
-------------------------------------------------------
Primary job terminated normally, but 1 process returned
a non-zero exit code.. Per user-direction, the job has been aborted.
-------------------------------------------------------
--------------------------------------------------------------------------
mpirun detected that one or more processes exited with non-zero status, thus causing
the job to be terminated. The first process to do so was:
Process name: [[46293,1],286]
Exit code: 1
--------------------------------------------------------------------------
gr1112.localdomain> pwd
/lustre/scratch2/turquoise/mpeterse/runs/c67n/forward
As I just discussed with @mark-petersen, this issue is holding up my work on wetting-and-drying under ice shelves. @pwolfram, something you should be aware of, too.
Activity
mark-petersen commentedon Jul 27, 2017
This is what I see. I am running from commit 2763249 on mark-petersen/ocean/smoothBLD_Luke_Phil_Todd. Identical runs with split explicit do not have this problem. I expect to see the same thing on MPAS-Dev/ocean/develop, but have not checked.
mark-petersen commentedon Jul 28, 2017
Note, this dies within 4 days on wolf, 256 cores. See
xylar commentedon Jan 22, 2019
As I just discussed with @mark-petersen, this issue is holding up my work on wetting-and-drying under ice shelves. @pwolfram, something you should be aware of, too.
pwolfram commentedon Jan 22, 2019
Thanks @xylar
mark-petersen commentedon Jan 22, 2019
Closing this issue to move conversation to the new MPAS-Model repo. See
MPAS-Dev/MPAS-Model#137