3.4.1 Managing memory and disk use in Abaqus
Products: Abaqus/Standard Abaqus/Explicit
References
Overview
For small analyses management of computer resources is generally of secondary concern, but with large models intelligent use of disk and memory resources is a critical part of the analysis process. For moderate to large analyses you will find it necessary to modify resource management settings.
Understanding resource use
For Abaqus disk and memory are effectively two similar means of storing data. Data that will be required after an analysis completes must eventually be written to disk; but during an analysis, disk and memory provide functionally equivalent storage mechanisms. Typically disk is a more abundant resource, while memory provides faster access to stored data. Management of Abaqus resources hinges on this simple trade-off.
Abaqus data
There are effectively two types of data generated by an Abaqus analysis. The first is “output” data that must persist after an analysis is complete. Output data are typically either results that you require for postprocessing or data that are necessary to restart an analysis. As mentioned above, these data must be stored on disk before an analysis completes.
In addition, an analysis generates a considerable amount of “scratch” or temporary data. These are data that are needed only while an analysis is running. The scratch data can be subdivided into two groups: performance-critical data and generic data. The performance-critical data are always stored in memory, while the generic data can be stored either in memory or on disk.
Requirements and considerations
To run an analysis, the following requirements must be satisfied:
There must be sufficient disk space available to hold the requested output data.
There must be sufficient memory available to hold all performance-critical data.
There must be sufficient disk space or memory resource available to hold all generic scratch data.
Typically Abaqus/Standard allocates a large portion of the available system memory on a machine during the analysis phase, but you can manually specify a limit for memory usage with the memory parameter (see “Resource management parameters” below). No scratch data are written to disk during the Abaqus/Explicit analysis phase, since the majority of scratch data are performance critical.
Resource management parameters
Abaqus resource management parameters fall into two classes: memory management and disk management. Each can be adjusted through one environment file parameter. The following sections explain how to best make use of this parameter. For information about the environment file, see “Using the Abaqus environment settings,” Section 3.3.1.
Memory management parameters
The memory parameter is used to limit the amount of memory that can be used during the analysis phase of Abaqus/Standard and during the input file processing phase, which is executed before both Abaqus/Standard and Abaqus/Explicit analyses.
If you do not define the memory parameter, Abaqus automatically detects the physical memory on the machine and allocates a percentage of this available memory. The default percentages are platform specific, but they typically represent a large portion of the available physical memory. For details on the default memory allocation settings, refer to the Dassault Systèmes Knowledge Base at www.3ds.com/support/knowledge-base.
You can override the default memory allocation by specifying the percent of physical memory or by specifying an absolute limit in units of megabytes or gigabytes. Percentages are indicated by a “%” sign following the specified limit. Units of megabytes and gigabytes are indicated by “mb” or “gb” following the specified limit. If no units are specified, megabytes are assumed. For example, with any of the following settings:
memory="2048 mb" memory="2 gb" memory="25 %"Abaqus uses up to 2 gigabytes of memory on a machine with 8 gigabytes physical memory. The memory setting value must be surrounded by quotes. The values specified for memory must be reasonable for the machine being used. Abaqus/Standard does not check the validity of the numerical values. To be consistent with operating system memory measurement tools, a megabyte is defined by Abaqus to be 1,048,576 bytes, not 1,000,000 bytes. A similar rule applies to the unit of gigabyte.
There are no memory management parameters for the Abaqus/Explicit analysis phase, since no scratch data are written to disk during this phase.
Environment file parameters can be set for a host, for a user, or for a particular job (see “Using the Abaqus environment settings,” Section 3.3.1, for further discussion). Because a default memory setting that works well for one machine with a large amount of memory may not be ideal for another machine that has less memory, it may be useful to vary the default memory settings by machine.
Disk management parameters
Management of output data is discussed in detail in “Output,” Section 4.1.1. Output data are written to files in the directory from which you launched the job.
Abaqus/Standard scratch files are written to a separate scratch directory. You can control the directory used to hold the scratch files with the scratch environment file parameter. Due to the frequent access of the scratch data throughout the analysis phase, ensuring high I/O speed of the scratch disks is essential to the analysis performance.
As explained above, no scratch data are written to disk for Abaqus/Explicit, so you have to be concerned only with proper management of output data.
Input file processing and data check
In general, the amount of memory required during input file processing is not large. The amount of memory and disk space needed for the analysis phase of a job is more likely to be a concern. It is not possible for Abaqus to estimate the amount of memory that will be required to complete input file processing. A data check run can be performed by using the datacheck parameter in the command for running Abaqus (see “Abaqus/Standard, Abaqus/Explicit, and Abaqus/CFD execution,” Section 3.2.2) to obtain an estimate of the required memory for completing the analysis phase. General guidelines for setting the memory parameter for performing the data check (which includes the input file processing phase) are given below.
Guidelines for memory settings
You will usually not have to change the default memory setting. If a job fails as a result of insufficient memory with the default setting, you will need to find a machine with more memory to run the job. If you need to override the default behavior by specifying a value for the memory environment file parameter, Table 3.4.1–1 lists some typical data check memory settings for problems of various sizes. The actual values required for memory may vary considerably from problem to problem depending on the features used in a model.
Abaqus/Standard analysis
Depending on the execution environment and typical job sizes run on the machine, memory can be set by machine or by job. More detailed guidelines are provided in the following section. When setting memory by job is needed, you are advised to run a data check analysis and set memory based on the memory estimates. These estimates are written to the printed output (.dat) file in a table under the heading “MEMORY ESTIMATE.” Two columns in this table are relevant to memory use. The first relevant column is labeled “MINIMUM MEMORY REQUIRED” and specifies the memory setting that is needed to hold critical scratch data in memory. An attempt to run the analysis with memory set below this value will result in a warning, and the job is not likely to run to completion due to the insufficient memory. The second relevant column is labeled “MEMORY TO MINIMIZE I/O” and specifies the memory that is required to hold all scratch data, both critical and generic, in memory. If the memory specified by memory is larger than the “MINIMUM MEMORY REQUIRED,” Abaqus/Standard automatically uses the additional memory up to the memory limit to improve speed of access to generic scratch data that would otherwise be written to disk. When the memory is not enough to hold all the generic scratch data in memory, Abaqus/Standard decides which data should be written to disk and which should be kept in memory based on their relative importance with respect to their effect on the analysis performance. Therefore, the actual disk space used by the scratch data can vary from very close to zero to the “MEMORY TO MINIMIZE I/O” depending on the memory setting. The memory setting can be changed in an analysis continued from a data check without the need of rerunning the analysis input file processor.
Guidelines for memory settings
The memory parameter allows you to specify the memory limit that can be used by Abaqus during the input file processing and analysis phases. You can specify the setting that should generally be available to Abaqus on a particular machine in the host environment file. Settings can be modified as necessary for individual jobs in job-specific environment files. Reasonable settings for a particular machine depend on the size of the problems being run and how the machine is being used in addition to the physical memory available on the machine. You should be aware of the difference between physical and virtual memory. When virtual memory is used, a machine's operating system simply uses disk for additional memory. While this can be useful, memory access may require I/O operations that add a considerable performance penalty. Therefore, the guidelines below for managing memory in Abaqus/Standard are always given relative to the physical memory on a machine. Virtual memory should be used only when necessary and with awareness of the associated performance penalty.
Setting memory on single-user machines
For a single-user machine that is dedicated to running Abaqus/Standard, using the default setting of memory is sensible. If the estimates indicate that the job requires more than this value, the job is too large to run efficiently on this machine. At this point you are urged to move the analysis to another machine with more memory resources.
For a single-user machine that is used to run both Abaqus/Standard and other applications simultaneously, setting a lower memory limit makes sense. If an analysis requires more than the specified value, you can decide to increase memory and continue the job. However, Abaqus/Standard will have to contend with the other applications for memory, which will impair the efficiency of both Abaqus/Standard and the other applications. If the other applications are interactive, the performance degradation could be problematic. In such a case you might decide to delay continuing the analysis until the machine can be dedicated to running Abaqus/Standard alone.
Setting memory on multi-user machines
The guidelines for setting memory on a multi-user machine are very similar to those for single-user machines, except that a judgement must be made as to the amount of memory that each user on the machine can expect to have for a single analysis. A reasonable approach might be to divide the machine's physical memory by the number of expected simultaneous jobs. Another sensible approach is to divide the machine's physical memory by the total number of CPUs and then multiply by the number of CPUs used for the current job. If the memory requirement among the simultaneous jobs is not even, you might want to divide the machine's physical memory in an uneven way accordingly. In general, to ensure acceptable performance, users on multi-user machines need to coordinate with each other to properly set the memory limit.
Setting memory when using queues
Often queues have an associated memory limit, and determining the appropriate queue for a job requires some judgement. You are advised to run a data check analysis and select a queue based on the estimates provided in the printed output file. However, for large analyses even a data check analysis can require a large amount of memory. Choosing an appropriate queue for a data check analysis requires some experience with particular classes of problems. You may want to submit data check runs initially to queues with very large memory limits to get the necessary estimates. An appropriate queue can then be chosen to actually run the job. If the jobs are to be submitted to shared memory machines, it makes sense to set memory to about 90% of the memory limit for the queue. If the jobs are to be submitted to computer clusters, it is reasonable to use the default memory setting.