NVIDIA introduces System Memory Fallback feature for Stable Diffusion

Published: Oct 31st 2023, 21:27 GMT   Comments

NVIDIA has a solution to Stable Diffusion maxing out the available memory

Drivers have a new feature called “System Memory Fallback”. 

The NVIDIA Control Panel may still look old, but it still continues to receive feature updates. In a recent driver release, NVIDIA introduced a new option in the Control Panel tailored for consumer GeForce graphics cards. This option is labeled as “CUDA – Sysmem Fallback Policy” and is specifically designed to address memory-intensive workloads, such as emerging generative AI programs.

This feature is primarily marketed for Stable Diffusion, a process that might demand up to 6GB of memory to be utilized simultaneously. However, since popular GPUs still feature maximum memory capacity between 6 and 8GB, the GPU memory might be still shared with other workloads preventing the full utilization by programs like Stable Diffusion. To address this, NVIDIA has implemented a new feature that prevents applications from exhausting GPU memory by efficiently switching to shared system memory.

NVIDIA Control Panel with Sysmem Fallback Policy, Source: VideoCardz

From a technical perspective, users have the flexibility to employ this feature for any application, including games. However, it may not be the best choice, as system memory is often notably slower than GPU memory, not to mention the necessary data transfer between the two.

The primary purpose of this feature is to avert potential crashes on graphics cards equipped with 6GB memory capacity. With the latest drivers, NVIDIA has thoughtfully incorporated an option to disable this fallback mechanism. Doing so can potentially enhance performance stability but does reintroduce the risk of depleting GPU memory.

How to enable or disable System Memory Fallback

  1. Open NVIDIA Control Panel
  2. Under 3D Settings, click Manage 3D Settings
  3. Navigate to Program Settings tab
  4. Select Stable Diffusion python executable from dropdown
  5. Click on CUDA – Sysmem Fallback Policy and select Driver Default, Prefer No Sysmem Fallback or Prefer Sysmem Fallback.
  6. Click Apply to confirm.
  7. Restart Stable Diffusion if it’s already open.

Source: NVIDIA

Many thanks to @HardwareBuddy for the tip!

Comment Policy
  1. Comments must be written in English and should not exceed 1000 characters.
  2. Comments deemed to be spam or solely promotional in nature will be deleted. Including a link to relevant content is permitted, but comments should be relevant to the post topic. Discussions about politics are not allowed on this website.
  3. Comments and usernames containing language or concepts that could be deemed offensive will be deleted.
  4. Comments complaining about the post subject or its source will be removed.
  5. A failure to comply with these rules will result in a warning and, in extreme cases, a ban. In addition, please note that comments that attack or harass an individual directly will result in a ban without warning.
  6. VideoCardz has never been sponsored by AMD, Intel, or NVIDIA. Users claiming otherwise will be banned.
  7. VideoCardz Moderating Team reserves the right to edit or delete any comments submitted to the site without notice.
  8. If you have any questions about the commenting policy, please let us know through the Contact Page.
Hide Comment Policy