Technical Developer: Out of memory? Easy ways to increase the memory available to your program

When you run your VB or C# application, you might get an OutOfMemoryException thrown, even if your machine has lots of memory.

Every 32 bit process has a 2^32 bit (4 Gig) address space. That means every pointer has a size of 32 bits (4 bytes) and thus is limited to 4 Billion.

That’s the equivalent of saying a vehicle license plate number consists of 6 digits and thus there are 1 million possible numbers.

That 4 Gigs is divided into half: the user application gets the lower half and the OS gets the upper. (This boundary can be changed: see below).

Start VS 2010. File->New->Project->VB or C# Windows WPF Application.

Paste the VB or C# code below. It creates a heap then allocates 100Meg of memory in a loop continuously until an exception is thrown.

On my 64 bit Windows 7 machine with 8 Gigs of RAM (your digital camera or phone might have more memory!), I get about 1.4Gig allocated before it dies.

Iter #10 1,048,576,000

Iter #11 1,153,433,600

Iter #12 1,258,291,200

Iter #13 1,363,148,800

Exception Exception of type 'System.OutOfMemoryException' was thrown.

Now choose Project->Properties->Compile->Build Events->PostBuildEvent Command and added these 2 lines

call "$(DevEnvDir)..\..\vc\vcvarsall.bat" x86

"$(DevEnvDir)..\..\vc\bin\EditBin.exe" "$(TargetPath)" /LARGEADDRESSAWARE

Note: the positions of the quotes are critical

The first line calls a BAT file that makes various tools available on the path.

The second runs EditBin on the target binary, using the LARGEADDRESSAWARE flag (that’s almost all left hand keys on the keyboard!)

Also uncheck the option: Project->Properties->Debug->Enable the Visual Studio Hosting Process

The only effect of these 2 lines is to call EditBin to toggle a bit in the EXE. When the EXE starts a process, that entire process is flagged as able to work with pointers above 2G.

With such pointers, the high bit is a 1, which, in 2’s complement notation, is a negative number, and some applications may not be designed to work with “negative” pointers.

Now when I run the code I get 3.5 Gigs: More than twice as much memory!

Iter #30 3,145,728,000

Iter #31 3,250,585,600

Iter #32 3,355,443,200

Iter #33 3,460,300,800

Exception Exception of type 'System.OutOfMemoryException' was thrown.

This Editbin “trick” works fine on a 64bit OS. For a 32bit OS it works too, but you need to "bcdedit /set IncreaseUserVA 3072" (reboot) and you won’t get as much extra memory.

Want even more memory? If you’re on a 64 bit OS, try compiling to 64 bit:

VB: Project->Properties->Compile->Advanced->Target CPU->Any CPU (or x64)

C#: Project->Properies->Build->Platform Target->Any CPU (or x64)

Iter #110 11,534,336,000

Iter #111 11,639,193,600

Iter #112 11,744,051,200

Iter #113 11,848,908,800

Iter #114 11,953,766,400

Exception Exception of type 'System.OutOfMemoryException' was thrown.

Yes, that really is almost 12 gigs on my 8 gig machine! You can also verify via Task Manager.

A 64 bit process has a pointer of size 64 bits (8 bytes). 2 ^ 64= 18,000,000,000,000,000,000. (1.8 x 10^19)

That’s about 18 exabytes, or 900,000 years of DVD quality video.

Still that’s nowhere near the number of possible chess games or even the waaaay smaller number of electrons that will fit in the universe J(Hubble's law, big bang physics)

When I first worked for Microsoft, I developed a thunking mechanism to allow 32 bit applications to call 16 bit DLLs. (http://support.microsoft.com/kb/139070 )

We thought there’s no way anybody could run out of 32 bit address space!

If a license plate number were to double its length from 6 to 12 digits, that would go from 1 Million (population of a medium sized city) to 1Trillion (1x 10^12) , 200 times the world population.

See also

BCDEdit Command-Line Options

Using multiple heaps efficiently

What is your computer doing with all that memory? Write your own memory browser

LARGEADDRESSAWARE

Create your own Test Host using XAML to run your unit tests

Undocumented APIs and 16 bit DLLs

<C# Code>

using System;

using System.Collections.Generic;

using System.Linq;

using System.Text;

using System.Windows;

using System.Windows.Controls;

using System.Windows.Data;

using System.Windows.Documents;

using System.Windows.Input;

using System.Windows.Media;

using System.Windows.Media.Imaging;

using System.Windows.Navigation;

using System.Windows.Shapes;

using System.Runtime.InteropServices;

using System.Diagnostics;

namespace Heapcs

{

/// <summary>

/// Interaction logic for MainWindow.xaml

/// </summary>

public partial class MainWindow : Window

{

public MainWindow()

{

InitializeComponent();

}

private void Window_Loaded(object sender, RoutedEventArgs e)

{

var hHeap = Heap.HeapCreate(Heap.HeapFlags.HEAP_GENERATE_EXCEPTIONS, 0, 0);

// if the FriendlyName is "heap.vshost.exe" then it's using the VS Hosting Process and not "Heap.Exe"

Trace.WriteLine(AppDomain.CurrentDomain.FriendlyName + " heap created");

uint nSize = 100 * 1024 * 1024;

ulong nTot = 0;

try

{

for (int i = 0; i < 1000; i++)

{

var ptr = Heap.HeapAlloc(hHeap, 0, nSize);

nTot += nSize;

Trace.WriteLine(String.Format("Iter #{0} {1:n0} ", i, nTot));

}

catch (Exception ex)

{

Trace.WriteLine("Exception " + ex.Message);

}

Heap.HeapDestroy(hHeap);

Trace.WriteLine("destroyed");

Application.Current.Shutdown();

}

public class Heap

{