Callbacks vs. GC

[Todd Fleming]

It doesn't look like the garbage collector provides a way to lock 
objects into position that works well with callbacks. Here's an example.

In Windows, every window has a callback called a window procedure 
(WNDPROC). It would be pretty easy to generate machine language WNDPROC 
stubs at runtime that call an appropriate C function (I do something 
similar in C++). The stubs would pass their arguments plus a pointer to 
a Slate block to the C function, which would then generate the 
appropriate LexicalContext and invoke the interpreter to execute the 
block. For this to work, the Slate block must be locked into memory; it 
can't move around.

Presently, the GC only provides one way to keep objects from moving 
around; they must be pointed to by something on the C stack. 
Unfortunately, the WNDPROC far outlives any references to it on the C 
stack. Here's a simplified chain of events:

1. Slate code creates a Slate block.
2. Slate code invokes a C function that creates the WNDPROC stub and 
passes it to Windows. If GC is invoked for any reason, a pointer to the 
Slate block will be on the C stack, preventing GC from moving it around.
3. The function returns. The block is no longer pointed to by the C stack.
4. The GC is triggered. The block moves.
5. Slate calls Window's GetMessage(), TranslateMessage(), and 
DispatchMessage() functions.
6. Windows invokes the callback. kaboom.

I looked at the JIT code to see how it handles a similar situation 
(non-relocatable machine code), only to discover that it doesn't even 
use the GC heap to store the generated code.

I started looking to see what it would take to add additional features 
to the garbage collector (support for immovable objects, support for 
weak references, support for finallizers, reducing the number of times 
marking is needed by using reference counting, etc.). Unfortunately, it 
looks like multiple .slate files are aware of how object headers work, 
so these changes would be huge.

So, I'll scale back my proposal. Would it be ok to rob a bit from idHash 
and add a new field, isFixedInMemory? Any object with this set to 1 
would be treated by GC like it was pinned by the C stack. Cleanup will 
be problematic without finallizers and weak references, but this will be 
a fast and dirty way to get where I'm trying to go.

Todd