[gclist] memory protections and system calls

[Giuliano Carlini]

Paul R. Wilson wrote:
> 
> We've come across an obnoxious interaction between memory protection
> tricks (for persistence, gc barriers, etc.) and system calls.
...
> The problem is this: when you access-protect memory and pass a pointer
> to something in that protected memory to a system call, most OS's will
> choke---they'll signal an unrecoverable error, or silently corrupt
> data, or simply die.  (We've brought SunOS down several times this way.)
...
> I'm wondering if anybody else has noticed and addressed this issue.
> (Hans, maybe?)  If there's an elegant way around this, we'd be very
> interested in hearing about it.

In no way do I qualify this as elegant. It's just a
possibly more efficient hack, which may also avoid the
problem you mention.

The following approach to implementing virtual dirty bits
(VDB's from now on) is on my list of things to try, but it
may be a while before I get to it. I suspect it will work
very well, but who knows...

Please excuse me if I say anything really stupid. I'm not
a kernel type, so I'm traveling in a foreign land.

My approach let's the OS inform me of the state of the
real dirty bits. To do this requires a device driver a
to create a psuedo device and mapping memory to a file
on the psuedo device. As the mutator modifies the mapped
memory, the hardware sets the real dirty bits. The OS
polls this to set up the LRU cache. Periodically it writes
out old pages. The driver will see this, update the VDB,
and pass the call on to the "real" device driver.

The VDB may be wildly out of date, and usually this
shouldn't be a problem. But at times, the VDB client will
require it to be accurate. In this case, the client calls
the kernel "flush" function on the psuedo device file
mapped into memory. A side effect is that all modified
memory is written to disk. At times this may be desired,
but often it is not. Writing can be blocked by calling the
driver with an ioctl before and after the flush call. The
calls tell the driver that the writes between them or only
for the purposes of updating the VDB; they should not pass
the call on to the real driver. Then, all the pages that
were "written", must be touched so that the kernel believes
them to be dirty again.

My motivation for doing this was to make it cheaper to
maintain dirty bits. It should also avoid the problem
where write protecting pages causes access faults in
kernel calls.

I believe it is cheaper for two reasons. First, it avoids
context switches into the kernel when writing to a
protected page, back into user space to execute the user
fault handler, back to the kernel when the fault handler
finishes, and then back to user space to continue the
mutator. Second, it avoids the user fault handler entirely;
this can be very expensive under Win32, but is reasonably
cheap under Unix.

It also, makes incremental GC more efficient. A problem with
incremental GC is that after rescanning a page, the mutator
may modify the page again, requiring the page to be scanned
again. We'd like to defer rescas. But, not too long. If the
page is swapped out and must be read back in, rescanning
becomes expensive. The psuedo-device driver, allows us to
rescan when the OS is swapping out a dirty page. This minimizes
the number of pages which are scanned several times. Of course,
at some point it becomes necessary to make the VDB accurate,
and to scan all remaining dirty pages.

If you decide to try this technique, I'm interested in how
it works for you.

Also, is this a technique which is suitable for writing up
as a paper?

Thanks,

g