[gclist] memory protections and system calls

[Paul R. Wilson]

>From creedy@mitre.org Mon Jul  1 08:52:47 1996
>Subject: Re: [gclist] memory protections and system calls
>Cc: wilson@cs.utexas.edu
>
>At  3:04 PM 6/28/96 -0500, Paul R. Wilson wrote:
>>We view this as a bug in the OS, but we still have to deal with it.  In
>>our view, a protected page should be treated the same way by the kernel
>>as by user code---the kernel should reflect the access violation back
>>to the user process in the form of a user-level signal, so that the
>>app can deal with it---typically by unprotecting the page and doing
>>some bookkeeping before returning.  Then the kernel should resume and
>>do whatever it's been asked to do.  (Like writing data from a buffer.)
>
>Having been something of an OS hack in the past:
>
>It is almost impossible to do this in _current_ operating systems without
>making the OS dependent on user code. 

In what sense(s)?

>This is something that no sane OS
>developer will ever do.  (The problems associated with user code causing
>the OS to crash and/or behave in strange and unusual ways are too much to
>deal with.)  Your typical OS developer would tell you that this is not a
>bug in the OS, but the OS defending itself from the user.

I'm not sure I understand your point.  What in particular is it about the
structure of current operating systems that makes this dangerous?  Is it
that they're not multithreaded, hence can't stop in the middle of a system
call and let a user process fix a problem, because the OS will lock up?
Or is it something else?  Why do you need capabilities (or whatever) to
fix it?

Conceptually, I don't see why it matters whether a user-level protection trap
actually occurs in user code, or in kernel code on behalf of the user process.
I'd think that the user process owns the memory mappings that must be
manipulated, hence it's okay for it to hold locks in the kernel on whatever
must be locked.  (Any dependent process that inherits those mappings is
vulnerable anyway.)  We don't have to allow the user process to do
arbitrary operations inside the kernel.