In defense of Prevalence

[Francois-Rene Rideau]

On Fri, Feb 13, 2004 at 01:32:59AM +0100, Massimo Dentico wrote:
>> http://cliki.tunes.org/Prevalence
With all this heat, the stuff should be moved to a "debate" page.

> If you like we can call a "paradigm shift" the return to an old
> *discarded* technique, but this does not change the substance of the
> problem.
It's all about a new, higher-level look, to an old technique, journalling.

> I thought that one of the points of Tunes is indeed that of getting
> rid of the "application" concept as an artificial barrier
> to the free flow of information);
Rather, Tunes is about having a more meta-level view of it.
The whole system is a generic/higher-order/reflective application;
it has sub-applications, characterized by a mapping to the generic framework
as defined in a programming language or another.
The meta-level approach means that instead of having to have
a one-size-fits-all runtime that interprets application requests,
we can dynamically and coherently generate ad-hoc code
that correctly and efficiently implements application semantics.

> Answer these questions: what is more important for you *as user*, your
> applications or your data?
The whole point is that this is a false dichotomy.
The application is what makes the data meaningful,
and the data is what the application uses to be meaningful.

> Is it not true that you can change
> algorithms, applications, data formats, operating systems, hardware but
> what *really* interests you are your data? This is true not only for a
> single user but even more for entire organizations.
What you fail to see is that in the dynamic "application" view,
the whole system is a meta-level application,
and that application change, schema evolution, etc.,
are part of the state transitions of the reflective system.
Certainly, a static view of immutable applications is doomed;
but so is a static view of immutable data schema.
What is doomed is the static point of view,
and no amount of moving around what your consider "static" will save you.
It's displacement of problems, not resolution.

> As programmer you values your programs *as data*:
Exactly - we must thus consider the development system
as an application at the meta-level.

> In particular declarative /integrity constraints/ are a key feature of a
> True RDBMS that SQL-based DBMSs don't implement correctly (as I
> understand it, the prevalence model lacks completely this concept).
What if the integrity constraints change?
A static schema is no solution.
And a dynamic schema takes you at the meta-level.
Now, with a meta-level at hand, you can meta-use prevalence directly,
and skip the need for a complex RDBMS.
And no, prevalence doesn't "lack" the concept of integrity constraint;
rather, it takes an algebra of integrity-preserving transactions
as *a parameter* to the functor that provides persistence to the application.
And a programmable meta-level allows you express your constraints
and automate their enforcement in what you feed to the prevalence functor.
Of course you may misspecify your algebra and your meta-programs;
just like you may misspecify your RDBMS constraints. Doh.
Where Prevalence+reflection frees you is that you can now tailor
your data representation algebra to your applicative needs,
instead of the other way around.

> [...] data model:
> · data types
> · structure
> · integrity
> · manipulation
Prevalence + reflection allows to decouple all these,
where a static RDBMS gives you a one-size-fits-all combination.

> Renouncing one only of these components
> means to delegate such component to each specific application, with
> possible redundancies, incompatibilities, conflicts, corruption and
> loss of data, difficulties in data access, etc ..
This implicitly supposes that there exist no other way
but centralization through the DBMS
to achieve factoring of such components accross applications.
Now, the very core belief of TUNES is that factoring is best done
through open systems with a good meta-level than through centralization.
Yes, there is a need for strong consistency-enforcement;
static thinkers see that as an argument for a centralizing kernel;
dynamic thinkers see that as an argument for declarative programming.
And indeed, in the end, the kernel will have to act upon declarations,
but will end up interpreting them in a clumsy paranoid way at runtime.
The open reflective system will compile these declarations before link-time,
so they are efficiently hardwired or optimized away at runtime.

> [...] All Requirements are Data Requirements
And dually, are application requirements.
That's just a shift in point of view.
But the meta-level shift is about seeing the previous shift,
and giving control over it to the developers.
So yes, data modelling is an important of application development,
but it's as wrong to fixate things on this point of view,
as to fixate things on algorithms and neglect data.

> This merit a comment apart: so you argue that because ".. few people
> encode the structure of 3D objects or music samples in relational
> tables" then the relational data model "needn't be the right one[sic]"?
> Well, that's science! What are the better alternatives? Hierarchical or
> network structures (graph theory)? When a data model is "the right one"?
Well, sometimes, sets, streams, trees, fifos, soups, matrixes,
hash-tables, pre-sorted binary decision trees, vertex arrays,
or whatever, are the right data-structure.
Why are strings not represented as tables in DBMSes?
Why a need for bobs and blobs at all?
In typed functional languages, even numbers and characters can be defined
in the internal type-system, and those provided by the system can be seen
as just optimization from the standard library rather than additional
features unexpressible in the programming algebra.
And yes M. Date, that's tree/graph/foo-theory; if you don't like it,
why not show everyone how much simpler it is to express all these things
(or show they are not needed) in a relational data model?

> "COMPLEX" DATA TYPES: WHAT OBJECT PROPONENTS DON'T TELL YOU
> http://www.pgro.uk7.net/fp1a.htm
I fail to see how an object database like, say, allegrostore,
fails to provide every requested feature. Can you pinpoint the problem?

> ***the
> only requirement is that whatever structures are physically stored
> must be mapped to relations at the logical level and be hidden from
> the user***.
And what if relations are clumsy for the data at hand?
In this frequent case, the relational model is just a PITA
that people have to work around, and the relations ends up being
the "physical" low-level layer to a higher-level "logical" view.

> For example, suppose the implementation in some object database [...]
> is changed from an array to a linked list. What are the implications
> for existing code that accesses that object [...]? It breaks.
No. You define new generic functions and/or you change-class your objects,
or you update their class. In any case, it's a problem
of static vs dynamic programming,
that static relational data schema doesn't solve.

> 9. It is better to have 100 functions operate on one data structure
> than 10 functions on 10 data structures.
It is better to have a meta-level algebra to define your arbitrary functions.
(In functional language lingo, they call that a type-system.)
And add generics and/or dynamics to be able to manipulate arbitrary data
the specific type of which is unknown.
So yes, Pascal-style strongly data-structures without either generics
or dynamics is dumb and clumsy.

> Hugh Darwen and C.J. Date in a book, "The Third Manifesto", show another
> approach, a complete integration between the relational data model and a
> theory of types with a proper concept of "inheritance" (this term IMO is
> an inappropriate choice, too confused; subtyping is better). This
> approach is incarnate in a *pedagogical* programming language, Tutorial
> D.
Well, indeed, if they want to be taken seriously,
proponents of relational data models will have to produce
serious programming languages that implement these models.
AP5 comes to mind - and Prevalence could be a way to add persistency to it,
just as it could help make persistent completely different data models.

[ François-René ÐVB Rideau | Reflection&Cybernethics | http://fare.tunes.org ]
[  TUNES project for a Free Reflective Computing System  | http://tunes.org  ]
A word for the epoch of free software and universal publishing:
voluntocracy n 1. governance by those who do the work. 2. the volunteers
who do the work. -- Aubrey Jaffer, http://swissnet.ai.mit.edu/~jaffer/