Metadata Driven SQL Requirements

Metadata-Driven SQL Requirements

One of the goals of the Persistence layer is to provide developers with a mechanism to develop database independent code. In order to do this, developers must specify the SQL to be used in a specific location that can easily be changed when the database is changed. In an attempt to save time for developers, the PDL definitions can be used to automatically generate SQL events and DDL statements, thus saving the developer time and energy. This text was written by Randy Graebner.

The second part of metadata-driven SQL builds on top of automatically creating SQL for developers. Since most web applications know what database schema they need to use well before they are deployed, it is possible for them to define these table definitions in some form (either SQL files or PDL files or both) before compile-time. However, it is often the case that site administrators, and not developers themselves, specify what information should be stored. For instance, a publisher of a site might want to start publishing movie reviews or hotel information. They may also want to change what information is stored about restaurants. Providing site administrators with the ability to make these changes without soliciting the help of the developer requires a mechanism in place to generate the correct SQL statements from the information input by the administrator.

Overview

The Metadata-Driven SQL Persistence subsystem must provide application developers with the ability to:

Use-cases and User-scenarios

Developers are the only users of the Metadata-Driven SQL system. They must be able to use the system to allow for flexibility at the application level. As a developer, I need the Metadata-Driven sql system to allow me to do the following:

Related Links

Requirements

10 Automatic SQL Generation for Pre-Defined Schemas

10.10 Provide additional PDL syntax

The PDL syntax must provide a way for developers to specify the properties of an object type, including attribute names and associations. Specifically, it will handle the following cases:

  • All object information contained in a single row in a single table.

  • Object information spread among multiple tables with zero or one row in each table, provided one table is specified as always have a row if the object exists.

  • Objects with 0..1, 1..N, and N..N associations. The 0..1 association can be stored as a row in another table (that may or may not exist) or as a column within the table of a normal object (the more common situation). 1..N and N..N associations must be stored within a mapping table and all N associations must be stored in the same mapping table. It is not possible to spread the same association among different tables. That is, while the information about the associated object can be kept in different tables, system system must be able to use a single table to find all IDs for the associated items.

10.20 Automatic CRUD Generation for Object Types

The system must be able to use information in the PDL file to automatically generate all create, retrieve, retrieve all, update, and delete SQL statements to act on the object.

10.30 Automatic CRUD Generation for Associations

The system must be able to use information in the PDL file to automatically generate all create, retrieve, update, and delete SQL statements to act on pre-defined associations for object types.

20 Dynamic Subtyping of Object Types

20.10 Extending PDL Syntax

The PDL syntax must allow application developers to designate columns within tables as available to be used for subtyping of a given Object Type. For instance, if the developer wanted to subtype the "ContentType" object type the PDL file would have something like

object type ContentType {
    <schema and events defined here>
    ...
    flexfields {
        wide_table.column1  varchar(700);
        wide_table.column2  varchar(700);
        wide_table.column3  integer;
        wide_table.column4  integer;
    }
}
      

The example above means that any object type that is a subtype of "ContentType" will have all of the attributes of ContentType and will also have the ability to have 2 extra String attributes and 2 extra Integer attributes. If an object type does not have this definition, a user of the system will not be able to create a subtype.

20.20 Dynamic Subtyping only available to Object Types that specify desire at compile time

20.20.10 PDL definition required to subtype

The ability to subtype an existing object type is only available to application developers if the extension is specified in the PDL before the code is compile.

20.20.20 Underlying schema may be altered.

Once an object type is dynamically subtyped, the developer may alter the schema used to store the new object type's data. However, the developer must make sure to keep all attribute names the same and update all appropriate files.

20.30 Public API for defining new object types

The system must provide a public API for defining a new object type. The API must provide the developer with an interface to specify the object type to be subtyped as well as information about each attribute such as the name and type (for example String, Boolean, etc.). The API must also allow the developer to remove dynamically created attributes from the dynamically defined object type. Attributes that belong to the parent object type cannot be removed.

20.35 Single and Multiple Attributes Available

The system must allow for three types of attributes: "0..1", "1..1", and "0..N". Thus, it must be possible for an object to posses multiple values for the same attribute.

20.40 Dynamic object types should act like predefined object types

20.40.10 Dynamic objects behave like predefined objects

After a dynamic object type is defined, an application developer should be able to access it in the same way he is able to access predefined object types. The main difference is that the Java code accessing the dynamically defined object type will not be able to hard code any of the attribute names in the database because doing so could cause an error if the site user tries to change the name. While the code will be able to use the same "get()", "set()" and "save()" methods for both dynamically defined and statically defined object types, the code leading up to those calls will be different for the two types.

20.40.20 Dynamic objects must persist over server restarts and must be shared by all servers running the application

The system must store information about dynamic object types so that it can recover this information after server restarts. The storage of dynamic object types must be ACID compliant.

20.50 Dynamic Associations Allowed

The system must provide a way to create associations (and link attributes) between dynamically defined object types and both statically defined object types and other dynamically defined object types. The associations can be one-to-one associations, one-to-many mappings, or many-to-many mappings.

20.60 Dynamic Subtyping Allowed

The system must allow dynamically generated subtypes to be subtyped. Concretely, if a content manager dynamically created an object type called "Lodging" he should be able to subtype the "Lodging" object type and create the object types of "Bed & Breakfast" and "Hotels".

30 Performance

30.10 Same Performance at Java Level

The dynamically generated SQL must operate at relatively the same speed as SQL that is predefined in the PDL files.

30.20 No Guarantee on Database Performance for Dynamic Subtyping

30.20.10 Database query time a responsibility of application developers

While the persistence layer should strive to make the database queries as fast as possible, no guarantees can be made as to the speed of database queries involving subtypes. If performance is a problem for a given subtype, it will be up to the application developer to convert a dynamic object type to a predefined object type.

30.20.20 Persistence will allow developer conversion from Dynamic Object Type to Statically Defined Object Type

It must be possible for developers to take a dynamically defined object type and convert it to a statically predefined object type.