<main>Document Schema Definition Languages (DSDL)</main> <complementary>Character reportoire validation</complementary>

ISO/IEC

International Standard

enquiry ANSI

Information Technology

Document Description and Processing Languages

320

19757

<main>Document Schema Definition Languages (DSDL)</main> <complementary>Character reportoire validation</complementary> 2004-04-14

1Interoperability framework 2Grammar-based validation — RELAX NG 3Rule-based validation — Schematron 4Selection of validation candidates 5Datatypes 6Path-based integrity constraints 7Character reportoire validation 8Declarative document manipulation 9Datatype- and namespace-aware DTDs 10Fill in

The structure of this committee-draft standard is as follows.

specifies the schema language as a Part 3 (Schematron) schema language.
specifies the schema language as a particular query langage binding of ISO Schematron.
describes conformance requirements for implementations of character repertoire validators.
Non-normative annexes provide motivating use-cases and examples.

This committee-draft standard specifies a schema language for declaring and validating the allowed character repertoire in data content, attribute values, identifiers and other markup in XML documents. The language is specified as a particular query language binding of Part 3 (Schematron.)

The schema language uses path expressions to specify contexts for assertions and character classes to specify repertoires.

This committee-draft standard establishes requirements for schemas and specifies when an XML document matches the patterns specified by the schema.

Each of the following documents has a unique identifier that is used to cite the document in the text. The unique identifier consists of the part of the reference up to the first comma.

XPath XML Path Language (XPath) Version 1.0 W3C Recommendation http://www.w3.org/TR/1999/REC-xpath-19991116

XSLT XSL Transformations (XSLT) Version 1.0 W3C Recommendation http://www.w3.org/TR/1999/REC-xslt-19991116

XSD XML Schema Part 2: Datatypes W3C Recommendation http://www.w3.org/TR/2001/REC-xmlschema-2-20010502/#nt-charGroup

The definitions of Part 1, Part 2 and Part 3 also apply to this committee-draft standard.

character class A grouping of characters, especially into named groups according to some property of the characters.

character repertoire The characters which may validly be used in some text node.

regular expression An artificial language or group of dialects for expressing patterns in sequences of characters.

Syntax

A schema conforming to this committee-draft standard shall be a correct Schematron schema according to Part 3 (Schematron.)

If the value of the language attribute of the Schematron schema element is xslt-charrep, in any mix of upper and lower case letters, then the default query language binding of this committee-draft standard shall be used.

This committee-draft standard reserves following query language names without further definition. Implementions which use different query language bindings shall use one of these names if appropriate: stx-charrep, xslt-charrep, xslt1.1-charrep, exslt-charrep, xslt2-charrep, xpath-charrep, xpath2-charrep, xquery-charrep.

The order in which nodes and characters are validated is not specified by this committee draft standard.

Default Query Language Binding

The value of the language attribute of the Schematron schema element shall be xslt-charrep, in any mix of upper and lower case letters.

If the value of the language attribute of the Schematron schema element is xslt-charrep, in any mix of upper and lower case letters, then the Default.

The following binding shall be used:

The rule context shall be interpreted according to the production 1 of , as returning any kind of node when applied using the semantics of Part 3 (Schematron).
The assertion test shall be interpreted according to production 13 of , returning a boolean. Each string shall be tested on a character-by-character basis. Newlines and tabs shall be removed from the query before it is used to allow longer strings; consequently these characters must be entered using the delimited forms \n\t\r or the character classes. It shall not be an error if the same character or class is entered more than once.
The name query shall be interpreted according to production 14 of , as returning a string value.
The value-of query shall be interpreted according to production 14 of , as returning a string value.
The let element shall not be used.
Abstract patterns shall not be used.

The nodes allowed as subjects for assertions in this committee-draft standard could be different from the nodes allowed by the default query language binding of Part 3 (Schematron).

The data model shall be used. For the purpose of this committe-draft standard an string is an string. For each node type, the strings to be tested are as follows:

For an element node: the string-values of each text node of that element node, but not the data content of any descendents.
For an attribute node: the string-value of the attribute, as would be accessible through the text() function for that subject.
For an comment node: the string-value of the comment, as would be accessible through the text() function for that subject.
For an processing instruction node: the string-value of the processing instruction, as would be accessible through the text() function for that subject.
For name, target and namespace nodes: the name as a string.
For any other nodes, including the document root node: error.

The key element shall not be used.

Conformance Simple Conformance

A simple-conformance implementation has the same requirements as a simple-conformance implemention of Part 3 (Schematron.)

Full Conformance

A full-conformance implementation has the same constraints as a full-conformance implemention of Part 3 (Schematron) and the following additional requirements.

The schema has a language binding attribute with a value terminated by the string of -charrep.
The schema with a language binding attribute with value xslt-charrep conforms to the language binding in this committee-draft standard.

Use Cases

Motivating use-cases for this committee-draft standard include:

ensuring that a Dutch document contains characters only used in typical Dutch documents; the constraint applies to mixed content and element content;
checking that a document does not use any Latin combining characters;
declaring that data content in a Japanese document shall not contain half-width katakana characters;
providing information to alert publishing staff if the data content of a document contains characters outside the Unicode Basic Multilingual Plane, surrogate characters, or Private Use Area characters;
verifying that the data content in a scientific document uses the Unicode character for micro symbol not the Greek small letter mu;
restricting the generic identifiers, attribute values and contents of elements to CP1252 character set;
verifying a school text book that data content of Japanese ruby annotations does not contain Han ideographs and that other data content of elements should contain only the restricted repertoire used for schools; because the reportoire is large, it must be declared in an external library and referenced; and
verifying that an attribute value giving a person's name in a Chinese document only uses approved characters.

Motivating use-cases for the schema language do not include:

constraints on parts of a string, such as that an attribute should start with a certain character;
semantic constraints requiring analysis of the particular string, such as that that an attribute may contain letters or numbers but not both;
repertoire constraints between different strings in the document, such as that an element can only use the character repertoire as used in some other part of the document; and
constraints involving arithmetic operations, such as that the sum of all code values in the string should not exceed 300.

As well, certain kinds of constraints are out-of-scope for this committee-draft standard:

the maximum and minimum length of strings;
the character encoding (character set) used for an entity;
the use of standard entities, numeric character references or literal characters;
that the characters of a Thai document are ordered correctly; and
that the initial characters of a portion of a string marked up with an entity or included by some macro mechanism is not a combining character.

Example Schemas for Use-Cases Simple Examples

Ensuring that a Dutch document contains characters only used in typical Dutch documents; the constraint applies to mixed content and element content


			
				If this document is a Dutch document, it should have only characters 
				used in typical Dutch publishing.
			
		 
]]>

Checking that a document does not use any Latin combining characters


			
				This document should not use any Latin combining characters.
			
		 
]]>

Declaring that data content in a Japanese document shall not contain half-width katakana characters


			
				Elements and attributes should not contain half-width katakana characters.
			
		 
]]>

Verifying a school text book that data content of Japanese ruby annotations does not contain Han ideographs


			
				Ruby annotations should not contain Han ideographs. 
			 
		 
		 
]]>

Providing information to alert publishing staff if the data content of a document contains characters outside the Unicode Basic Multilingual Plane, surrogate characters, or Private Use Area characters


			This document should not 
					contain characters outside the Unicode Basic Multilingual Plane.
			
			This document should not 
					contain characters surrogate characters.
			
			This document should not 
					contain Private Use Area characters.
			
		 
]]>

Verifying that the data content in a scientific document uses the Unicode character for micro symbol not the Greek small letter mu


			
				The micro symbol should be used, not the Greek small letter mu.
			
		 
]]>

The preceding schema fragments should be placed in the following wrapper for them to be correct schemas.


	Examples of Use Cases
	
  ...
  

]]>

Complex Examples

The following mechanisms allow schemas to be divided into separate sections, files or documents to reduce reclaration and for better management:

Character class fragments can be invoked in tests using XML entity references;
Abstract rules; and
Abstract patterns.

Examples of each of these approaches follow.

The following examples do not attempt to provide all possible schemas that express the constraints.

Restricting the generic identifiers, attribute values and contents of elements to characters in the CP 1252 character set using an internal entity reference and a pattern with three rules.



]>

	Character Repertoire Schema for CP1252
  
  
		 
			
				Attribute values should be CP1252 repertoire.
			
		 
		 
			
				Attribute values should be CP1252 repertoire.
			
		 
		 
			
				Data content should be CP1252 repertoire.
			
		 
	

	]]>

Restricting the generic identifiers, attribute values and contents of elements to characters in the CP 1252 character set using an abstract pattern, then three concrete instances of the pattern are created, one for generic identifiers and one for attribute names, and one for data content.


	Character Repertoire Schema for CP1252

	 
			
				Text should be CP1252 repertoire.
			
		 
		
	
	
	  
	
	
	  
	
	
	  
	
	
]]>

Restricting the generic identifiers, attribute values and contents of elements to characters in the CP 1252 character set using an abstract rule in an external document. First, an external XML document is created at URL http://www.eg.com/cp1252.sch containing an abstract rule.

 
			
				Text should be CP1252 repertoire.
			
		 
]]>

Then this external abstract rule is included in the pattern. In this example, the rules are combined into a single rule.


	Character Repertoire Schema for CP1252

  
    
    
		 
			
		 
		
	


]]>

Other use-cases which may be solved by the same mechanisms are:

verifying that other data content of elements should contain only the restricted repertoire used for schools; because the reportoire is large, it must be declared in an external library and referenced; and
verifying that an attribute value giving a person's name in a Chinese document only uses approved characters.