Site Navigation

Overview

This document can be viewed online, but it is really made to be printed off, as a sort of stand-alone manual. Note that the some of the code output lines are too wide to fit on a page oriented vertically (portrait). Because of this, you may want to print this document at a reduced size. We have found that printing at 80% size works well.

Major Sections

Mobius Software Requirements and Basic Installation

Software Prerequisites

Build Instructions

The QuickStart examples use the binary distribution, but building Mobius from source is simple. You will need Apache ant and it should be in your path.

Extract the Mobius...full-source.zip file and the Mako, GME, DTS, Registrar, Common, etc. sub-directories. cd into the sub-directory for the service you want to build. Each component package will build independently—there is no need to build all of the component packages. After cd-ing into the directory for the service you want to build, type ant all. This will build and compile the software.

Note: The directory structures documented in these QuickStarts are based on the binary distribution. You will notice some minor differences, if you are working from from-source compiled packages.

Basic Software Installation, Saving Your Mobius Work Sessions, and Discussion of Mobius Services

We will use the binary distribution for the QuickStarts.

Extract the binary package. Within it, you'll have the following zipped archives:

  • mobius_common... (Mobius base classes)
  • mobius_gme... (A grid service that provides DNS-like, schema registry and exchange services)
  • mobius_mako... (A grid service for exposing data services as XML data services)
  • mobius_registrar... (A grid service for discovery & communication between Mobius services)
  • mobius_dts... (A grid service for translation of semantically similar schema)
  • mobius_tools... (Mobius command-line utilities and viewers for all packages)
  • mobius_factories... (Mobius grid service factories)

Extract the package you wish to work with. Each package can stand by itself. There is no need to extract all of them, though you may want to do so, for convenience, especially if you are going to work your way through all of the QuickStarts.

For the guides, we will be using the .bat scripts. Use the .sh scripts on Linux/Unix and .bat scripts for Windows machines.

The QuickStart guides are written so that you can view them in any order. Just start with whichever service interests you most.

Note: You must have MySQL installed and running on machines on which you start any of the Mobius services. Database schemas will be created within MySQL and so it will be necessary to know the "root" or "admin" MySQL passwords on all machines.

These QuickStarts discuss the Java Swing-based GUI viewers for both Mako and GME. During a GME or Mako work session you will often make changes to the viewers lists of services, XML collections, schema, etc. At any time during a session, under the "File" menu within the viewers, you can "Save Configuration", which will save out an XML file that you can then reload during subsequent sessions. This will allow you to pick up your work where you left off.

The Mobius Framework is made up of a number of services, listed here. You can read a full overview of Mobius on this web site.

Mako
a data instance management service to create new databases, integrate existing databases, validate models against the global model, and run federated queries
GME
(Global Model Exchange), a DNS-like, global, data definition registry and exchange service
Service Registry
a basic registry to facilitate discovery of and communication between running Mobius services
DTS (no QuickStart guide at this time)
a data translation service to translate between existing data models that have similar semantic content but variable structures

Support

Mobius users are encouraged to join our mailing list. The list and its archives will become a primary source for answers about Mobius usage.

Helpful Online Resources

Mobius Configuration Files and Shared APIs [top]

Mobius Application Programming Interfaces (APIs) (common to all Mobius services)

The Mobius API consists of a hierarchy of well defined interfaces that provide application developers access to perform operations on Mobius Grid Services. GridServiceFactory implementations are the foundation of the Mobius API. Given a Grid Service Identifier (serviceId), a GridServiceFactory will create an handle to an instance of the proper implementation of the GridService interface. The handle to the GridService can then be used to perform operations on a Grid Service.

Factories are a key component to the Mobius API. They provide an abstraction between the client interfaces and the implementation, thus allowing multiple implementations on the client side. As the Grid evolves or application requirements change, implementations that use different communication layers or meet different performance requirements can be swapped in, without changing application code.

For more details on creating and using factories, see the GridServiceFactory section below.

The GridServiceResolver provides a method for managing factories, such that multiple factories may be used within an application. It is also used to get handles to Grid Services. Handles to grid services are obtained by providing the service identifier of the service. Handles to grid services are represented by the GridService object. The GridService object contains accessors for common grid service operations. The GridService object can be casted to a specific GridService object (e.g., XMLDataService) which would provide accessors to operations for that type of Grid Service. Below we show code for getting a handle to a Grid Service: 


GridServiceResolver resolver = GridServiceResolver.getInstance();
GridService gs = resolver.getGridService("MAKO://localhost");
XMLDataService xds = (XMLDataService)gs;
List cols = xds.getCollection();

The GridServiceResolver resolves grid services by using GridServiceFactory objects. By default, the GridServiceResolver uses the default factory to resolve a grid service identifier; however, this requires that the default factory be set. Set the default factory as follows: 


GridServiceResolver resolver = GridServiceResolver.getInstance();
resolver.setDefaultFactory(new NetworkServiceFactory());

Although using a default factory is often sufficient, there are cases where using the default factory is inadequate—if you have multiple Grid services that are interacted with through different implementations of client interfaces or if clients have varying configurations. To support the above case, the GridServiceFactory employs associations. That is, the GridServiceResolver allows you to associate a factory with a group and it allows you to associate Grid Service identifiers with that group. The code snippet below illustrates this: 


GridServiceResolver resolver = GridServiceResolver.getInstance();
resolver.setDefaultFactory(new NetworkServiceFactory());
resolver.associateFactory(new SpecialFactory(),"SpecialGroup");
resolver.associateService("SPECIAL://localhost","SpecialGroup");

In the above code we set the default factory as well as create an association for a special group of services. Given the above configuration of the GridServiceResolver, if one wishes to get an instance of the service SPECIAL://localhost, the grid service resolver will use the SpecialFactory to resolve the service. If one requests an instance of the grid service MAKO://localhost, the GridServiceResolver would still use the default factory, however.

GridServiceFactories

As discussed above, Grid Service Factories provide an abstraction between the API and interaction with the Grid Services. In other words, they provide an abstraction between the client interfaces and implementation of those interfaces. Grid Service Factories are created by extending the abstract class, GridServiceFactory, and by implementing the resolveGridService() method.

Grid service factories can be configured through arguments, which can be defined by the implementer of the GridServiceFactory. Upon requesting a grid service, Grid Service Factories are validated to ensure that required arguments were specified. Below we show an example implementation of a GridServiceFactory class: 


public class MyFactory extends GridServiceFactory {
  public MyFactory() {
    ArgumentDefinition protocol = 
      new ArgumentDefinition("PROTOCOL","The communication protocol",true,1,1);
    this.addArgumentDefinition(protocol);
  }
  protected GridService resolveGridService(String serviceId)
    throws MobiusException {
    String protocol = (String)this.getArgument("PROTOCOL").getValue();
    return new MyHandle(serviceId, protocol);
  }
}

The above code shows the implementation of a GridServiceFactory, MyFactory. You will notice in the constructor that an ArgumentDefinition is created for the communication protocol and that protocol is a required argument that must have only one value. In other words, the instantiator of MyFactory must specify one and only one protocol argument. The other area to note is the resolveGridService() method. It takes the value of the protocol argument and the service id specified by the caller to create an instance of MyHandle, which is assumed to implement the GridService interface. Next we will illustrate how one would instantiate and use MyFactory: 


GridServiceFactory factory = new MyFactory();
Argument arg = new Argument("PROTOCOL","TCP");
factory.addArgument(arg);
GridService service = factory.getGridService("MY_GRID_SERVICE");

In the above code we create an instance of MyFactory, create the protocol argument, and add it to the factory. We then request a handle to a Grid service, which, assuming the factory can be validated, creates and returns a handle.

Mobius Configuration File Settings (common to all Mobius config files)

The Mobius configuration resource block occurs at the top of every configuration file in Mobius. Among other things, this block defines the Network Service Descriptor the service will use when identifying itself on the Grid, and the handlers for the various protocols in the server. This resource is defined under the element <resource name="mobiusConfig" class="org.projectmobius.services.common.MobiusConfiguration">. (All resources in Mobius are specified in this way; the name attribute is the name the resource will have in the resource manager and the class attribute is the class within Mobius that will be associated with that resource.)

The element <mobius-configuration>, the outermost element of the resource, contains five potential sub-elements. Each is listed here, along with its children. Below this list we have elaborated the purpose of the elements.

  1. MobiusNetworkServiceDescriptor
    • serviceIdentifier
    • ports
    • aliases
  2. handlers
    • handler
  3. gridServiceFactories
    • gridServiceFactory
  4. associations
    • factoryAssociations
    • serviceAssociations
  5. serviceRegistry
<MobiusNetworkServiceDescriptor>
All Mobius services have a Mobius Network Service Descriptor that they use to describe themselves to other services within Mobius. The "serviceType" attribute defines what kind of service this is. In Mobius you can have the following service type descriptors:
  • MAKO
  • GME
  • REGISTRAR
  • DTS
The "hostname" attribute defines the name of the host this service is running on. You may use the fully-qualified domain name or any host name that can be resolved on your local network. The children of the Mobius Network Service Descriptor describe the service:
<serviceIdentifier>
This element describes the service identification string used to uniquely identify a Mobius Service within the Grid.
<ports>
This element describes the ports and protocol the service will use for communication. Currently only TCP communication is supported.
<aliases>
This element defines any aliases for the host of the service. Aliases are needed when a host has more than one physical network connection. An <alias> element takes the attributes "id" and "hostname" that define the identification and the hostname that each network interface maps to. A host might have multiple hostnames if it is part of a cluster and/or has multiple network interface cards installed. If the host does not have any extra network interfaces, it is safe to leave the <aliases> element empty.
<handlers>
The individual handler elements define what classes Mobius should use to handle requests and responses. They map a protocol to an implemenation that can handle that protocol. Out of the box, changing these from the way they are defined in the example files will likely break some functionality and should only be done when you are customizing Mobius or extending it to use other services. The "name" and "class" attributes in each handler will have values that depend on the Mobius component being used. That is, the Mobius configuration resource blocks in the configuration files in the Mako component's /conf directory will have handlers specific to Mako; the Mobius configuration resource blocks in GME config files will have handles specific to GME, etc.
<gridServiceFactories>
Grid Service Factories are responsible for establishing communication with other services on the Grid. A Grid Service Factory abstracts the communication between Grid services—the factory mapping the details of the communication appropriately between the services. The Mobius portion of a configuration file must contain at least one (default) Grid Service Factory.
<gridServiceFactory>
The Mobius config must have at least one of these elements with its "isDefault" parameter set to "true". You can set up an arbitrary number of Grid Service Factories, allowing you to hook up Mobius with any Grid Service for which you write the factory (or for which there is already a factory available within Mobius). Grid Service Factories can take any number of parameters. You can pass parameters to a Grid Service Factory by using the <param ...> sub-element of the <parameters> element. See the localhost-mako-registrar-config.xml configuration file for an example.
<associations>
Associations are responsible for guaranteeing that a specific type of request is handled by an appropriate Grid Service Factory. The Mobius API documentation contains an example of creating an association for handling a request for a user-defined service (called "SPECIAL"). And you can see in the Mako /conf/localhost-mako-registrar-config.xml config file an example that uses the association element to associate requests made from a Mako to a GME with a particular Grid Service Factory, in order to facilitate inter-service communication.
<factoryAssociations>
This element contains any number of <factoryAssociation ...> sub-elements. Each sub-element takes two attributes, "associationName" and "factory". The value of "associationName" is the name of the service type for the Grid Service Factory you are attempting to associate. The "factory" must be set to the value of the "name" attribute of the Grid Service Factory you want to map. For example, if you want to make it so that requests for GME connections get handled by the "default" Grid Service Factory, you would set "factory" to "default" and "associationName" to "GME". (See the Mako /conf/localhost-mako-registrar-config.xml config file for this specific example.)
<serviceAssociations>
This element contains any number of <serviceAssociation ...> sub-elements. Each sub-element takes two attributes, "serviceId" and "associationName". Set the "serviceId" attribute to the service ID of the service you want to handle requests for the service type you specify in the "associationName" attribute. The "associationName" attribute must align with the "associationName" defined in a "factoryAssociation ...> element. (Again, see the Mako /conf/localhost-mako-registrar-config.xml config file for an example.)
<serviceRegistry>
This element defines the service registry your Mobius service will use so that other Grid services can discover it. When the Mobius service is instantiated, it will attempt to register itself with the registry specified in this element. The element is required, even if the Mobius service does not register itself with a remote service registry. In that case, it "registers" itself with itself: the "serviceId" will be "localhost" and the "registryClass" attribute will be set to "org.projectmobius.client.common.DefaultRegistry". To have the Mobius service self-register on instantiation, however, we need to use the <serviceRegistry> element to identify some other Grid service registry (such as that within Globus or OGSA-DAI). A developer can extend Mobius and write a class that implements the service registry class interface (in "org.projectmobius.client.common") to perform registration with some non-Mobius Grid service registry, or she can use the Service Registry that is part of Mobius (registrar). In the latter case, set the "serviceId" attribute to a machine running a Mobius Service Registry instance, e.g., "REGISTRAR://resolvable.machine.name", and set "registryClass" to "org.projectmobius.common.registry.MobiusRegistrar".

Mako and VMako QuickStart, Config File, APIs, Command Line Utilities [top]

Mako/MakoDB QuickStart

Introduction

Mako is a service for exposing data resources as XML data resources on the Grid. In this QuickStart we will be exposing MakoDB, a Mobius XML database, layered on top of MySQL and optimized for working within a Grid environment.

Minimal Configuration

If you have not already done so, install the Mobius software and its prerequisites.

Change to the mobius_mako.../conf directory and edit localhost-mako-config.xml

Toward the bottom of the file, in the <resource="makoDBConfig" ... > resource block, fill in your password, thus <password>your-mysql-password</password>,

MySQL needs to be installed and running for this example and the <user> should be set to an account that has administrative privileges on MySQL (usually root or admin).

MakoDB, an XML Database layered on top of MySQL, will be used for this quickstart example. The sample localhost-mako-config.xml file contains parameters for establishing collections within MakoDB. You should note that there are configuration files in the conf directory with parameters established for allowing a Mako to talk to other XML databases, including Xindice, XQuark Bridge, and eXist.

Starting the Mako and Adding a Data Service and Collection

cd to /mobius_mako.../scripts

$ ./startMako.bat ../conf/localhost-mako-config.xml &

You should see output like the following: 


INFO - Dec 14, 2004 11:08:35 AM  -- The collections database MobiusMako_makodb was created
INFO - Dec 14, 2004 11:08:35 AM  -- The collections table(s) were created
INFO - Dec 14, 2004 11:08:35 AM  -- Server started on MAKO://localhost at Tue Dec 14 11:08:35 EST 2004
INFO - Dec 14, 2004 11:08:35 AM  -- Setting Service Identifier to MAKO://localhost
INFO - Dec 14, 2004 11:08:35 AM  -- Listening using the protocol TCP on the host 0.0.0.0, port 3940

Note: If you are upgrading from a previous version of Mobius, starting a Mako will automatically upgrade your database schemas.

Hit return and type,
$ ./getCollectionList.bat -service MAKO://localhost to see the list of collections currently running on localhost. At this point, there will be no collections listed. For this quickstart, you will create collections using the Mako View.

For a full list of available command line utilities, see the Mako command line utilities reference.

$ ./makoViewer.bat & starts the Mako View Java GUI application.

You'll see a window with a world icon and "Mobius Grid Services." This is the root of all services you'll establish within the Mako.

  • Right-click the icon and select "Add Grid Service"
  • Enter MAKO://localhost in the "Grid Service Id" field and click "Add". (See Fig 1)
  • Add a collection to the service: Right-click on the localhost icon in the Mako View and select "Create Collection"
  • Enter a name for the collection in the "Collection" field at the top of the window. For this example, use testMakoDB as the name. You will see below that "MAKO://localhost" has been set as the name for your XML data service. If you had already established other data services, you would be able to put checks in the boxes next to any of the service names upon which you wished to establish a new collection.
  • Click "Create." (See Fig 2)
dialog window showing how to add a grid service
Fig 1. Adding a grid service
the Mako View main window
Fig 2. Populated Mako View main window

After you add a default data service (a MakoDB service, in this case) and create a collection, in this case, testMakoDB, you will see that MySQL has two new databases, named for their connection and function:
mobiusmako_makodb which will hold information that identifies what collections you have established in your Mako; in this case, it contains a single row, since you only have one collection set up, and
mobiusmako_makodb_collection_1 which holds the attributes of the collection; in this case, it will hold all of the attributes for the first collection you have established in the Mako (testMakoDB).

Adding a Schema, Data, and Performing Queries

Mako is a strongly typed system. The data elements that you store in and query from a Mako need to be described before they can be stored. In order to do this, we use XML schemas. For a Mako to accept an XML document, it is required that a schema describing that document be submitted to the Mako or that the Mako be able to locate a schema describing the XML data. In this QuickStart, we are running a Mako as a stand-alone service and so, below, before adding XML data to the Mako, we manually add a schema describing that data.

In the usual case—when Mako is used within a larger Grid environment—the Mako would be configured to obtain schemas from a GME. For a description of this architecture, see our overview of Mobius. Also see the Mako configuration page for how one sets up a Mako so that it requests schema from a GME on startup.

  1. Let's submit an xml schema describing the structure of some XML data,
  2. submit some xml data that conforms to the schema,
  3. and perform some XPath queries against that data to glean some information from the xml.

Submit a schema to the Mako

  • Right-click on the testMakoDB collection in Mako View and select "Submit Schema." The window will show you the path of the collection to which you are applying a schema, testMakoDB in this case.
  • Click the "Add..." button and browse to the xml directory. Select "bookstore.xsd," the schema definition that will describe the XML entries you'll import in the next step. The path to the file will be entered into the "Add" field.
  • Make sure the checkbox next to MAKO://localhost is checked and click "Submit." A schema result window will display, describing the action you just took and giving the "Status" of the submission (OK).
  • You may close the results window.

Submit some XML conforming to the schema

  • Right-click the "testMakoDB" icon and select "Submit XML" in the dialog window
  • "Browse" to xml/BookStore.xml, a sample file containing fake book descriptions.
  • Click "Submit" and you will see another result dialog window. The document you submitted has been assigned a unique ID.

If you want to see a representation of the XML, click the XML button on the result dialog and you'll be given a two-panelled window. The left pane shows a browsable tree view of the XML file via which you can inspect all of the elements and attributes of the BookStore.xml file. The window also allows you to add XML namespaces to the document. (See Fig 3)

xml browser window
Fig 3. A view of the XML browser

Perform a couple of XPath queries on the data

  • Again, right-click on testMakoDB
  • Select "XPath Query." The query editor window appears. You'll see a field containing the testMakoDB collection and an XPath, which is set to /, the root of the collection, by default.
  • Click "Query" and, in the dialog window, the "XML" button. You'll see the XML browser window that you saw earlier. We want to add namespaces definitions to the XPath query editor so that we can perform queries on them.
  • To do this, click the "Add Namespaces to Namespace Definitions" button, which will parse the XML file and grab every namespace associated with that document.
  • Close the XML browser and the XPath query dialog window and then reload the XPath query editor by selecting "XPath Query" from the dropdown menu associated with the "testMakoDB" icon. This time you'll see that the three namespaces from the XML document have been added to the "Available Namespace Definitions" area.

Let's type in an XPath query to return results for the titles of all of the books in our bookstore:

First, we choose a default namespace upon which to perform the query:

  • Click "Edit" on the XPath Query dialog window. You'll see the three namespaces associated with the BookStore document.
  • Hover the cursor over the last one in the list, with the namespace prefix mobius. A tooltip will appear: "Right click a definition to set as default." The projectmobius/1/Bookstore namespace appears in the "Default Namespace" field.
  • Click "Ok."

Back in the query dialog window, enter the following query in the XPath field to return items for every title in the bookstore:

/BookStore/Book/Title

Each returned result contains the name of the data service, collection, and a unique document id generated for the query result. The query results return well-formed XML results that can be submitted to other collections or re-added to the current collection. Clicking the "XML" button on any entry will bring up the XML browser containing the XML result document.

Here is a query that grabs all books classified as "drama" (See Fig 4):
/BookStore/Book[@section="drama"]

a query using the attribute syntax
Fig 4. Query result demonstrating use of the XPath attribute syntax

More complicated queries can be performed. Mako supports the following XPath query syntax and its permutations:

/A
Selects all documents with root element A.
/A/B
Select all B elements which are children of a document with root element A.
//B
Select all B elements, regardless of their position in their document.
/A/B[@name]
Select all elements of B which are children of the root element A and have elements which have attribute name.
/A/B/[C/@name]
Select all elements of B which are children of the root element A and have child C elements with the attribute name.
//B[@name]
Select B elements which have attribute name, regardless of their position in their document.
/B[@name='b']
Select all documents with root element B which have attribute name with value 'b'.
/B[@name1='b' and @name2='c']
Select all documents with root element B which have attribute name1 with value 'b' and attribute name2 with value 'c'.
/B[@name1<='b' or @name2>'c']
Select all documents with root element B which have attribute name1 with value less than or equal to 'b' or name2 attributes greater than 'c'.
/A/text()
Select the value of the text node child of all documents with root element A.

There is an excellent XPath tutorial at zvon.org.

Removing XML Data and Collections From the Mako

To remove XML, select "Remove XML" from the testMakoDB contextual menu. You can either enter the document id or an XPath. When using the Mako View, you can only remove an entire document from a collection. If it's necessary to remove a portion of a document or perform an update to the document, you can use the submitXUpdate script in the scripts directory. In coming releases, the Mako Viewer will support XUpdate.

To remove a collection, again, use the contextual menu that appears when you right-click any of the icons within the Mako View main window. Choose "Remove Collection" and enter the name of the collection, testMakoDB in this case. Click "Remove."

In this quickstart example, we are using the MakoDB XML database. After removing the testMakoDB collection, you will notice that the database instance "mobiusmako_makodb_collection_1" has been deleted from MySQL and the "mobiusmako_makodb" database has had its "collections" table updated to reflect the removal of the collection.

Stopping the Mako

To cleanly shutdown the localhost Mako service, cd into /scripts:
$ ./stopMako.bat -services MAKO://localhost

Virtual Mako (VMako): Aggregating Mako Servers

A VMako is a single Mako server that acts as a front-end to multiple remote Mako instances. The single VMako instance maps its (virtual) collections onto a user-defined set of remote Mako collections. This reduces the complexity for the client application, presenting a single virtualized interface to a number of distinct, federated Makos. The primary purpose for a Virtual Mako is to enable distributed query execution. In a Virtual Mako, requests are broken down into sub-queries and sent to appropriate remote Makos. Responses are then aggregated at the VMako and returned to the client. Additionally, collections added to a VMako can be distributed across the remote Makos, thus reducing the load carried by any single instance. The default "ingestor" class uses a round-robin algorithm for distributing load across the federated Makos.

As in the other QuickStarts, before instantiating a VMako, you will want to edit the configuration file. There is a localhost-vmako-config.xml file in the /conf directory. It contains the configuration to stand up a basic VMako instance. In the <MobiusNetworkServiceDescriptor ...> tag, edit the "hostname" attribute as appropriate. Also set the "serviceId" in the <serviceIdentifier ...> tag, immediately below it.

A VMako aggregates other Mako services. Because of this, it needs no configuration for MySQL. Instead it can either be instantiated alone and then you can use the Mako Viewer or command line utilities to manually aggregate other Makos "within" it. Or you instantiate the VMako so that it reads a cache file and automatically "ingests" collections from already running Makos.

In the config file, the resource element whose "name" attribute is set to "vmakoConfig" contains a sub-element called <vmako-storage ...> which takes only a "file" attribute. In the distribution's localhost-vmako-config.xml file, that attribute is set to "vmako-storage.xml".

Let's start the VMako
./startMako.bat ../conf/localhost-vmako-config.xml

If you have left the <vmako-storage ...> element with its default "file" value, then any collections you add (via the Mako Viewer or the command line) will be recorded in the vmako-storage.xml file. The file will be created in the /scripts directory by default (since that is where the startMako script is). But you can also specify a path when setting the value for "file". The collections you add to your VMako during the session will be recorded and, the next time you start the VMako, the file will be read in and the collections ingested.

In the /conf directory, there is an example VMako cache file called vmako-cache-example.xml. To eliminate the need to manually add collections, we can load the remote Makos and collections specified in this file when we start our VMako.

You'll notice that the vmako-cache-example.xml file has two child elements below the root <vmako> element. The first child is <xml-data-services>. It contains tags that reference the Makos that you desire to aggregate.

The second child (<virtual-collections>) contains two sub-elements. The first (<virtual-collection ...>) specifies the name of a virtual collection that you will use to run queries against. Queries run against that virtual collection will be executed on the aggregated collections specified in the other sub-element, <xml-collection-handle ...>. Each of these collection handle elements takes the service ID of a Mako specified as a "xml-data-service" and the name of the collection on the host where it resides (specified by the "collectionName" attribute).

The value of the "collectionName" attribute must specify a real collection on the host described by the "serviceId". Collection names do not have to be identical to the name of the virtual collection. In fact, it will usually be the case that the names are different. For example, you might set up a VMako to aggregate data from a number of differently purposed Makos: You want simultaneously to query references to a specific allele from a Makos referencing data from cancer genetics, proteomics, and SNP studies. You craft your single XPath query so that it returns the appropriate set of attributes from the multiple, separate, virtually aggregated stores.

Let's set up our example. In the localhost-vmako-config.xml file, change the <vmako-storage ...> element's "file" attribute so that it points at your cache. For example, ../conf/vmako-cache-example.xml.

Edit the vmako-cache-example.xml file and point the "serviceId" attribute within the <xml-data-service ...> at a remote Mako service. In the <xml-collection-handle ...> tag, repeat the service id and fill out the "collectionName" attribute with the XPath path to a collection on that server that you would like to aggregate under your VMako. In the <virtual-collection ...> tag, fill out the "name" attribute with a virtual collection name of your choosing (below we have used "TestVirtualCollection").

Restart the VMako, as above. You will see output similar to the following: 


INFO - Nov 5, 2004 1:02:27 PM  -- Adding service MAKO://dc01
INFO - Nov 5, 2004 1:02:27 PM  -- Adding service MAKO://dc02
INFO - Nov 5, 2004 1:02:28 PM  -- Adding the virtual collection TestVirtualCollection
INFO - Nov 5, 2004 1:02:28 PM  -- Server started on localhost at Fri Nov 05 13:02:28 EST 2004
INFO - Nov 5, 2004 1:02:28 PM  -- Setting Service Identifier to localhost
INFO - Nov 5, 2004 1:02:28 PM  -- Listening using the protocol TCP on the host 0.0.0.0, port 3940

Now, any XPath queries performed via command line or viewer against the TestVirtualCollection collection will, in fact, be run against the collections on the remote nodes you configured in the vmako-cache-example.xml file (in this case above, dc01 and dc02).

Mako Configuration File Settings

There are a number of configuration files available in the /conf directory. Each addresses a different Mako set-up—connecting Mako to other services, data stores, etc. The file localhost-mako-config.xml contains a basic setup for a Mako server to run on the local machine and will provide a good example of how the configuration files are structured. See the mobius configuration overview for information on the elements in the top resource block of the configuration files.

The localhost config files should never be used for more than single Mako testing purposes. If you are running multiple Makos, customize the config files by assigning unique service identifiers and making any other changes necessary to specify the service as unique. When other Makos connect to a Mako started as localhost, the Mako will identify itself as "localhost" to the connecting service. This can cause problems with service name resolution.

The Mako server contains two resources that provide certain functionality and information to other Mako components. These resources are instantiated by the Mako server at startup, and are configured by the config file. The two resources for the Mako are:

  1. Mako Configuration <resource name="makoConfig"... >
  2. Mako Database Configuration <resource name="makoDBConfig"... >

These resources must be specified in order in the config file due to dependencies on the other resources, and all of them must exist.

Mako Configuration

The Mako Configuration resource keeps track of the information specific to the Mako Server. This resource is defined under the <resource name="makoConfig" class="org.projectmobius.services. mako.MakoConfiguration"> element. This element has a child called <mako-configuration> that specifies the information Mako Configuration may potentially need to establish connections with GMEs —for example, to use a GME in order to obtain schema for the Mako to use to publish XML documents.

The <mako-configuration> element has one child, <gme-client-configuration>. It can have any number of <gme ... /> children, each of which has a single attribute that specifies the service ID of a GME the Mako is to contact when it starts up. When you're running Mako in a Grid environment, it will often be the case that you'll want to start your Mako and have it query a GME for schema to use.

The localhost-mako-gme-config.xml configuration file contains an example of how you would specify the service ID of a GME client to connect to on start up. Swap out the example service ID with an actual running GME service ID and start the Mako, passing it your modified config file. The Mako will then be able to use the schema stored in the GME to publish XML documents.

Mako Database Configuration

This resource contains information about the databases the Mako will use and enables access and modification of the databases. This resource will have various definitions depending on the sort of database or database connection service the Mako will use. The files in the /conf directory contain many examples of useful database configurations for Mako. Here is a sample:

  • in localhost-mako-xindice-config.xml and localhost-mako-exist-config.xml <resource name="xmlDBConfig" class="org.projectmobius.xmldb. XMLDBConfiguration"> is used for connecting to the Xindice or eXist XML databases
  • in localhost-mako-xbridge-config.xml <resource name="xbridgeConfig" class="org.projectmobius.xbridge. MakoXBridgeConfiguration"> is used for connecting to the XQuark Bridge XML database
  • in localhost-vmako-config.xml <resource name="vmakoConfig" class="org.projectmobius.services. vmako.VMakoConfiguration"> is used when we want to aggregate multiple Mako servers into a VMako (see our VMako QuickStart example)
  • and, in localhost-mako-exampleds-config.xml, you will see syntax for how it is possible use this resource to establish a connection to a dataservice for which you've written interface classes

In many cases, it will be adequate to use Mobius's own XML database service, MakoDB, for storing data. We will discuss the configuration elements for using MakoDB in more detail, below.

The resource for using MakoDB is defined under the element <resource name="makoDBConfig" class="org.projectmobius.services. makodb.MakoDBConfiguration"> The resource element has a child called <makoDB-configuration ...>. Its "id" attribute sets the name of the database in which the schema are stored. This is important because, if you want to run multiple Makos on a host and use MakoDB as the database services in them, you must make sure to start each Mako from a config file having different values for the "id". Otherwise, there will be database name collision within the MakoDBs.

The Mako Database Configuration (for MakoDB) has several important configurable elements:

<name>
This element specifies the name of the database. For MySQL's root database, this should be nothing.
<driver>
The driver class for accessing the MySQL database.
<urlPrefix>
The url prefix for accessing the database.
<host>
The host the database lives on. Usually, this will be localhost.
<port>
The port via which the database can be accessed.
<username>
The username to log into the database with. This username must have privileges to add and remove databases and tables.
<password>
The password to authenticate the username.
<pool>
This element determines how many connections to the database will be made initially. When the MakoDB needs to communicate with the MySQL database, it will get a connection, and, when its done, release it. Should it run out of available connections, it will make a new one, but this causes a slight delay. Set the pool value in anticipation of how many concurrent database operations will be needed.

Mako Application Programming Interfaces (APIs)

The Mako API is the XMLDataService API. It uses the base Mobius API as its foundation. The XMLDataService interface provides accessors that allow the execution of operations on Grid XML data services. The XMLDataService interface is an extension of the GridService interface (contained in the base API). It thus provides base grid service operations as well as XML related operations. Handles to XML data services can be created using the grid service resolver as follows: 


GridServiceResolver resolver = GridServiceResolver.getInstance();
GridService gs = resolver.getGridService("MAKO://localhost"); 
XMLDataService xds = (XMLDataService)gs;

An XML data service consists of a set of collections that contain XML documents and their corresponding schemas. XML collections are represented in the Mako API with the XMLCollection interface. When performing operations on a XMLDataService that returns or requires the use of an XML collection, the XMLCollection interface is used. The XMLCollection provides operations that can be performed on XML collections. Such operations include, but are not limited to, adding a schema, adding a document, and performing an XPath query.

XML collections consist of a collection of XML documents, which are represented in the Mako API by the XMLDocument interface. The XMLDocument interface provides accessors to operations that can be performed on XML documents.

Finally the MakoAPI consists of two other objects that represent components that could be returned in the results of a query. The XMLAttribute interface represents an attribute in an XML document and the XMLText object represents a CDATA section in an XML document.

The remainder of this page will be devoted to giving code examples for performing operations on XML data services. The code examples assume that the GridServiceResolver has been configured with a default GridServiceFactory, this can be done as follows: 


GridServiceResolver resolver = GridServiceResolver.getInstance();
resolver.setDefaultFactory(new NetworkServiceFactory());

The code snippet below shows how to get a handle to an XML Data Service and prints all the root level collections that exist on an XML data service. 


GridServiceResolver resolver = GridServiceResolver.getInstance();
GridService gs = resolver.getGridService("MAKO://localhost"); 
XMLDataService xds = (XMLDataService)gs; 
List collections = xds.getCollections();

for(int i=0; i<collections.size(); i++){
  XMLCollection col = (XMLCollection)collections;
  System.out.println(col.getName());
}

The following code shows an example of how to print out all the sub-collections for a collection, called "myCollection". 


GridServiceResolver resolver = GridServiceResolver.getInstance();
GridService gs = resolver.getGridService("MAKO://localhost"); 
XMLDataService xds = (XMLDataService)gs; 
XMLCollection collection = xds.getCollection("/myCollection");
List collections = collection.getSubCollections();

for(int i=0; i<collections.size(); i++){
  XMLCollection col = (XMLCollection)collections;
  System.out.println(col.getName());
}

The code snippet below shows how to create a collection on an XML data service. 


GridServiceResolver resolver = GridServiceResolver.getInstance();
GridService gs = resolver.getGridService("MAKO://localhost"); 
XMLDataService xds = (XMLDataService)gs; 
XMLCollection collection = xds.createCollection("/myCollection");

The code snippet below shows how to create a sub-collection on collection. 


GridServiceResolver resolver = GridServiceResolver.getInstance();
GridService gs = resolver.getGridService("MAKO://localhost"); 
XMLDataService xds = (XMLDataService)gs; 
XMLCollection collection = xds.getCollection(/"myCollection"");
XMLCollection subCol = collection.createSubCollection("mySubCollection"");

The code snippet below shows how to remove a collection from a XML data service. 


GridServiceResolver resolver = GridServiceResolver.getInstance();
GridService gs = resolver.getGridService("MAKO://localhost"); 
XMLDataService xds = (XMLDataService)gs; 
xds.removeCollection("/myCollection");

The code snippet below shows how to remove a sub-collection from a collection. 


GridServiceResolver resolver = GridServiceResolver.getInstance();
GridService gs = resolver.getGridService(""MAKO://localhost"); 
XMLDataService xds = (XMLDataService)gs; 
XMLCollection collection = xds.getCollection("/myCollection");
collection.removeSubCollection("mySubCollection");

Below is an example for adding a schema to an XML collection. 


GridServiceResolver resolver = GridServiceResolver.getInstance();
GridService gs = resolver.getGridService("MAKO://localhost"); 
XMLDataService xds = (XMLDataService)gs; 
XMLCollection collection = xds.getCollection("/myCollection");
collection.addSchema(schemaText);

The following code illustrates how to submit an XML document to an XML collection. 


GridServiceResolver resolver = GridServiceResolver.getInstance();
GridService gs = resolver.getGridService("MAKO://localhost"); 
XMLDataService xds = (XMLDataService)gs; 
XMLCollection collection = xds.getCollection("/myCollection");
collection.addDocument(xmlText);

XML Documents and, in most cases, elements within documents are all assigned unique identifiers within a Mako. This provides the ability for users to support retrieval of subsets of documents or even request partial documents. These unique identifies are referred to as document references. Document references are compromised of a three tuple identifier consisting of the service id of the Grid service it resides on, the collection that the XML entity resides in, and the unique identifier assigned to it by a Mako. A document reference is a unique identifier within a grid.

Documents can be retrieved from a Mako by requesting them directly or through a query such as XPath. A subset of an XML document can be retrieved by requesting the sub entity by its document reference. Partial documents can be retrieved by requesting the XML depth of the XML document. For example if a depth of 2 is requested, the root element and it children will be returned with document references to its children's children. This feature allows clients to work efficiently with large documents.

Mako can also distribute a single XML document across multiple Makos. When retrieving such documents one may specify whether or not entities on other Makos should be resolved. If they are not resolved, Document References are included instead.

Below we show an example of retrieving an XML document from a Mako. The XMLRetrievalPolicy object is used to specify how the document is to be retrieved. The example below retrieves the entire XML document. For a custom materialization, one could create a custom XMLRetrievalPolicy. 


GridServiceResolver resolver = GridServiceResolver.getInstance();
GridService gs = resolver.getGridService("MAKO://localhost"); 
XMLDataService xds = (XMLDataService)gs; 
XMLCollection collection = xds.getCollection("/myCollection");
XMLDocument doc = collection.getDocument("28000000000000001");
String xml = doc.getContent(XMLRetrievalPolicy.FULL_MATERIALIZATION);
System.out.println(xml);

A Mako can be queried using XPath. Like direct document retrieval, XPath document retrieval requires an XMLRetrievalPolicy to specify how the results are retrieved. Performing an XPath query also requires a NamespaceManager, which allows XML namespaces to be built into XPath queries. An XPath query returns a list of result objects. Depending on the query the result list will contain XMLDocument, XMLText, or XMLAttribute objects. Below is a code snippet illustrating how to perform an XPath query: 


GridServiceResolver resolver = GridServiceResolver.getInstance();
GridService gs = resolver.getGridService("MAKO://localhost");
XMLDataService xds = (XMLDataService) gs;
XMLCollection collection = xds.getCollection("/myCollection");
NamespaceManager nm = new NamespaceManager();
Namespace defaultNS = new Namespace("projectmobius.org/1/Person");
Namespace ns1 = new Namespace("projectmobius.org/1/Address");
nm.setDefaultNamespace(defaultNS);
nm.addNamespace("ns1", ns1);
String xpath = "/person/ns1:address";
List results = collection.performXPath(nm, xpath, XMLRetrievalPolicy.FULL_MATERIALIZATION);

for (int i=0; i<results.size(); i++) {
  XMLDocument doc = (XMLDocument) results.get(i);
  String xml = doc.getContent(XMLRetrievalPolicy.FULL_MATERIALIZATION);
  System.out.println(xml);
}

The code snippet below illustrates how to submit and then remove a document from a Mako. 


GridServiceResolver resolver = GridServiceResolver.getInstance();
GridService gs = resolver.getGridService("MAKO://localhost"); 
XMLDataService xds = (XMLDataService)gs; 
XMLCollection collection = xds.getCollection("/myCollection");
XMLDocument doc = collection.addDocument(xmlText);
String docId = doc.getDocumentId();
collection.removeDocument(docId,true);

The code snippet below illustrates how to remove XML documents from a Mako via an XPath. 


GridServiceResolver resolver = GridServiceResolver.getInstance();
GridService gs = resolver.getGridService("MAKO://localhost");
XMLDataService xds = (XMLDataService) gs;
XMLCollection collection = xds.getCollection("/myCollection");
NamespaceManager nm = new NamespaceManager();
Namespace defaultNS = new Namespace("projectmobius.org/1/Person");
nm.setDefaultNamespace(defaultNS);
String xpath = "/person";
collection.removeDocumentsByXPath(nm,xpath,true);

This code illustrates how to perform an XUpdate on a collection: 


GridServiceResolver resolver = GridServiceResolver.getInstance();
GridService gs = resolver.getGridService("MAKO://localhost");
XMLDataService xds = (XMLDataService) gs;
XMLCollection collection = xds.getCollection("/myCollection");
collection.performXUpdate(xupdateText);

Mako Command Line Utilities Reference

The command line utilities are located in the /scripts directory. Each uses the Mako API to implement common Mako functionality. All of the command line tools that run during service uptime take a -factory argument that allows you to specify the factory class that will be used to get handles to Mobius network services. Omitting this flag will cause the tools to use the default factory for handles. Each of these will also take a -factoryXML argument that specifies the path to a Grid Service Factory configuration file, an example of which is given here.

Passing the -help argument will list the utility's required and optional arguments. Scripts ending .bat are for Windows, .sh for *nix.

Start a Mako
Starts a Mako server, parsing the config file passed in. Starting a Mako is a generic action, the specifics of which are determined by your config file.
Script name: startMako
Arguments:
Path to config file (required)
Create a Collection
This script adds a collection to the data service.
Script name: createCollection
Arguments:
-collection Name of collection (required)
-services List of data services to create the collection on (required)
Submit a Schema
This script submits an XML Schema to a Mako.
Script name: submitSchema
Arguments:
-collection Collection the schema belongs to (required)
-xsd Schema file name (required)
-services Service IDs of the Makos to send the schema to (required)
Submit Multiple Schemata
This script allows for batch submission of XML Schemata to a Mako.
Script name: submitSchemata
Arguments:
-collection Collection to submit schemata to (required)
-file Schema file name to be added
-service Service ID of the Mako to send the schemata to (required) -directory Path to a directory containing .xsd schema descriptions to be added
Submit XML
This script allows the submission of an XML document or a directory of XML documents to a collection existing on a Mako.
Script name: submitXML
Arguments:
-collection The collection to sumbit XML to (required)
-file A single XML file to sumbit
-service The service ID of the Mako (required)
-directory A directory full of XML files to sumbit
Submit an XPath Query
This script allows the execution of a XPath query on an XML collection within a Mako. The script allows you to specify how many levels of XML data to return. For example specifying 2 levels will return the root node and the root nodes children with a pointer to the root node's children's children. Specifying a level of 0 will return pointers or references to all the documents satisfying the query. The pointers can later be used to materialize the results. materialize refers to the documents in a Mako which are lazy loaded. They can be either materialized (displayed in full content) or lower levels can be represented by "DocumentReferences". A DocumentReference is a pointer to the actual document. It requires less overhead to return than the actual document. levels takes an integer and represents the number of levels below the root (target of the XPath) you wish to materialize.
Script name: submitXPath
Arguments:
-collection The collection to use (required)
-materialize Whether or not to materialize included XML (true or false)
-levels How many levels of includes to materialize
-file An XPath file
-service The service id of the Mako (required)
-query An XPath query
-namespaces A list of prefix=namespace definitions which define the prefixes you can use in XPath elements
-defaultNamespace The namespace to use when no prefix is used for an XPath element
Modify Existing XML
Using the XUpdate language, the XML in the database can be modified. The most flexible way to use this command is pass in the query via the file flag and put the query into a seperate file. Note: The rename command and the comment node type are not supported.
Script name: submitXUpdate
Arguments:
-collection The collection name (required)
-file An XUpdate file
-service The service ID of the Mako (required)
-query An XUpdate statement
Remove a Schema
Schema removal is not supported, as schemas are built into the database. To remove a schema, you must remove the collection and recreate it fresh with the new schema.
List Collections
Prints out a list of the collections on the specified data service.
Script name: getCollectionList
Arguments:
-collection Name of the parent collection
-service Service ID of the Mako to get the collection list from (required)
Retrieve Schema
Prints a list of schemas supported in the specified collection.
Script name: getSchemaList
Arguments:
-collection The name of the collection (required)
-service The ID of the Mako service (required)
Retrieve XML Documents
This script retrieves XML documents or references to XML documents from a collection on a specified Mako. The script allows you to specify how many levels of XML documents to return. For example specifying 2 levels will return the root document and the root document's children with a pointer to the root document's children's children. Specifying a level of 0 will return pointers or references to all the documents requested. The pointers can later be used to materialize (display full contents of) these documents. Documents can be either materialized or be represented by "DocumentReferences". A DocumentReference is a pointer to the actual document. It requires less overhead to return than the actual document. levels takes an integer and represents the number of levels below the root document you wish to materialize.
Script name: retrieveXML
Arguments:
-collection The name of the collection (required)
-materialize Materialize the returned XML documents (true or false)
-levels How many levels of documents to materialize
-service The ID of the Mako service (required)
-xmlids The IDs of the XML document(s) to retrieve (required)
Submit and Retrieve XML
In a single command, this script submits a number of XML documents and then retrieves all of them.
Script name: submitAndRetrieve
Arguments:
-collection The collection to submit documents to (required)
-outfile A path to an output file or directory (required)
-service The service ID of the Mako (required)
-times The number of times to retreive the submitted documents (required)
-directory The directory containing the XML documents to submit (required)
Retrieve Service Information
Prints the service data of the data service specified.
Script name: getServiceData
Arguments:
-onlyXML Return the raw XML with no formatting (true or false)
-service The service ID of the Mako (required)
Remove a Collection
Removes a collection and all of its data from the data service.
Script name: removeCollection
Arguments:
-collection The collection to remove (required)
-service Service ID(s) of the Mako(s) (required)
Remove XML
Clears a collection of the specified items, whether they are docIds or XPaths. The wait boolean ensures proper removal occurs because it waits for the database to do the actual data removal.
Script name: removeXML
Arguments:
-collection The collection to remove documents from (required)
-docs Document Ids to remove
-service The service ID of the Mako (required)
-xpath Xpath by which documents will be removed
-wait Wait for each document to be removed (true or false) (required) -namespaces A list of prefix=namespace definitions which define the prefixes you can use in XPath elements
-defaultNamespace The namespace to use when no prefix is used for an XPath element
Make a Configuration File
A replacement script that changes the localhost config files into the config file of the same type with the appropriate dataservices changed.
Script name: makoConfigurationMaker
Arguments:
-template The template configuration file to build from. (required)
-output Directory to write new config files into
-hosts Hostnames to make config files for (required)
Destroy a Mako
Destroys a specified Mako or list of Makos, killing the processes and removing database schemata.
Script name: destroyMako
Arguments:
-services The service IDs of Makos to destroy (required)

GME QuickStart, Config File, APIs, Command Line Utilities [top]

Global Model Exchange (GME) QuickStart

Introduction

GME is a DNS-like data definintion registry and exchange service that can run in a Grid/distributed computing environment. GME is responsible for storing and linking together data models, meta-descriptions of data in the form of XML schema. It enables other services to publish, retrieve, discover, and, remove data models. GME services are arrayed via a DNS-like architecture, in which parent-child namespaces organize the connection of GME nodes into a hierarchical tree structure.

Minimal Configuration

If you have not already done so, install the Mobius software and its prerequisites.

To demonstrate basic use of GME, we'll edit the localhost-gme-config.xml in the /conf directory. There is also a localhost-gme-registrar-config.xml file, which serves as an example for how to connect a GME to a Registrar service. We use this file for a later QuickStart demonstration.

Note that localhost config files should not be used for anything other than single GME instance testing. When other GMEs attempt to connect to instances running as localhost, they will be unable to satisfactorily resolve the name and GME will not function properly.

For this QuickStart, we'll be running GME nodes on multiple, remote machines and accessing the nodes from the local machine. This will give a truer picture of how GME is used in practice. If you don't have access to other machines, you can follow this QuickStart, starting a GME using the localhost-gme-config.xml config file.

Change to the conf directory in the mobius_gme... subdirectory of the gme directory and open the localhost-gme-config.xml file in an editor.

Put in your MySQL "root" or "admin" password and change the "username" as appropriate within the <database> section of the <resource name="gmeConfig"...> resource block (the second major element in the config file).

GME config files contain a lot of parameters. There are two key resource blocks:

  1. Mobius Configuration
    <resource name="mobiusConfig">, which defines the Network Service Descriptor the service will use when identifying itself on the grid, how to connect to the grid, what requests can be handled, and how to talk to other services
  2. GME Configuration
    <resource name="gmeConfig" ... >, which defines things specific to the GME Server, including namespaces, how to find authoritative and subordinate related GMEs, server policies, and database connection information

Log onto a number of remote machines and edit the localhost-gme-config.xml files on those machines, as well. On the remote machines, in addition to any necessary modifications to the MySQL username and password, you will also want to edit two other parameters.

In the first resource block, on each remote machine:

  • In the tag <MobiusNetworkServiceDescriptor ...> change the "hostname" attribute value to the fully qualified domain name of the remote machine.
  • In the <serviceIdentifier ...> change the "serviceId" to a host-resolvable name, preceded by GME://. The service id must be unique within the set of GMEs that you instantiate, that is, it must be unique within your "Grid". In the examples in this QuickStart, our ids match the names of the machines on which we run the GMEs.

Note: The two requirements for service ids are that they be unique within your Grid and that they be resolvable by the other Grid services that will connect to them.

For this example, we're using a minimal set up and default parameters for everything else. The configuration files are discussed in more detail on the gme configuration page of this online documentation.

Starting the GME

Change to the mobius_gme.../scripts directory on each remote machine.

Type:
$ ./startGME.bat ../conf/localhost-gme-config.xml &
You should see output like the following: 


INFO - Dec 14, 2004 5:09:26 PM  -- Creating Database: Kensdc02MobiusGME_GME_REGISTRY
INFO - Dec 14, 2004 5:09:26 PM  -- Creating database: Kensdc02MobiusGME_GME_SCHEMA_STORE
INFO - Dec 14, 2004 5:09:27 PM  -- Server started on GME://kensdc02gme at Tue Dec 14 17:09:27 EST 2004
INFO - Dec 14, 2004 5:09:27 PM  -- Setting Service Identifier to GME://kensdc02gme
INFO - Dec 14, 2004 5:09:27 PM  -- Listening using the protocol TCP on the host 0.0.0.0, port 1111

You will see that MySQL now has (at least) two new databases:
mobiusgme_gme_registry contains the descriptors, namespaces, and other information relevant to GMEs registered in Mobius.
mobiusgme_gme_schema_store contains all of the schema registered with the various GME instances.

GME has a large complement of command line tools to, among other things, add authorities, add namespaces, resolve namespaces, list and publish schemas, destroy GMEs, and check the services' status. These tools are discussed in the GME command line utilities page of this online documentation. We will be using the Java GUI GME View to perform many of these functions.

Starting GME View

On the local machine, in the /scripts directory, type:
$ ./GMEView.bat

Note: Under the "File" menu, you will see "Save Current Configuration". Selecting that option at any time during your GME View session will open a dialog window allowing you to save your session information--the list of GMEs you have added, any other special settings you have established. In a subsequent session, you can pass GME View your previously saved session file, like so:
./GMEView.bat ../config/your_session_parameters
and GME View will load all of the GMEs from your previous session.

Right-click on the "Mobius Grid Services" icon and select "Add Grid Service". In the "Grid Service Id" field in the dialog window, type in GME:// followed by the host-resolvable name of the GME to which you want to connect, for example, GME://dc02.bmi.ohio-state.edu, and click "Okay." For this QuickStart, we've added three remote hosts. Each was added in a similar manner--by specifying a resolvable hostname upon which a GME is running. (See Fig 1.)

You can add subordinate GMEs to any running GME by right-clicking on its icon and choosing "Add XML Data Model Service." Adding a GME to an already running GME instance adds the new GME as a subordinate, making the already running GME its parent.

screen shot of GME View GUI tool, showing running GMEs and menus
Fig 1. GME View with three remote GMEs running, menu expanded

Removing a GME is a simple operation. Right-click on its icon and select "Remove XML Data Model Service." Select "OK" in the confirmation dialog that appears. Removing a GME does not stop it but does do a number of tear down operations behind the scenes to correctly set authorities and subordinates on the involved nodes. When you remove a node, its subordinates are removed, as well.

Let's go through a number of typical operations. We'll add and remove a namespace, attempt to resolve a namespace, try to lookup authoritative namespaces, and publish and request a schema. Configure your Grid of GMEs as depicted above, with a single parent node and two subordinate nodes.

Add a Namespace
Right-click on the parent GME icon and select "Add Namespace." Put in a namespace that you wish to establish as authoritative. Let's put in projectmobius.org. The GME will now become the authority for that namespace. Whenever you add a namespace, the authority for your GME will be notified of the new namespace ownership. For proper resolution, namespaces must be unique in the tree of GMEs. Adding a namespace that is already taken will produce an error. Also, any namespaces you add to child nodes must be subnamespaces of a parent. For example, if the parent is projectmobius.org, a child node's namespace must be a subnamespace, e.g., childof.projectmobius.org. For this QuickStart, go ahead and add the projectmobius.org namespace to the parent node and the childof.projectmobius.org namespace to a subordinate node.
Remove a Namespace
Right-click the GME icon for which you want to remove the namespace. Select "Remove Namespace" and okay the dialog. Closing the dialog without clicking "Okay" will not commit any changes. You are prevented from removing namespaces on any node for which there are subordinate nodes that contain subordinate/related namespaces. For example, if the parent node had a namespace of projectmobius.org and one of its children had childof.projectmobius.org for a namespace, you would be prevented from removing the parent's projectmobius.org namespace.
Resolve a Namespace
To discover which GME is the authority for any known namespace, right-click any GME icon and select "Resolve Namespace". A dialog window asks for the namespace to resolve and lists the GME that will be queried. Type in the namespace and click "Ok". Namespace resolution requests first move up the tree from the requesting node to its authorities. If no node is found with that namespace, the search procedes down the tree. Either a dialog indicating the node that is the authority of the queried namespace will appear or you will get an error message telling you the namespace cannot be located. Namespaces can only be resolved within the branches of the tree on which the search was initiated. Searches are not global for all GMEs, only those within the connected tree of GMEs. For this QuickStart, you should be able to lookup projectmobius.org and childof.projectmobius.org from any GME node.
Lookup Autoritative Namespaces
To see which namespaces a GME has authority for, right-click on the GME icon and select "Lookup Authoritative Namespaces." This will produce a dialog containing a list of all the namespaces the GME has authority for. For this QuickStart test each of the nodes in your tree to verify the namespaces each has authority for.
Publish a Schema
Data in Mobius are described by XML schemas. GME is a service for storing schemas so that they can be discovered and used by Mobius services. To publish a schema, right-click a GME node icon and select "Publish Schema". Note that schema cannot be removed from the GME, though they can be re-added. Attempting to add a schema that has already been published will produce an error. The inability to remove schema prevents the accidental obsoleting of XML data that might be dependant on the schema. Let's add the schema describing a bookstore. You'll find this schema in the /xml directory. Use the "Browse" button to locate the directory and select and "Open" bookstore.xsd. The path will appear in the "Publish To Service" field. Select "OK". Note that we would not be able to add this schema unless the projectmobius.org namespace had already been registered and the bookstore.xsd schema was part of that namespace.
Request a Schema
To retrieve a schema published on a GME, right-click a GME and select "Request Schema". In the "Namespace" dropdown, choose a namespace from the list. The list is populated only by namespaces for which your node has authority. The "Schema" dropdown will also only list schemas that are available for the chosen namespace. To retrieve the a selected schema, click "Ok". This ability to retrieve schemas can be very useful. For example, you might copy the text out of this window and use it to create a new schema that builds on or modifies the retrieved one.
Request References to All Schema Containing an Element
There will be cases in which you will want to see which schemas contain elements with the same names. For example, you might want a quick way to determine if any of the schemas stored by a GME contain "Book" elements so that you can examine those schemas and see which is the appropriate one for describing the sort of book you want to store or search for. To retrieve references to all schemas within a GME that contain a particular named element, right-click a GME and select "Get Schema Element References" from the dropdown. In the dialog, fill in the the name of the element you want to search for. You can narrow your search by specifying the type of element ("Simple" or "Complex") in the "Element Type" option field. Click "Get References". The "Referencing Schemas" text area will be populated with XML indicating the name of the schema in which your searched for element occurs. See Fig 2., below, which shows that the "Title" is an element within the "BookStore" schema. screen shot of GME View result window, showing the name of the schema in which the requested element occurs
Fig 2. Result of a query for a particular schema element

GME Configuration File Settings

There are a number of configuration files available in the /conf directory. Each addresses a different GME set-up—connecting GME to other services, etc. The file localhost-gme-config.xml contains a basic setup for a GME server to run on the local machine and will provide a good example of how the configuration files are structured. See the mobius configuration overview for information on the elements in the top resource block of the configuration files.

The localhost config files should never be used for more than single GME testing purposes. If you are running multiple GMEs, customize the config files by assigning unique service identifiers and making any other changes necessary to specify the service as unique. When other services connect to a GME started as localhost, the GME will identify itself as "localhost" to the connecting service. This can cause problems with service name resolution.

The GME server contains a single resource block that provides certain functionality and information to other GME components. The resources defined in the block are instantiated by the GME server at startup and are configured by the config file. The resource for the GME is:

  1. GME Configuration <resource name="gmeConfig"... >

The element <gmeConfiguration-configuration>, the outermost element of the resource, contains three sub-elements. Each is listed here, along with its children. Below this list we have elaborated the purpose of the elements.

  1. managementDatabaseFactories
    • authorityManagerDatabaseFactory
    • subordinateManagerDatabaseFactory
  2. policies
    • performance-caching
    • notification-policy
  3. root-database
<managementDatabaseFactories>
This element is a container for the elements that define how the GME will determine which related GMEs are authoritative and/or subordinate to it.
<authorityManagerDatabaseFactory>
The Authority Manager keeps track of information about authorities for the GME server. The class attribute specifies the factory used for creating an instance of the Authority Manager Database.
<subordinateManagerDatabaseFactory>
The Subordinate Manager keeps track of information about subordinate GMEs for the GME server. The class attribute specifies the factory used when creating an instance of the Subordinate Manager Database.
<policies>
This element establishes parameters for how long the GME should keep old data and what other hosts it should notify of its existence.
<performance-caching>
The <namespace-caching> and <schema-caching> sub-elements of this element tell the GME how much namespace and schema data it should cache and for how long it should maintain the cache.
<notification-policy>
This element contains a <notification-list> sub-element that specifies which running GMEs it should notify upon instantiation.
<root-database>
This element configures the MySQL root database information. The children of this element configure the database. Its "id" attribute is the base name of the MySQL databases that will be created and used by the GME. If you configure multiple GMEs to use a single MySQL installation, be sure to give a unique value to the "id" for each GME or there will be problems with name collision in MySQL.
<name>
This element specifies the name of the database. For MySQL's root database, this should be nothing.
<driver>
The driver class for accessing the MySQL database.
<urlPrefix>
The url prefix for accessing the database.
<host>
The host the database lives on. Usually, this will be localhost.
<port>
The port via which the database can be accessed.
<username>
The username to log into the database with. This username must have privileges to create and delete databases and tables.
<password>
The password to authenticate the username.
<pool>
This element determines how many connections to the database will be made initially. When the GME needs to communicate with the database, it will get a connection, and when its done, it releases it. Should the Database Manager run out of available connections, it will make a new one, but this causes a slight delay. Set the pool value in anticipation of how many concurrent database operations will be needed.

GME Application Programming Interfaces (APIs)

coming soon...

GME Command Line Utilities Reference

The command line utilities are located in the /scripts directory. Each uses the GME API to implement common GME functionality. All of the command line tools that run during service uptime take a -factory argument that allows you to specify the factory class that will be used to get handles to Mobius network services. Omitting this flag will cause the tools to use the default factory for handles. Each of these will also take a -factoryXML argument that specifies the path to a Grid Service Factory configuration file, an example of which is given here.

Passing the -help argument will list the utility's required and optional arguments. Scripts ending .bat are for Windows, .sh for *nix.

Start a GME
Starts a GME server, parsing the config file passed in. Starting a GME is a generic action, the specifics of which are determined by your config file.
Script Name: startMako
Arguments:
Path to config file (required)
Add An Authority
This script will add an authority to a GME.
Script name: addAuthority
Arguments:
-authority The service ID of the GME to add as an authority (required)
-service The service ID of the GME (required)
Add A Namespace
This script will add a namespace to a GME.
Script name: addNamespace
Arguments:
-namespace The namespace to add (required)
-service The service ID of the GME (required)
Add A Subordinate
This script will add a subordinate to a GME.
Script name: addSubordinate
Arguments:
-subordinate Service ID of the GME to be a subordinate (required)
-service Service ID for the GME (required)
Add A Subordinate Namespace
This script will add a namespace that a subordinate has authority over to a GME.
Script name: addSubordinateNamespace
Arguments:
-namespace The namespace to add (required)
-subordinate The service ID of the subordinate GME (required)
-service The service ID of the GME (required)
Get Authoritative Namespaces
This script will give you a list of the namespaces that a GME has authority over.
Script name: getAuthoritativeNamespaces
Arguments:
-service The service ID of the GME (required)
Get Authorities
This script will give you a list of Mobius Network Service Handles describing each GME that is an authority of the specified one.
Script name: getAuthorities
Arguments:
-service The service ID of the GME (required)
Get Subordinates
This script will give you a list of Mobius Network Service Handles describing each GME that is a subordinate of the specified one.
Script name: getSubordinates
Arguments:
-service The service ID of the GME (required)
Publish A Schema
This script will publish a schema to a GME.
Script name: publishSchema
Arguments:
-service The service ID of the GME (required)
-schema The xsd schema file name (required)
Request A Schema
This script requests a schema from a GME and prints its text to the console.
Script name: requestSchema
Arguments:
-service The service ID of the GME (required)
-uri The uri of the schema (required)
Request A Schema and All the Schemas it References
This script requests a schema from the GME and prints its text to the console. It also retrieves all the schemas that are referenced, and all the schemas that are referenced by the referenced schemas, ad infinitum.
Script name: getSchemaAndReferenced
Arguments:
-uri The schema URI to retrieve (required)
-service The Service ID of the GME (required)
Request Schema Element References
This script produces a list of schema element references for a GME.
Script name: getSchemaElementRefs
Arguments:
-name The schema element name to list (required)
-type The element type
-service The Service ID of the GME (required)
Resolve A Namespace
This script will attempt to resolve the authority for a give namespace.
Script name: resolveNamespace
Arguments:
-namespace The namespace to resolve (required)
-service The service ID of the GME (required)
List Schemas For A Namespace
This script will print out a list of schema URIs for a given namespace on a GME.
Script name: schemaList
Arguments:
-namespace The namespace to get schema URIs from (required)
-service The service ID of the GME (required)
Check Status of the GME
This script will request the status of a GME and print some useful information (uptime, service descriptor, supported requests).
Script name: statusCheck
Arguments:
-service The service ID of the GME (required)
Stop the GME
This script stops one or more GMEs.
Script name: stopGME
Arguments:
-services The service IDs of the GMEs to stop (required)
Destroy the GME
Under certain circumstances, you may wish to completly throw away the GME and all of its information and start over. This script causes the GME to drop its databases and exit, wiping out any schemata, namespace, or authority/subordinate information it may have previously held.
Script name: destroyGME
Arguments:
-services The service IDs of the GMEs to destroy (required)

Service Registry (Registrar) QuickStart, Config File, APIs, Command Line Utilities [top]

Service Registry (Registrar) QuickStart

The Service Registry allows you to register running Mobius services with it, so that services can discover each other. For example, it is possible to use a Service Registry from inside the Mako and GME viewers to populate their lists of running collections and model exchange services. This ability is key to Grid computing. The Service Registry is a very basic service at present, however. It will be extended and improved in the future and we will be publishing guides to integrating Mobius with more robust service registry systems, such as The Globus Toolkit. Still, for basic discovery of Mobius services, the Service Registry is a viable tool.

There are two ways to use the Service Registry. You can either use the command line tools within the Service Registry to add running Mobius services to its repository. Or you can instantiate other Mobius services, such as the GME or Mako, so that they automatically register themselves with a running Service Registry. The latter procedure would be the one most commonly used in practice, but the former might be useful in cases in which a user wanted a means for aggregating a number already running, unregistered remote Mobius services for convenient discovery.

Minimal Configuration and Starting the Service Registry

If you have not already done so, install the Mobius software and its prerequisites.

As with other Mobius services, change to the /conf directory. Edit the <MobiusNetworkServiceDescriptor ...> and change the "hostname" as appropriate. Also assign the registry a "serviceId" in the <serviceIdentifier ... /> tag, for example, REGISTRAR://localhost, and put in the appropriate password for your MySQL database in the <password> tag pair.

From the /scripts directory:
./startRegistrar.bat ../conf/localhost-registrar-config.xml

Registering a Service Using Service Registry Command Line Utilities

Start a GME and a Mako. For this example, we have started them on the local machine.

Now let's register the GME and Mako:


./registerService.bat -ports TCP,3940 -host MAKO://localhost -registry REGISTRAR://localhost -type MAKO
./registerService.bat -ports TCP,1111 -host GME://localhost -registry REGISTRAR://localhost -type GME

If the Mako and GME were successfully registered. You'll see output like the following on the consoles where those services were started. 


INFO - Nov 2, 2004 3:19:33 PM  -- 1 Received a connection from REGISTRAR://localhost:2055
INFO - Nov 2, 2004 3:20:03 PM  -- 2 Received a connection from REGISTRAR://localhost:2065
INFO - Nov 2, 2004 3:20:33 PM  -- 3 Received a connection from REGISTRAR://localhost:2157

The Service Registry attempts to connect to its registered Mobius services every 30 seconds (by default, you can change the interval in the config file's pingInterval element). If it fails to contact a service after a number of attempts, the service's entry is dropped from its database. You can see a list of registered services by executing the following command from inside of the Service Registry /scripts directory:

./getServiceList.bat -registry REGISTRAR://localhost
will display information about the services added to the Service Registry and will show the automatically generated valid service ids which can be used to contact the services through the Service Registry.

Registering a Service Automatically When Starting a Mako or GME

In this example, we will start a Mako so that it registers itself with a running GME and Service Registry.

Start a Service Registry and a GME on any hosts. In the Mako /conf directory, edit the localhost-mako-gme-registrar-config.xml file.

For the Mako, you will need to edit the <MobiusNetworkServiceDescriptor... > and <serviceIdentifier... > tags, changing the "hostname" and "serviceId" values, as appropriate.

For the Service Registry, in the <xmlDataServiceFactory> element, edit the "serviceId" value in the <serviceRegistry... > tag and the "value" value in the <param name="registryServiceId"... /> parameter tag.

For the GME, in the <dataModelServiceFactory> element edit the "value" value in the <param name="registryServiceId"... /> parameter tag and in the <gme-client-configuration> element, edit the <gme serviceId=... /> value.

Finally, set the <password> for your MySQL database.

Start your Mako with:
./startMako.bat ../conf/localhost-mako-gme-registrar-config.xml

If everything went well, you will see output like the following:


INFO - Nov 4, 2004 3:13:07 PM  -- Server started on MAKO://localhost at Thu Nov 04 15:13:07 EST 2004
INFO - Nov 4, 2004 3:13:07 PM  -- Listening using the protocol TCP on the host 0.0.0.0, port 3940
INFO - Nov 4, 2004 3:13:07 PM  -- Setting Service Identifier to MAKO://localhost
INFO - Nov 4, 2004 3:13:22 PM  -- 1 Received a connection from REGISTRAR://localhost:1059
INFO - Nov 4, 2004 3:13:52 PM  -- 2 Received a connection from REGISTRAR://localhost:1074
Notice the two lines showing a connection to the Mako from the Service Registry, confirming that that Mako registered itself upon instantiation.

The section immediately below describes contacting running Mobius services through the Service Registry.

Service Registry (Registrar) Configuration File Settings

The file localhost-registrar-config.xml contains a basic setup for a Service Registry to run on the local machine and will provide a good example for how to structure and populate customized Service Registry config files. See the mobius configuration overview for information on the elements in the top resource block of this configuration file.

The Service Registry contains a single resource block that provides certain functionality and information to other Service Registry components. The resources defined in the block are instantiated by the Service Registry at startup and are configured by the config file. The resource for the Service Registry is:

  1. Registrar Configuration <resource name="registrarConfig"... >

The element <registrar-configuration>, the outermost element of the resource, has an "id" attribute, the value of which should be set to the name for your instance of the Service Registry. You should not run multiple Service Registries with the same names on the same host—each registry should have a unique name. The element contains two sub-elements. Each is listed here, along with its children. Below this list we have elaborated the purpose of the elements.

  1. database
    • name
    • driver
    • urlPrefix
    • host
    • port
    • username
    • password
    • pool
  2. pingInterval
<database>
This element configures the MySQL database information. The children of this element configure the database:
<name>
This element specifies the name of the database. For MySQL's root database, this should be nothing.
<driver>
The driver class for accessing the MySQL database.
<urlPrefix>
The url prefix for accessing the database.
<host>
The host the database lives on. Usually, this will be localhost.
<port>
The port via which the database can be accessed.
<username>
The username to log into the database with. This username must have privileges to create and delete databases and tables.
<password>
The password to authenticate the username.
<pool>
This element determines how many connections to the database will be made initially. When the Service Registry needs to communicate with the database, it will get a connection, and when its done, it releases it. Should the Database Manager run out of available connections, it will make a new one, but this causes a slight delay. Set the pool value in anticipation of how many concurrent database operations will be needed.
<pingInterval>
This element has one attribute, "milliseconds", which takes an integer value that determines how often the Service Registry should ping its registered services. The Service Registry will automatically drop any service registered with it that fails to respond to the ping.

Service Registry (Registrar) Application Programming Interfaces (APIs)

The Registrar API uses the base Mobius API as its foundation. The ServiceRegistry interface provides accessors that allow the execution of operations on Grid service registries. The ServiceRegistry interface is an extension of the GridService interface (contained in the base API), and thus provides base grid service operations as well as ServiceRegistry related operations.

Handles to Service Registry services can be created using the grid service resolver as follows. 


GridServiceResolver resolver = GridServiceResolver.getInstance();
GridService gs = resolver.getGridService("REGISTRAR://localhost"); 
ServiceRegistry reg = (ServiceRegistry)gs;

The current incarnation of the registrar is as a Mobius service for keeping track of Mobius services running in the Grid. The Registrar is a registry for Mobius services. Services and users can contact the Registrar to discover services of a particular type or to find out what ports and protocols services communicate on. Currently the registrar can only list services that can be described with a valid Mobius Network Service Descriptor. In future versions there are plans for supporting more generalized descriptors.

The remainder of this page is devoted to code examples of the Registrar API.

The following example illustrates how a service can be registered with the Registrar. 


Port port = new Port("TCP",3940);
NetworkServiceDescriptor nsd = new      
NetworkServiceDescriptor("MAKO://localhost","MAKO","localhost",port);
GridServiceResolver resolver = GridServiceResolver.getInstance();
GridService gs = resolver.getGridService("REGISTRAR://localhost"); 
ServiceRegistry reg = (ServiceRegistry)gs; 
reg.registerService(nsd);

The next example shows how to request that the Registrar remove a service entry. 


GridServiceResolver resolver = GridServiceResolver.getInstance();
GridService gs = resolver.getGridService("REGISTRAR://localhost"); 
ServiceRegistry reg = (ServiceRegistry)gs; 
reg.removeService("MAKO://localhost");

The following code example illustrates how to look up a service by serviceId. 


GridServiceResolver resolver = GridServiceResolver.getInstance();
GridService gs = resolver.getGridService("REGISTRAR://localhost"); 
ServiceRegistry reg = (ServiceRegistry)gs; 
NetworkServiceDescriptor des = reg.lookupService("MAKO://localhost");

The following code example illustrates how to look up a service by service type. 


ServiceRegistry handle =
  (ServiceRegistry)GridServiceResolver.getInstance().getGridService("REGISTRAR://localhost");
List services = handle.lookupServices("MAKO");
for (int i=0; i<services.size(); i++) {
  NetworkServiceDescriptor nsd = (NetworkServiceDescriptor) services.get(i);
  System.out.println(nsd.toString());
}

Service Registry (Registrar) Command Line Utilities Reference

The command line utilities are located in the /scripts directory. Each uses the Registrar API to implement common service registry functionality. All of the command line tools that run during service uptime take a -factory argument that allows you to specify the factory class that will be used to get handles to Mobius network services. Omitting this flag will cause the tools to use the default factory for handles. Each of these will also take a -factoryXML argument that specifies the path to a Grid Service Factory configuration file, an example of which is given here.

Passing the -help argument will list the utility's required and optional arguments. Scripts ending .bat are for Windows, .sh for *nix.

Start a Service Registry
This script starts a Service Registry server, parsing the config file passed in. Starting a Service Registry is a generic action, the specifics of which are determined by your config file.
Script name: startRegistrar
Arguments:
Path to config file (required)
Get Service Data
This script prints