Welcome!

Government Cloud Authors: Elizabeth White, Pat Romanski, Dana Gardner, Liz McMillan, Gopala Krishna Behara

Related Topics: Industrial IoT

Industrial IoT: Article

Using OAGIS for Integration

Using OAGIS for Integration

The Open Applications Group Integration Specification (OAGIS) is implemented because it simply works - and it's available today. While it's not as "cool" as the Web services protocols, it does provide the missing piece for Web services.

Without OAGIS, a canonical business language that enables integration, these cool protocols would simply pass proprietary data. Yes, the data could be in XML, but there wouldn't be a common understanding of what the messages mean. OAGIS includes the most XML messages defined in any XML dialect, and more are being defined every day.

OAGIS Simply Works
OAGIS is the common content model needed to represent messages that enable communication between business applications, whether the messages are application-to-application (A2A), business-to-business (B2B), or vertical industry- to-vertical industry (V2V) in nature. In short, OAGIS enables everywhere-to-everywhere integration.

OAGIS Is the Content
To see how OAGIS works, let's use the mail analogy. If I send you a message using postal mail, I write the message in one of a few accepted formats. I then address an envelope with the sending and receiving addresses, put the message inside the envelope, seal the envelope, apply postage, and place it in the mailbox. My mail delivery person retrieves the package. It's routed through the mail system, and after some time it arrives at your address. You retrieve the package from your mailbox, open the envelope, and read the message. Assuming you and I understand the same format and speak the same language, you can understand the message.

If my application needs to send a message to you and your application(s), something similar takes place. My application creates a message in one of several formats. Next, the application or an agent for my applications wraps the message into a transport envelope (Web services, ebXML, RosettaNet Information Framework, or Message-Oriented Middleware [MOM]) and applies the sender's and receiver's addresses to the envelope. The package is placed on the transport mechanism, which routes the package to your application. Your application retrieves the package from the transport mechanism, opens the envelope, and reads the message. Again, assuming our applications understand the same format and speak the same language, our applications understand the message.

If both applications speak OAGIS, they can understand the message. The problem is that many applications use their own proprietary formats and languages for integration.

OAGIS provides a common data model that provides a common basis of understanding, allowing all the applications involved in the information exchange to understand the intent of the message. The messaging specifications are analogous to the envelope. The integration engine used to transport and route the message is analogous to the postal service. The message and message format are analogous to OAGIS, which enables the understanding of the information on each end.

Just as it doesn't matter what brand of envelope you use to send a message via postal mail, it doesn't matter what framework you use to carry your OAGIS messages. So, use the framework that makes the most sense for your business, your supply chain, or your integration. It doesn't matter if the exchange of information is A2A, B2B, or V2V in nature. It's always critical that both sides understand the message - if the receiver doesn't understand the purpose or the content of the message, it's worthless. In many integrations, messages have to be translated for each application, business, and vertical industry for which they're intended. Doing this in a point-to-point manner is messy, not to mention inefficient.

Let's assume that it takes one-tenth of a person's time to maintain each integration mapping. (By all estimates this is extremely low, but it makes the point.) The formula for point-to-point integrations is n(n-1).

Using a canonical business language to translate to and from, it's possible to use a common format in the center. The formula for a canonical integration is n * 2. If we use the same one-tenth of a person's time to maintain each integration mapping, it's easy to see the savings possible. Figure 1 shows the number of people needed to support each integration model as the number of interfaces grows. The math speaks for itself.

OAGIS can be used with any framework and integration engine to communicate information between applications, businesses, supply chains, and vertical industries. The natural next questions are, "Where do I get OAGIS?" and "How do I get started using OAGIS?".

Getting Started
Where to get OAGIS

OAGIS is freely available from the Open Applications Group, at www.openapplications.org. Follow these steps to download OAGIS:
1.   Click on the Free Downloads button.
2.   Click on the OAGIS or OAGIS Archives link, depending on which version of OAGIS you're interested in. The archives page gives you access to OAGIS releases all the way back to OAGIS 6.0, the first XML release based on DTDs.
3.   Fill in the requested information on the form; click the Submit button at the bottom of the form.
4.   For OAGIS 8.x, simply click on the OAGIS 8.x link. (OAGIS 8.x includes all the documentation schemas and examples of Business Object Document [BOD] instances in a single, self-contained zip file.) Prior to OAGIS 8.0, the OAGIS documentation and the different instantiations of OAGIS were distributed in separate zip files. Simply download the version and instantiation of OAGIS that you need along with its supporting documentation.

If you're new to OAGIS and this is your first implementation, it's best to start with the most recent version of OAGIS. If you're supporting existing versions, you'll want to upgrade, just as you would with any software package. We'll assume OAGIS 8.0 as it is, at the time of this writing, the most recent release of OAGIS.
5.   Unzip the zip file, making sure to maintain the directory structure contained in the zip file.

Congratulations, you've downloaded and installed OAGIS.

Navigating OAGIS directories
Now that you've installed OAGIS, let's look around a bit at what is installed in the OAGIS 8.0 directory from the OAGIS 8.0 distribution. Inside the OAGIS 8.0 directory you'll find the following directories and files:

  • Documentation folder: Contains all of the OAGIS documentation for OAGIS 8.0.
  • OverlayExamples directory: Contains contrived examples of extending OAGIS.
  • Utility directory: Contains several scripts used by the OAGI architects in the creation of OAGIS 8.0. They are provided here as is. These can be helpful in testing overlays of OAGIS. Utility also contains other tools that were used in the creation of OAGIS 8.0.
  • index.html page: The starting point for the documentation. This is where we will go next.
  • license.txt: Contains the licensing information.
  • OAGIS8.0.spp: An XML Spy Project file to help navigate the OAGIS XML Schema files. Other XML Integration Development Environment (IDE) project files will be provided in future releases. They also may be available from the OAGIS 8.0 home site.
  • Readme.txt file: To get started, read the Readme.txt file to see what updates have been made to any new release and see any known bugs. From here open the OAGIS directory, where the XML Schema for the specification is contained.
  • OAGIS directory: Contains the XML Schema files and BOD example instances for OAGIS.
  • BODConstraints directory: Where all of the cardinality constraints for the OAGIS BODs are kept. In a future release by co-occurrence constraints can be added. Inside of this directory are a Generated directory and a Rules directory.
    -Generated is the generated XSL that tests for the existence of fields in the XML instances.
    -Rules is the simple rules files that were used to capture the rules for the constraints that were then used to generate the generated files in the Generated directory.
  • BODExamples directory: Where the generate XML instances of the BODs are kept. These files are intended to show what an instance of each BOD may look like.
  • BODs directory: Where all of the BOD XML Schemas are located.
  • Resources directory: Where all of thecomponents used in OAGIS are kept.Resources contains:
    -Nouns directory, which contains all of the Nouns available in this release of OAGIS.
    -Verbs directory, which contains all of the Verbs available in this release of OAGIS.
    -Components.xsd file, which contains the components reused throughout OAGIS.
    -Enums.xsd file, which contains all of the enumerated type lists that are reusable through out OAGIS.
    -Fields.xsd file, which contains all of the types used for fields within OAGIS.
    -Meta.xsd file,which contains the types used to define the meta data model used with in OAGIS.
    -MfgComponents.xsd, which contains the manufacturing specific components.

    Steps for Integration
    Before you can start, you must know what you are integrating - identify the business applications and the components of each application to be integrated (the integration scenario). OAGIS includes 61 example business scenarios to get you started. Again, these are example scenarios, not a fixed set. OAGI does not assume that these 61 scenarios are the only integration scenarios that the BODs can be used within.

    Second, identify the messages that need to flow between the applications, along with the intent of each message. The OAGIS scenarios reference the OAGIS BODs that have been identified to communicate the needed information. Once a scenario has been identified as a close match, drill down and look at the BODs.

    Third, identify how you'll get access to the data. With knowledge of the business applications, you'll know how to best get access to the data.

    Fourth, make it happen. Create the integration code (if it doesn't already exist) that accesses the application APIs and formats the information into OAGIS BODs, define the routes to transport the information to its destination, and create the code (if it doesn't already exist) to load the information into the target system.

    Note: Instead of defining your own proprietary integration scenario and data formats, which won't work once you start trying to integrate with your extended supply chain, use a canonical business language that will allow you to reuse the same message definitions and scenarios whether you're inside your enterprise's four walls or integrating with your extended supply chain. You can take advantage of support for OAGIS. If you do have to create your own integration code, once you create a mapping for one BOD from one application, that same mapping can be reused for the same BOD coming from a different application. This allows you to leverage the canonical format in order to reduce the number of mapping points.

    In either case, as you identify your scenario, it's important not to bite off too large a piece of the integration at one time.

    Remember, the only way to eat an elephant is one bite at a time. So keep your pieces small and manageable. As you can see in Figure 2, OAGIS breaks the integration scenarios into small, manageable pieces. Viewed from end to end, the supply chain integration (manufacturing to purchasing, order management, billing, shipping, and financials) scenario is large; within OAGIS it's broken into several smaller scenarios that are easier the handle. This is often the case with OAGIS.

    Not only does this make the scenarios easier to handle, the smaller modularized scenarios can also be reused in larger scenarios, which means that once the exchange is defined in your integration engine, it can simply be reused wherever needed and does not have to be recoded multiple times.

    Note: OAGI understands that no matter how smart the developers of the specification are, there will always be additional scenarios in which the BODs may be used. This is especially true of a horizontal specification that enables industry vertical groups to layer their needs on top of it. While it may be possible to define a fixed set of integration scenarios with preset timeout limits, etc., in certain vertical industries this is not possible in a horizontal specification. OAGIS enables this capability for vertical industries so that these additional requirements can be defined in their overlay of OAGIS.

    Using OAGIS scenarios
    The scenarios identify the business applications and component modules being integrated, the messages, and the intended actions of those messages (in the BODs). The BODs define the data elements used to communicate the message. Once you know the business applications, the modules within each of these applications, and the information you want to communicate, simply look through the list of the OAGIS scenarios until you find one that matches or is similar to your particular needs. From this scenario you can drill down into each BOD to identify the content you need to communicate. Remember, the BODs have been defined with a base set of required information. Your applications and/or implementation may require additional fields to always be present. Address this by extending OAGIS via the UserArea or by using an overlay extension. Also, you may need to add additional messages (BODs) to a given scenario; these may already exist within OAGIS, or maybe in an area that OAGIS hasn't addressed. In these cases, you may need to define your own BODs or use BODs based upon other specifications.

    Note: In the case of human resource messages, OAGI has agreed to reference the work of HR-XML instead of reinventing the HR specifications that have already been defined. Likewise, HR-XML reference OAGIS for non-HR messages (e.g., purchase orders, invoices, etc.).

    In most cases, you'll find a scenario that is close to your particular integration needs. In cases in which you don't find a match in the OAGIS scenarios, take a look at the specifications from organizations that have partnered with OAGI for specific domains. If you still don't find a scenario that meets your needs, you'll need to define a new scenario. Often you can use existing scenarios as sub-scenarios. Similarly, it's often possible to reuse existing BODs within new scenarios. OAGI asks that anyone creating new scenarios or BODs bring them forward for possible inclusion in a future release of OAGIS.

    To see the OAGIS scenarios, simply open the OAGIS documentation by opening the index.html page at the top level of the OAGIS8.0 directory. Select the Scenarios link. From here simply scroll through the list of scenarios. A table of contents provides links to each scenario.

    Identifying the BODs
    Once the scenario is identified, drill into it by looking at the BODs indicated in the scenario that need to be passed between the applications. Do these messages make sense for your integration? Are they needed? For example, a few of the BODs in the scenario shown in Figure 2 are:

  • Process PurchaseOrder: From the external customer's order management system into the supplier's order management system.
  • SalesOrder: If items are to come from inventory, the SalesOrder is synchronized with the Shipping module, which in turn picks the items from inventory to ship, per the SyncSalesOrder instructions.

    By using the above BODs Figure 2 integrates the business models:

  • Supplier's Order Management system: Determines whether to buy, build, or use existing inventory to meet the order. In some cases it may choose to do all three.
  • Order Management system: If it decides to buy, a request is made to the Purchasing system to add a new PurchaseOrder (AddPurchaseOrder) for the items to be outsourced or purchased.
  • Production system: If items are to be manufactured, a request to create a production order is sent to the Production system.

    As you can see, there are many other messages that can be communicated in this scenario. In an integration it's necessary to review the intent of each BOD and compare it to what you need to do with your integration in order to verify the match.

    Note: OAGIS is defined as a horizontal specification. Because of this a term that you're familiar with may be called something else within OAGIS or within another industry. If possible, we should look past this, to the meaning of the message, component, or field. Where possible, we should agree to use the same names looking at the definitions of these elements.

    In some circumstances, it makes sense to use synonym names. We can do that, but we all must be aware of what we're doing - creating a message, component, or field that has context within a particular domain or industry vertical.

    Only by working together will vertical industries be able to agree on a common dialect for business integration.

    Reviewing the BODs
    After identifying the BODs to be used, drill into the BODs to ensure that the information you need to communicate is provided in them. Do this by reviewing the OAGIS documentation: select the OAGIS Documentation link from the home page. Next, find the BOD you're interested in. The OAGIS documentation includes a list of the BODs sorted alphabetically by Noun and by Verb.

    For the scenario, review each BOD by drilling down into each of its layers to determine whether the information you need to pass is present. The documentation for each field, component, and the corresponding type is provided both in the documentation and in the XML Schemas themselves. The HTML documentation and the annotation within the XML Schema will always match because the HTML documentation is generated from the XML Schema.

    This analysis can be done using spreadsheets or an XML analysis schema, anything that lets you see the fields, components, and structures and do a mapping from the application's format into the OAGIS BOD and from the OAGIS BOD into to the receiving application's structure. Once you've done your analysis for one Noun, it is done for the other BODs using that Noun, at least on the BOD side.

    One of the key benefits in the modularization of OAGIS is that once you've created code (classes) to consume or produce an OAGIS field, compound, component, or type, you can reuse that code (class) any time the field, compound, component, or type occurs again. This is very powerful, because OAGIS reuses fields, components, and types throughout. Once you've coded a few BODs, the others become easier to implement because you're reusing your previous work.

    BOD structure
    Every BOD has an ApplicationArea and a DataArea (see Figure 3). The ApplicationArea serves to:

  • Identify the sender of the message
  • Identify when the document was created
  • Provide authentication of the sender through the use of a digital signature, if applicable
  • Uniquely identify a BOD instance (The BODId field is the Globally Unique Identifier for the BOD instance.)

    The ApplicationArea includes the following elements: the sender of the message; the CreationDateTime; the signature; the BODId, the unique identifier for the BOD instance; and a UserArea.

    The DataArea contains the instance(s) of the data values for the business transactions. For example, the ProcessPurchaseOrder from our scenario earlier includes one Verb (Process), which can be applied to one or more Nouns (PurchaseOrder), as shown in Figure 4.

    As you can see in Figures 3 and 4, the data (the Noun) and the action (the Verb) have been separated in OAGIS 8.0. This means OAGIS has one definition for PurchaseOrder instead of one for each Verb that is used with PurchaseOrder, in this case eleven different copies of similar PurchaseOrders would have needed to be maintained. The action information for each BOD is contained in the Verb. The Verb uses XPath to reference the elements to be retrieved in the case of the Get and GetList BODs.

    The cardinality constraints are provided by the BODConstraints, again by using XPath to point the elements and by applying rules to check whether the elements exist in the instance. This is done using the files in the BOD Constraints/Rules directory from which the OAGI architects generated the BODConstraints/Generated directory. The BODConstraints/Generated directory contains the constraints in XSL, which can then be applied to the XML BOD instances.

    For more information on how the OAGIS BODs are structured, please see the OAGIS documentation Architecture link for a full description and definition of each element.

    Accessing the data
    Most business applications provide Application Programming Interfaces (APIs) that provide external systems access to their data. With the proliferation of XML, most APIs come in some form of XML. Some applications go as far as supporting the consumption and production of OAGIS BODs natively or through an add-on piece of software. For those applications that do not have APIs it is necessary to access the data through whatever means possible. This may include screen-scraping or accessing the data directly from the application's database.

    Screen-scraping is the process of having an application that drives an application's graphical user interface (GUI) to provide the input and to capture the output of an application. While it's slow and error-prone and often specific to a version of the application's GUI, it does reuse the application's original logic and controls.

    Accessing the data directly bypasses the logic of the applications, grabs the data, and inserts it in the database. This is always dangerous because inserting data in one area of an application may generate additional data in another area, which must be replicated.

    For more information on accessing data within your business applications, consult an expert for each application being integrated to identify how to obtain the data for that application in the safest, most efficient manner possible.

    Implementing the Integration
    The scenario and the BODs have been identified, along with where the information is coming from and going to, and the analysis has been done to identify the fields and structures to be used to communicate the information. It's now time to implement the integration. Do this using the tools that meet your integration needs. Your choices include any of the following:

  • Enterprise Application Integration (EAI) tools and servers: These allow you to create a working graphical depiction of your integration, similar to that shown in Figure 5, where you can enable the flow of messages through the enterprise. Once you've identified the scenario and BOD, begin creating the scenario within the EAI tool - many include mapping tools that facilitate the mapping exercise. The maps produced are executable by the EAI engine either on the server side or by having adapters in front of the sending and receiving applications.
  • Web services: Web service development applications aid in the development of Web services for communicationg BOD messages.
  • Hand coding: While it's possible to code for yourself the functions of an EAI tool and a Web services application development kit, it's better for companies to focus on their core competencies. Unless your company is in the business, use off-the-shelf packages that meet your needs. This mapping code can take several forms: XSL translations for XML-to-XML mappings, scripts, or code written in any language (like Java). Most of the commercially available products give you options as to what this code can be.

    Producing BODs
    To produce a BOD, an application must provide the data in the format defined by the XML Schema; do this using an XML parser and driving the APIs that populate the XML DOM tree. The XML parsers provide a method to serialize the information out. Commercially available EAI, Web services application development kits, and the applications that support XML and OAGIS do this, translating the information from the internal formats into OAGIS.

    It's recommended that the application producing BODs verify that the BOD is valid. While it is possible to turn validation off after the application has demonstrated that it can reliably produce a valid BOD, it is not recommended.

    Eating BODs
    Similarly, to consume a BOD, an application will make use of an XML parser to read in and validate a message using the XML Schema definitions to validate the schema-level structure. An application also needs to use an XSL processor to verify that the BOD instance meets the BOD constraints.

    Remember that an XSL processor uses an XML parser underneath it. Because of this it's possible to do the XML Schema and BOD constraint validation in a single pass through an XSL processor (see Figure 5). Many commercially available off-the-self applications allow you to plug in your on validation capability to do this.

    Similarly, you can do the same type of thing using an SAX parser to handle larger BODs.

    Summary
    OAGIS, available today, is enabling the integration of small, medium, and large organizations and vertical industries. OAGIS increases the efficiency of integration implementations by allowing companies to shorten the time required to realize the return on investment. It has reduced the maintenance cost of these implementations by improving the efficiency of the integrations.

    In short, OAGIS is good for business, everyone's business! OAGIS is freely available from the Open Applications Group Web site at www.openapplications.org. If you'd like to learn more about OAGIS, take a look.

  • Comments (0)

    Share your thoughts on this story.

    Add your comment
    You must be signed in to add a comment. Sign-in | Register

    In accordance with our Comment Policy, we encourage comments that are on topic, relevant and to-the-point. We will remove comments that include profanity, personal attacks, racial slurs, threats of violence, or other inappropriate material that violates our Terms and Conditions, and will block users who make repeated violations. We ask all readers to expect diversity of opinion and to treat one another with dignity and respect.


    IoT & Smart Cities Stories
    The challenges of aggregating data from consumer-oriented devices, such as wearable technologies and smart thermostats, are fairly well-understood. However, there are a new set of challenges for IoT devices that generate megabytes or gigabytes of data per second. Certainly, the infrastructure will have to change, as those volumes of data will likely overwhelm the available bandwidth for aggregating the data into a central repository. Ochandarena discusses a whole new way to think about your next...
    DXWorldEXPO LLC announced today that Big Data Federation to Exhibit at the 22nd International CloudEXPO, colocated with DevOpsSUMMIT and DXWorldEXPO, November 12-13, 2018 in New York City. Big Data Federation, Inc. develops and applies artificial intelligence to predict financial and economic events that matter. The company uncovers patterns and precise drivers of performance and outcomes with the aid of machine-learning algorithms, big data, and fundamental analysis. Their products are deployed...
    Dynatrace is an application performance management software company with products for the information technology departments and digital business owners of medium and large businesses. Building the Future of Monitoring with Artificial Intelligence. Today we can collect lots and lots of performance data. We build beautiful dashboards and even have fancy query languages to access and transform the data. Still performance data is a secret language only a couple of people understand. The more busine...
    All in Mobile is a place where we continually maximize their impact by fostering understanding, empathy, insights, creativity and joy. They believe that a truly useful and desirable mobile app doesn't need the brightest idea or the most advanced technology. A great product begins with understanding people. It's easy to think that customers will love your app, but can you justify it? They make sure your final app is something that users truly want and need. The only way to do this is by ...
    CloudEXPO | DevOpsSUMMIT | DXWorldEXPO are the world's most influential, independent events where Cloud Computing was coined and where technology buyers and vendors meet to experience and discuss the big picture of Digital Transformation and all of the strategies, tactics, and tools they need to realize their goals. Sponsors of DXWorldEXPO | CloudEXPO benefit from unmatched branding, profile building and lead generation opportunities.
    Digital Transformation and Disruption, Amazon Style - What You Can Learn. Chris Kocher is a co-founder of Grey Heron, a management and strategic marketing consulting firm. He has 25+ years in both strategic and hands-on operating experience helping executives and investors build revenues and shareholder value. He has consulted with over 130 companies on innovating with new business models, product strategies and monetization. Chris has held management positions at HP and Symantec in addition to ...
    Cell networks have the advantage of long-range communications, reaching an estimated 90% of the world. But cell networks such as 2G, 3G and LTE consume lots of power and were designed for connecting people. They are not optimized for low- or battery-powered devices or for IoT applications with infrequently transmitted data. Cell IoT modules that support narrow-band IoT and 4G cell networks will enable cell connectivity, device management, and app enablement for low-power wide-area network IoT. B...
    The hierarchical architecture that distributes "compute" within the network specially at the edge can enable new services by harnessing emerging technologies. But Edge-Compute comes at increased cost that needs to be managed and potentially augmented by creative architecture solutions as there will always a catching-up with the capacity demands. Processing power in smartphones has enhanced YoY and there is increasingly spare compute capacity that can be potentially pooled. Uber has successfully ...
    SYS-CON Events announced today that CrowdReviews.com has been named “Media Sponsor” of SYS-CON's 22nd International Cloud Expo, which will take place on June 5–7, 2018, at the Javits Center in New York City, NY. CrowdReviews.com is a transparent online platform for determining which products and services are the best based on the opinion of the crowd. The crowd consists of Internet users that have experienced products and services first-hand and have an interest in letting other potential buye...
    When talking IoT we often focus on the devices, the sensors, the hardware itself. The new smart appliances, the new smart or self-driving cars (which are amalgamations of many ‘things'). When we are looking at the world of IoT, we should take a step back, look at the big picture. What value are these devices providing. IoT is not about the devices, its about the data consumed and generated. The devices are tools, mechanisms, conduits. This paper discusses the considerations when dealing with the...