There are essentially five types of data in corporations:

Unstructured :

Unstructured data is a generic label for describing any corporate information that is not in a database. Unstructured data can be textual or non-textual. Textual unstructured data is generated in media like email messages, PowerPoint presentations, Word documents, collaboration software and instant messages. Non-textual unstructured data is generated in media like JPEG images, MP3 audio files and Flash video files.

Transactional :

Transactional data, in the context of data management, is the information recorded from transactions. A transaction, in this context, is a sequence of information exchange and related work (such as database updating) that is treated as a unit for the purposes of satisfying a request. Transactional data can be financial, logistical or work-related, involving everything from a purchase order to shipping status to employee hours worked to insurance costs and claims.

Metadata :

Metadata describes other data. It provides information about a certain item's content. For example, an image may include metadata that describes how large the picture is, the color depth, the image resolution, when the image was created, and other data. A text document's metadata may contain information about how long the document is, who the author is, when the document was written, and a short summary of the document.

Hierarchical :

Hierarchical data is data that is grouped into a tree-like structure, with repeating parent/child relationships. It stores the relationships between other data. It may be stored as part of an accounting system or separately as descriptions of real-world relationships, such as company organizational structures or product lines. Hierarchical data is sometimes considered a super MDM domain, because it is critical to understanding and sometimes discovering the relationships between master data.

Master :

Master Data is common data about customers, suppliers, partners, products, materials, accounts and other critical “entities,” that is commonly stored and replicated across IT systems. Master Data is the high-value, core information used to support critical business processes across the enterprise.

lets look at how data flows in and out of the MDM solution. Starting at the top right – this explains how data flows into the Hub and becomes the single source of truth in BATCH MODE.

- Land Process :

The landing process is where data enters the Informatica Hub from external sources that contain information (e.g. call center databases, billing applications, etc.). Data can be inserted into landing tables using any type of data movement tool (e.g. ETL, EAI, web services, etc.)

- Stage Process :

In the Staging process, data moves from the landing tables to staging tables after operations like delta detection, data cleansing, transformation and reject management.

- Load Process :

Next, the load process applies trust and validation rules to the data in a staging table and loads the resulting data into a table in the target data model. The load process updates existing records in the target data model and inserts any new records provided by the source system.

- Match Process :

Moving up in the Target Data Model part of the diagram, the Match Process applies a set of user-defined match rules that results in merge (or link) candidates.

- Merge Process :

Merge candidates are consolidated either automatically or with user input using Informatica Data Director.

One of the most powerful capabilities of the Informatica Hub is the way it dynamically computes at merge time (and at update time) the best information – at cell-level from multiple sources and ensures survivorship of this information over time.

When two matched records are merged to create a consolidated record only one of the two values from the source records survive in each corresponding cell of the consolidated record. To ensure the most reliable information in every cell of the consolidated record, Informatica Trust Framework™ uses a concept of “Trust” that can be assigned for each column of the source systems.

The lower right section of the slide explains how data flows into the Hub in REAL TIME.

For the most part, the REAL TIME data flow process and the BATCH data flow process are similar with a few exceptions:

1. 1. Records in message based transactions are processes one at a time, as you might expect.
2. 2. Instead of using landing and staging tables, the mappings are instantiated as web services with the defined input and output parameters on the Application Server.

Finally, Once these consolidated records are processed by the Hub, they are considered “master records” . These master records can then be consumed by analytical systems such as Data Warehouses or as a data source for operational systems (applications). The master records are sent to these different systems via web services, using message queues, or even using ETL tools.