Most enterprises have a dedicated customer support group / central email for creating / routing incidents. These emails are designed to go out from a single email account configured in the incident management system. Managing these emails or backtracking them for an event occurrence or analysing across severity, priority, business groups is a cumbersome task. This requires going back through incident numbers and replies to the same. Usually these reports can be done from within the incident reporting tool. If the enterprise has multiple incident reporting tools for different systems, it makes it very hard to collate and consolidate data across these systems to get a unified view along with technical challenges, given their different data dictionaries and export mechanisms.
In the use case above, one common feature of all these incident management systems is that they use emails to record and respond to incidents, if these emails can be integrated into a data warehouse or a centralized database, they can then be mined for insights which could offer deeper operational insights about Incident/Change/Problem management effectiveness in an organization.
SAP Business Objects Data Services or SAP Data Services has a mechanism called as a Data Provisioning Bridge (DP Bridge) which can be used along with existing Data Services Infrastructure to extract data from the PST files of the email account into the data warehouse.
To do this, set up DP (data provisioning) Bridge as a connection to the SDI (smart data integration) functions. Set up DP Bridge as an adapter instance in Administrator (Data Services Management Console) and use configuration values that support the specific SDI functionality that you want to use (which is this case is a PST file). Currently you can set up the DP Bridge adapter to import Microsoft Outlook PST data, which includes mail message and attachment data.
Block Diagram to illustrate the email data mining
After you create an instance of the DP Bridge adapter, you can create an adapter datastore in Data Services.
Below are the datastore configuration details
| Adapter option | Value |
| Datastore Name | Specify name of the datastore |
| Datastore Type | Choose adapter as a datastore type |
| Job server | Choose your job server |
| Adapter Instance Name | Choose the DP Bridge adapter instance that you created in Management Console. |
| PST file location | Enter the path of the PST file |
| Support large object | Default value is Yes.
Yes – Imports the BLOB and CLOB data types (large objects).
No: Does not import large data type fields. All large object data is skipped during job execution |
| Default Base64 LOB field length in kilobytes (KB) | Default value is 16 KB.
Imports the number of KBs for each large object type field. |
Support for large object:
Imports the BLOB and CLOB data types (large objects) from the PST file. Many fields from email messages and attachments are large object data types. Importing this data type can slow down job performance.
Data services considers these BLOBS and CLOB objects as varchar (16384), In case the target database has limited support of large objects we can truncate the string in data services like varchar (4000) to get chunk of data from large object for our reference. Do bear in mind that even though Attachments come in as CLOB or BLOB objects, they do not get translated into an understandable format within the Datawarehouse. The CLOB and BLOB objects are used for fields other than attachments as well. The table structure of the source highlights the fields you get when importing the PST file through DP Bridge.
Metadata
Once you create the datastore configuration, you can import the table from external metadata. You can see your inbox and attachment in the external metadata which will be imported as a table and loaded into target table.
Outlook mail attachment table structure
| Field Name | Datatype |
| MSG_ID | Varchar (1024) (primary key) |
| LONG_FILENAME | NVarchar (4096) |
| FILENAME | NVarchar (4096) |
| DISPLAY_NAME | NVarchar (1024) |
| PATH_NAME | NVarchar (1024) |
| CREATION_TIME | Timestamp(datetime) |
| MODIFICATION_TIME | Timestamp(datetime) |
| SIZE | Integer |
| COMMENT | NCLOB (varchar (16384)) |
| CONTENT | BLOB (varchar (16384)) |
Outlook mail message table structure
| Field Name | Datatype |
| MSG_ID | Varchar (256) (primary key) |
| SUBJECT | NVarchar (4096) |
| SENDER_NAME | NVarchar (4096) |
| CREATION_TIME | Timestamp(datetime) |
| LAST_MDF_TIME | Timestamp(datetime) |
| COMMENT | NCLOB (varchar (16384)) |
| DESC_NODE_ID | Varchar (1024) |
| RECIPIENTS | CLOB (varchar (16384)) |
| DISPLAYTO | CLOB (varchar (16384)) |
| DISPLAYCC | CLOB (varchar (16384)) |
| DISPLAYBCC | CLOB (varchar (16384)) |
| IMPORTANCE | Varchar (100) |
| PRIORITY | Varchar (100) |
| ISFLAGGED | Tinyint |
| MESSAGEBODY | NCLOB (varchar (16384)) |
Other possible enterprise use cases:
- Organising the mail receiving from scheduled batch job for error handling, failure of job etc., using DP bridge.
- In some cases, we have integrated the multiple third-party system, legacy system in to our landscape which will send mail to notify the activities, issues etc., It will be organised via DP bridge in SAP data services.
Subscribe to our Newsletter
The post Email Data Mining for Incidence Management Using SAP Data Services appeared first on Visual BI Solutions.