jit br mining

8/13/2019 Jit Br Mining

1/18

Just in Time (JIT) Business Rules Mining

When Timeis All You Have and the Documentation JustIsnt There

Shirley J. Sartin, PMP

Abstract--- To maintain an application or reengineer it, you need to understand its

functions or background processes and the business rules that drive them. However,there are times when supporting documentation is unavailable or insufficient to aid you.

Without the budget for consultants or production solutions, how can the analyst manually

mine the business rules? This paper provides methods for determining functions, events,

and inferred business rules for existing applications or background processes, and

describes how to derive and verify a logical process model from source code or a user

interface.

I. IntroductionAt the Seventh International Business Rules Forum (2004), Terry Moriarty spoke aboutcurrent statebusiness rules. In the session Get Ready for Your Business Rules

Management, Ms. Moriarty asked how much of the current state Business Rules (BRs)should be harvested and suggested that we could reengineer to a new vision. At the

least, existing systems could be engineeredjust enoughto understand the current

environment as the business understands itself. But then again, Ms. Moriarty wonderedabout the dead or forgotten business rules and suggested that maybe we could pull out

just enough information to understand and cross link the discovered BRs with the new

system.

However, many analysts do not have access to data, synchronized code, or analysis

artifacts. Harvesting current business rules by reverse engineering is sometimesnecessary, and vendor tools are available to perform this task. These tools can be very

expensive, but without this specialized software to automatically mine the business

rules, what can analysts do? How can they verify that the inherent BRs found in thecode are true and current?

By returning to the basics of systems analysis and reverse engineering using only

source code and/or Graphical User Interfaces (GUIs), a logical process model based on

event-driven processes may be manually constructed and used to derive businessprocessing requirements and BRs. Verification and validation of the resultant

requirements and rules with both business and Information Technology (IT) application

development personnel will serve to ensure inferred BRs are on target.

This paper relates methods and steps used to determine functions, events, and inferred

BRs by logical process modeling of existing applications or background processes forthe following:

An e-Commerce Web site with no viewable source code.

An order processing application built with Powersofts PowerBuilder.1

1PowerBuilder, a GUI development tool from Powersoft, is used for developing client-server database applications inPowerScript, a language loosely based on BASIC.

1


2/18

Structured Query Language (SQL)2stored procedures automatically pushingmaster account data from one system to another.

The logical process model developed is an interim step in system design and may later

be transformed into a physical process model (or application schema) for the chosentechnical architecture.

The next section provides steps in reverse engineering a department stores online

catalog. Section III discusses manually mining BRs from events. Client-server

application reverse engineering is covered in Section IV, Structured Query Language(SQL) stored procedures are discussed in Section V, and additional information is

provided concerning the use of help text and training materials in Section VI. The

paper concludes in Section VII. A glossary is provided in Appendix A and information

concerning building a context diagram is presented in Appendix B.

II. Reverse Engineering Using Logical Process EngineeringThe logical process model, also known as the conceptual or business model, has a

process focus and a system owner and/or system-user perspective and is not concernedwith implementation details or technology. Models are created using eventpartitioning,where a system is separated into subsystems based on business events and

responses to those events.3

A logical process model is especially effective for determining BRs because this type

of modeling does not include redundant activities, is not technology dependent, and has

no physical limitations or requirements. The completed model presents a business viewof activities from which BRs may be inferred.

This section provides details for creating a logical process model while reverseengineering an online catalog. The instructions provided may be used for building

most logical process models.4

Logical Process Modeling Artifacts

In order to create a logical process model, a systems logic must be organized anddocumented. Since it is considered to be a conceptual or business model, the logical

process model is intended to show what a system does,not how it is done, and thus

does not reflect implementation of a technical solution. In creating a logical model

for use in BR mining, specific steps are followed to produce the following artifacts:5

System Context Diagram Indicates the events or transactions the system mustrespond to; system responses, sources and destinations for both; and the identity

of all external data stores.

2Structured Query Language (SQL) is an industry language for creating, updating, and querying relational databasemanagement systems.

3Jeffrey L. Whitten and Lonnie D. Bentley, Systems Analysis and Design Methods, (Irwin/McGraw Hill, 1998): 236.4Primitive and system diagrams are typically created during logical process modeling. However, these diagrams may

not be necessary for determining BRs, and thus the logical process modeling efforts may end after building all event

diagrams.5Whitten and Bentley, Systems Analysis and Design Methods, 237.

2


3/18

Functional Decomposition Diagram Provides top-down functionaldecomposition or structure of the system.

Event-Response List Outlines the business events the system must respond toalong with appropriate responses. Also serves to identify how input data flow

will occur for the system.

Event Diagrams Derived from a combination of the event list and a furtherpartitioning of the functional decomposition diagram, show inputs, outputs, and

data store interactions for each event.

Reverse Engineering Steps

The artifacts created during logical process modeling are used in analysis verification

and validation (V&V) efforts and are invaluable for tracing features/functions throughthe process diagrams. Basic steps used when reverse engineering using logical

process modeling methods are listed below.6Appendix Aprovides a glossary of

process modeling terms and definitions.

1) Identified FeaturesA logical process model was built by first identifying features of the software or

process. These features could be considered the bullet points on a shrink-

wrapped software box. By looking at the Web pages of the department storesonline catalog, the following features were identified:

Search Product (by Category/Department)

View Product List (by Department)

Create/Manage Account

Manage Shopping Cart

View Store Locations and Events

Status of Orders Inquire on Shipping Methods & Charges

Login to Site

Apply for Credit

Each of these features was considered to be a separate subsystem or process.

Focus was placed on only one process at a time for each process modeling step,

below, through to the final step where an event diagram was created for allprocesses. By focusing on individual processes, the analysis effort was more

manageable.7

6Detailed instructions for developing logical process models may be found in systems analysis or design methodtextbooks.

7Only the Create/Manage Accountprocess modeling is documented in this paper.

3


4/18

2) Determined FunctionsIn reviewing the online catalog site, the Create/Manage Accountprocessappeared to provide for customer accounts with the functions listed in Table 1,

below. These functions were identified by stepping through the online catalogs

web pages and noting the actions that occurred.

Function Description

Create Account Create user account.

Remind Password E-mail password hint to existing

customer.

Select Profile Functions Select a profile function:

Update Account Profile

Access Shopping Bag

Review Order History/Status

Setup Express CheckoutTable 1. Create/Manage Account Functions

3) Created Context DiagramBy considering context diagram symbol descriptions (Appendix B)and

reviewing each of the functions in Table 1,inputs, outputs, data flows, data

stores and functions initiated by external agents of the Create/Manage Account

process were identified and placed in Table 2,below. This information wasdetermined by reviewing the online catalog pages in more detail.

Process

(Feature)

External

AgentsSub Processes (Functions) Events/Responses

External Data

Stores

NewCustomer

Create AccountRegister Account

New Acct InfoAccount Info Creation

User Data8

Remind Password

Ask for Password

E-mail Address

Get Password Reminder

User Data

Function Selection

Maintain Account

Account Info UpdatesUser Data

Maintain Account

Order History/Status

User Data

Orders

Maintain Account

Express Checkout

User Data

Shipping/Billing

Create /

Manage

Account ExistingCustomer

Select Profile Function:Update Account Profile

Access Shopping Bag

Review Order History/Status

Setup Express Checkout

Maintain AccountShopping Bag Items Shopping Bag

Table 2. Create/Manage Account Function Information

8The external data stores User Data, Shopping Bag, Order Table, and Shipping/Billingimplied data relating toevents/responses. In actuality, a data store may exist but the name is unknown at this point in the modeling process.

4


5/18

By using information in Table 2,above, and context diagram symbols in

Appendix B,the context diagram in Figure 1,below was created.

Figure 1. Create/Manage Account Context Diagram

This context diagram provided an understanding of the Create/Manage Account

process. It assisted in the next logical process modeling step where the

sub processes were partitioned into a functional decomposition diagram.

5


6/18

4) Created Functional Decomposition DiagramA functional decomposition diagram (also known as a hierarchy chart) wasdrawn to partition the process into logical functions thus providing the

beginnings of an outline for drawing data flow diagrams. The functional

decomposition diagram provided a top-down functional decomposition or

structure of the system.

9

The Create/Manage Accountcontext diagram was decomposed into subprocesses (functions). As shown in Figure 2,below, the sub processes were

represented with meaningful names.10

Figure 2. Create/Manage Account Functional Decomposition Diagram

9Whitten and Bentley, Systems Analysis and Design Methods, 237.10Where necessary, processes are divided into sub-processes.

6


7/18

5) Created Event Decomposition DiagramBy focusing on events in the Create Accountfunction (reference dashed area incontext diagram Figure 3,below), events were decomposed and compiled to

identify and confirm business events to which the system must provide a

response.11

The completed Event Decomposition Diagram appears in Figure 4.

Account Info

Creation

Create/Manage

Account

New

Customer

Existing

Customer

Register Account

Maintain

Account

Orders

Shipping/

Billing

Shopping Bag

User Data

Select

Profile

Function

New Acct

Info

Create

Account FunctionSelection

Order History/

Status

Express

Checkout

Shopping

Bag Items

Account Info

Updates

User Name

Remind

Password

E-mail

Address

Ask for

Password

Acct

Info

Function

Figure 3. Create Account Function

11Whitten and Bentley, Systems Analysis and Design Methods, 237, 245.

7


8/18

Derived from a combination of the context diagram (Figure 3,above) and a

further partitioning of the functional decomposition diagram (Figure 2,above),the event decomposition diagram below was created to show decomposition of

the Create Accountfunction and its associated events tagged with unique

numbers. These numbers relate to event diagrams or data flow diagramsin

Step 7.

Figure 4. Create Account Event Decomposition Diagram

8


9/18

6) Event Decomposition DiagramIn following the Create Accountfunction in the context and eventdecomposition diagrams (Figures 3 and 4, above), triggers and responses for

each event were determined as listed in

Table 3, below. The event-response list outlines the business events the system

responded to along with the appropriate responses. This list also served toidentify how input data flow occurred for the system.

Event Event

Description

Trigger (Inputs) Responses (Outputs)

RA-01

RegisterAccount

New customer

chooses option

to register a

new account.

Create/Manage Account Module

(View: Login Page)

Choose Register Option


(View: Registration Form)

RA-02

New

AccountInfo

New customer

provides

accountinformation.


(View: Registration Form)

Provide account information:

User Data

User Name

E-mail AddressPassword

Zip/Postal Code

E-mail Update Selection

Create Account Sub-Module

Validation/verification form

inputs.

(View: Error or acceptance

messages.)

RA-03

Account

Info

Creation

System writesdata to user

data store and

provides

account

validation.


(Background Process)

Register User Data


Validation/verification user data.

(View: Welcome new user

Error/Acceptance Message.)

Table 3. Create Account Event-Response List

9


10/18


11/18

easier to identify those certainevents. Narrowing the focus from high level functions to

individual events aided in identifying critical events and their associated BRs.14

In Figure 5,above, it was determined that a new customer must register an account.

When the new customer chose the option to register an account, they were presentedwith a registration form. Should a BR be considered for this event? Since the external

agent (entity) was identified asNew Customerand the action was to provide a

registration form, the BR could have been:

Customers must have accounts to access the online catalog.

Figure 6,above, depicted the event where a new customer provided account

information, and theNew Account Infoevent (RA-02) returned either an error or an

acceptance message. A BR for this event might have been:

Only valid account information will be accepted.

While building a data flow diagram for the eventAccount Info Creation(RA-03), itwas determined that this event should include a data store. As shown in Figure 7,thisevent passes validated user data to the User Datadata store and either a Welcome New

User or a System Errormessage is returned to the New Customer. BRs for this event

could have been:

Registered user account information must be maintained in a system datastore.

User account information must be valid.

Once all features had been processed as in the Create/Manage Accountprocess, above,

and all event diagrams for functions and identifying potential business rules werethought to be discovered, the logical process model was completed by creating a systemdiagram (not discussed in this paper). All discovered BRs were verified with the

business users and a rule repository created and populated with the discovered BRs.

Note that a repository could be a spreadsheet, a database, or tables in a word processingdocument.

Reverse engineering by logical process modeling has also been used for client-serversystems, as discussed in the next section.

IV. Reverse Engineering a Client-Server SystemThe previously documented logical process modeling methods were used to reverse

engineer an online catalog. These same modeling methods were also used whilereverse engineering a telecommunication companys order processing system built with

PowerBuilder.

14Interpretations of these events may vary depending upon the business. BRs created during the logical processmodeling exercise above, serve as examples of BR mining and may not, in fact, be actual BRs created during a truemodeling situation.

11


12/18

Logical process model diagrams were created based on features discovered by stepping

through GUIs and reviewing help text and training documentation. An objective of thisproject was to provide enough technical details so the development staff could correct

application defects; thus it became necessary to identify technical details such as table

names, field attributes, etc. By using PB Code Analyzer,15cross references

(relationships) to objects were discovered and the tools Table Locatorfunctionidentified tables used by each of the objects.

Objects identified by the tool were included as external agents in the logical process

model. For example, in theEdit Requestevent, external agents were identified as the

Request Originator (a person), and the two objects PROV_DL and NUM_ASGN wereprovided the meaningful names Provisioning Directory ListingandNumber

Assignment. The analyzer tool also served to verify functions/events. The tables used

in this event were identified by the tool and appeared in the event response list as

below:

Event Event Description Trigger (Inputs) Responses (Outputs)

Edit

Request

Request Originator, Provisioning

Directory Listing (PROV_DL) and

Number Assignment (NUM_ASGN)edits general information or creates

number of lines on open number

reservation request.

Number Reservation

Module

(View: Open Reservation

Request, double-click on

selected request, perform

edit in left frame, click

Save icon.)

General Info Update

Line Count Increase

Open Request

general information

is updated.

Table 4. Edit Request Event on the Event-Response List

TheEdit Requestevent (2-4d) was represented in an event diagram as follows:

Figure 8. Edit Request Event Diagram

Once all modeling efforts were completed and BRs were identified, they were verified

with the business users and the development team.

15Ascension Labs, LLC.

12


13/18

SQL stored procedures automatically pushing master account data from one system to

another were reverse engineered using this logical process modeling method. The nextsection provides information concerning process modeling for SQL procedures.

V. Reverse Engineering SQL Stored ProceduresMainframe legacy systems are sometimes used to host master account information.

With the mainframe acting as a data repository, the account information is pushed on a

scheduled basis to subsidiary repositories on other platforms. This pushed data isconsumed by applications not located on the mainframe platform.

During a reengineering project, reverse engineering became necessary in order todetermine the processes and BRs of the stored procedures pushing master data into

tables on a SQL server. These procedures involved one set of 10 SQL procedures and

one set of seven SQL procedures accessing and pushing data from two separate datasources. The procedures were created by a consultant in 1999, and only a brief

definition of the SQL steps and one process flow were available as documentation. It

was acknowledged that the original code had undergone several changes, but it couldnot be readily determined what BRs were underlying the code.

By using the limited documentation and reviewing comments embedded in the code, a

potential list of functions and events were discovered. These functions and events were

then used to build a partial logical process model including a context diagram and a

functional decomposition diagram. SQL keywords INSERT INTO, CREATE TABLE,and UPDATE were considered events in the procedures. Code leading up to the

keywords was reviewed to document the actual event that was occurring. Event-

response lists were then created and used in hyperlinked word processing documentsfor use when reviewing events linked to specific code.

V&V of the process model documentation and code was held with the Data BaseArchitect and a Data Base Administrator (DBA) to verify code interpretation. Uponapproval of the process model, the Analyst, Project Manager and DBA reviewed the

approved hyperlinked documents and derived BRs. Business users verified the inferred

BRs and the reengineering project continued.

VI. Help Text and Training MaterialsThe cases provided in this paper described reverse engineering by viewing onlinecatalog pages, stepping through a client-server system using help text along with

analyzed code, and by reviewing SQL program code. When preparing to reverse

engineer, it is also helpful to gather documentation from sources such as help text or

training materials.

The help text table of contents assisted with reverse engineering the client-serversystem because it was organized by system features. These system features narrowed

the focus for the logical process modeling effort. Even without a table of contents, the

help text alone would have provided assistance since it contained informationconcerning system functionality. Depending on format and content, training materials

may also be helpful in identifying features or functions.

13


14/18

VII. ConclusionIn order to provide effective software maintenance and/or reengineering, it wasnecessary to understand functions of a software application or system background

processes and the business rules that drove them. When support documentation was

not available or insufficient for this purpose, the Analyst was able to manually mine

BRs by using proven methods and steps, thus combining comprehensive businessrequirements analysis with logical process modeling.

14


15/18

Appendix A. Glossary

Term Definition

Context Diagram Defines the scope and boundary for one process and

identifies inputs and outputs. External agents of the

process are indicated as well as data stores andinteractions with the process.

Event Logical unit of work that must be completed as a

whole.

Event Diagrams Data flow diagrams of individual events based on thecontext diagram.

Event Partitioning Factors a system into subsystems based on business

events and responses to those events.

External Agent External agents/entities are the source for net inputsinto the process and/or destination for net outputs from

the process.

Function Set of related ongoing activities of the business that

group logically related activities and tasks.

Processes Activities that have a beginning and an end, transform

data, and can be decomposed into other processes or

into events.

System Diagram High level diagram showing a composite of all eventsin the system or subsystem (optional for BR

discovery). The system diagram is constructed by

merging event diagrams.

A-1


16/18


17/18

Using Table 2. Create/Manage Account Function Information and applying the context

diagram symbols in Table 5,above, a context drawing was created (Figure 9,below).Note the context diagram shows only one high level process (feature) as Create/Manage

Accountand all associated sub processes (functions) as Create Account, Remind

Password, and Select Profile Function. The Create Accountfunctions events are called

out asRegister Account,New Acct Info, andAccount Info Creation.

Create/Manage

Account

New

Customer

Existing

Customer

Register Account

Maintain

Account

Orders

Shipping/

Billing

Shopping Bag

User Data

Create

Account

Select

Profile

Function

New Acct

Info

Account Info

Creation

Function

Selection

Order History/

Status

Express

Checkout

Shopping

Bag Items

Account Info

Updates

User Name

Remind

Password

E-mail

Address

Ask for

Password

Get

PW

Reminder

ProcessFeaure)(

t

External

Agents

Sub Process

(Function)

Events

Figure 9. Online Catalog Create/Manage Account Context Diagram

B-2


18/18

Shirley J. Sartin, PMP(BSBA MIS, MS CIS) is currently performing requirements analysis. Her 20+

years in Information Technology have encompassed roles in project management, systems analysis,

business analysis, programming analysis, systems administration, operations administration, and

technical training in a variety of industries, including broadband cable, telecommunications, aerospace

engineering, engineering product development, software, and transportation. Skilled in all aspects of ITincluding back- and front-office, Shirley effectively interfaces with development groups and customers

with a focus on empathizing and appreciating the needs of the end-user. She can be reached at

[email protected].

jit br mining

Documents