Architects Corner

Blogs on What and How of doing Enterprise, Business & IT Architecture. Did you hear "fail early"?

  • Home
    Home This is where you can find all the blog posts throughout the site.
  • Categories
    Categories Displays a list of categories from this blog.
  • Tags
    Tags Displays a list of tags that have been used in the blog.
  • Bloggers
    Bloggers Search for your favorite blogger from this site.
  • Archives
    Archives Contains a list of blog posts that were created previously.
  • Login
    Login Login form
Posted by on in IT Architecture
  • Font size: Larger Smaller
  • Hits: 4211

Data Architecture Assessment

Problem Definition

I had a chance to review Data Architecture of a small ERP (I’m citing an example of sales module where address and invoice tables de-normalized to accommodate some of address attribute in the Invoice table) in a firm which has small database (in terms of table only having 50 tables and few stored procedures as well because of bad coding practicef) but I found that those are quite messy. They were using structured data architecture and implemented database in RDBMS server so, in that case, the implemented database structures should have been in, and at least, the 3rd Normal Form (better to have BCNF) but that was not the case. When I voiced my concern, then the answer I received was that they had de-normalized it purposefully to reduce SQL complexity.

In my opinion, they were facing more issue after changing data structure like data inconsistency as a result of having duplicate columns, which were difficult to maintaining ACID properties of transactions, performance issues while creating and updating records, etc. I would like to discuss the issue. The issue starts from the moment when client says I did not get current invoice and say this is my new address. Now executive change the address from the Client maintenance screen and record is being modified in Customer -> Address table and send request to generate duplicate invoice but since Address is being copied in Invoice table itself instead to refer Invoice -> Customer -> Address table so you can imagine blunder .

So, the motivation behind data architecture assessments should be to find gaps between architecture/design and implementation.

Data Architecture Assessment

An assessment is a part of governance and it is as important as the architecture to build an enterprise. So for building an enterprise we should emphasize on architecture as well as governance and it should be a repeatable exercise to prevent failure.

Data architecture assessment is the most important assessment because the data is an asset to any enterprise, so keeping data in right shape is very important. Assessment ensures that the solution is exactly what an enterprise needs and the supporting architecture is relevant, effective and efficient.

There is another aspect of assessment, which is to understand the enterprise, the enterprise asset, the maturity level of stakeholders, the process to build data architecture, etc.

Objective of the statement should be refined and should provide feedback in data strategy, in data governance at design time, in roadmap, in mature standards, in identify gaps and in providing best solution, best practices, etc. Assessment should not be just for finding issues or gaps; instead this should be a hand in hand work with the client for identifying root cause and solving through the industry standard pattern.

An Assessment should have three phases: discovery, analysis and recommendation. The stakeholder should participate in analysis phase and recommendation phase while analysis should be done by the contractor.

How to Start

Any assessment requires motivation, so find out the list of motivation to start assessment.

ü  Define timeframe for assessment.

ü  Define agenda for assessment if it is not in routine

ü  Architecture assessment should start from contextual model and should be reviewed contextual model, conceptual model, logical model, physical model and implementation model (not implementation code because current exercise is about architectural assessment).

ü  Prepare a questionnaire for data architecture independent of technology

ü  Prepare separate questionnaire for different types of data (RDBMS, DBMS, Flat File, NoSQL, etc) architecture

ü  No need to address technology specific data because it is the implementation rather than the architecture

How to Conclude

A detailed plan should be made with the client to implement suggestions and gaps as per motivation behind the assessment.

ü  Suggestion should be realistic and based on industry standard pattern

ü  Gaps should be categorized according to risk

ü  Gaps should be prioritized as per client input and should be followed

ü  Whether the Next assessment is required or not, should be discussed with the client

Sample Questioner

All following questions should be categorized and assessed for relevant architecture only. This questioner is just a sample.

Questioner

Sub question

Purpose

Why Important

Common Assessment for all type of data structure
  1. Do you use metadata based data? If so
Do you have metadata strategy and/or standards? Assessment of meta model and/or schema  
  1. How do you classify data
  Assessment of vocabulary, thesaurus, taxonomies  
Assessment of Structured Data (Assessment of Logical Model or Physical Model)
  1. What level of normalization have you implemented
If data structured is in 3rd or above normal form. Assessment of good data structure and physical mode should reflect that. Normally ignored during assessment which bypass major gaps between design and implementation.
Assessment of Code generation capability from Model
  1. How do you create DDL
  Assessment of capability to use tools to create DDL/script. Normally developer write code manually which increases chances of gap between design and code/script

Conclusion

I found during the last assessment that architect don’t realize that all the perspective is essential element rather than architectural fashion. I found that in this case architecture was started with correct design patterns but somehow at implementation level it has been diluted to fix some of the issue but ground reality was that implementation team has introduced unwillingly more issues to fix identified issue. So assessment is also a integral part of architecture governance.

Short Assessment Estimation and Scope of Work

This assessment is one week activity and scope of work is as follow:

 

Data Architecture - Sample Assessment

           
 

Common Exercise

  Prerequisite : Requirement document of 15 Business Entity, Business Glossaries, architecture governance document, data architecture standards
  Architectural Artifact to review MDM Document, Data architecture document and model
           
# Questioner Sub question Purpose Parameters/Volume Time Required (hr)
1 1.       Do you use metadata based data? If so Do you have metadata strategy and/or standards? Assessment of meta model and/or schema   1.00
2 2.       Which kind of data you are using? Like structured/semi structured/unstructured Assessment of data definition will be done accordingly Structured data but implemented at physical level (review of list of 15 business entity 1.00
3 3.       How do you classify data   Assessment of vocabulary, thesaurus, taxonomies   1.00
4 4.       Do you use governance structure to govern changes in data structure   Indicates maturity of EA practice   1.00
5 5.       In your organization data security comes under Security Architecture or Data Architecture   Security & vulnerability analysis and assessment Comes under security architect  
6 6.       Do you have list of business entity If yes, in what form, model or text Assessment of asset management. Contextual model review 1.00
7 7.       Do you have list of business entity along with relationship? If yes, in what form, model or script? Assessment of maturity if having architectural standard & process. Conceptual model review 1.00
8 8.       Do you have logical data model If yes Assessment of maturity of architectural tool, standard, process and framework. logical data model review 1.00
9 If no, supporting documentation is required. Assessment of valid reason to not having logical model. NA 0.00
10 Are logical models connected with relationship Assessment of understanding importance of relationship. yes, reviewed against BRD, Entity and its attribute, number of entity 25 6.00
11 Is this first level of model or you have transform/derived from list/model business entity. Assessment of architectural process and governance to maintain all these models. review governance process 1.00
12 Do you use business name in logical model? Assessment of logical model in terms of business entity. yes, reviewed against BRD 0.00
13 9.       Do you use any repository to baseline model If yes Assessment of architectural process and governance. assessment comes under CMMi 0.00
14 If yes, what is the change process Assessment of design time governance. assessment comes under CMMi 0.00
15 10.   What is the mechanism to assure maximum reusability of data   Assessment of reusability. not yet decided  4.00
16 11.   How do you ensure data security   Assessment of security Comes under security architect 0.00
17 12.   How do you ensure complete data related requirement has been converted into model?   Assessment of requirement mapping with model. Assessment of data model, BRD, mapping mechanism 1.00
18 13.   How do you ensure data access in architectural perspective.   Assessment of user mapping with data. not yet decided, currently do in implementation only  3.00
19 14.   How do you ensure availability of data within QoS agreement?   Assessment of QoS considering while data modeling. Assessment of QoS and data model mapping 4.00
20 15.   Do you model domain wise.   Assessment of understanding domain and segregation of component. no 0.00
21 16.   Do you maintain vocabulary to become stakeholder in same page?   Assessment of vocabulary and understanding stakeholder’s need. assessment of business glossaries for data 1.00
22 17.   Have data quality parameter documented and reflects in model?   Assessment of QoS parameter. QoS document review 1.00
           
 

Assessment of Structured Data

           
  Questioner Sub question Purpose Parameters/Volume Assessment Time Required (hr)
23 1.       Do you have physical data model If yes, Assessment of architectural maturity to implementation number of entity 45 (after implementation normalization) 12.00
24 Is it transformed from logical model Assessment of using tool and capability to using Architectural Framework yes, verify custom scrip if any to generate corresponding model  
25 Is there change in logical model and physical model Assessment of understanding difference between these models. Physical model of structured data should be entirely different from logical model as having normalized data. yes, normalization is being implemented in physical model to achieve "Time to Market" if new technology adapted. 4.00
26 Is physical model being required detailing also? Assessment of understanding physical model. yes, some attributes is being added to support technology framework 4.00
27 How do you ensure QoS related to physical model is being addressed? Assessment of QoS parameter that required in physical layer and chosen technology. verify mapping document/matrix 5.00
28 2.       Do you follow naming convention If yes, what is the strategy Assessment of maturity if implementing standards. no, technology specific naming convention is not being followed. 0.00
29 3.       What level of normalization have you implemented If data structured is in 3rd or above normal form. Assessment of good data structure and physical mode should reflect that. 3rd normal form 0.00
30 4.       What level of normalization have you implemented If data is less than 3rd normal form Assessment for upgrading normalization    1.00
31 5.       What level of normalization have you implemented What is the nature of application in terms of transaction, OLAP or OLTP Assessment for OLTP application because highly structured data is required for OLTP only. half part is in 3rd normal form while rest in 2nd normal form, because some of the report is OLAP kind  4.00
32 6.       Do you have done de-normalization, If so, need documentation Assessment of the document for valid reason of de-normalization or just done accidently. no written evidence against de-normalize 3.00
33 7.       Do you have OLAP type of any report If yes Assessment for architectural elements how it is being implemented. already done 0.00
34 8.       How do you create DDL   Assessment of capability to use tools to create DDL/script. Manual 9.00

 

 

Last modified on
Rate this blog entry:
1
(0.00 points earned)
Amlendu Kumar has not set their biography yet
Awarded Medals: - empty -
Current ranking: No rank.
Author's recent posts

Comments

  • No comments made yet. Be the first to submit a comment

Leave your comment

Guest
Guest Saturday, 25 March 2017
SSL