Surely everyone loves documentation! Especially if your customer is a government organisation (which often happens to us here in Wellington) your clients will often feel uncomfortable if anything on the project is not documented thoroughly. However, I have noticed that new people joining teams tend to ignore the existing documents completely, opting instead to ask loads of questions. Now, I’m only talking about developers, a very specific section of the population raised on the Agile principle “working software over comprehensive documentation”. They often think that the code is, or should be, self-explanatory.
Unfortunately for these people, there is a particular type of information that really does need thorough documentation on the BI project, and that is the business rules. The format of the rules document is always a struggle.
Users want to see them written in pure English.
Report developers are interested in exact field names.
Developers and testers want any twist of logic to be there as well.
On my previous projects, we usually chose user-readable format over the developer’s wishes, leaving the latter to read the code itself if they were that way inclined. This time, on my latest project, it is different. We have many technical rules, as we called them. Data source systems are so old that business processes have changed since they were initially built, so there are many additional pieces of logic applied to the data before it is shown on the user’s screen. We need these to be in the Data Warehouse as well, so BI users could get the same results as the source system shows. Most of the business users were not aware of these improvements, thereby they are “technical”. We could finally document rules for techies! After some thought, we had two candidates for our rules template, “Given-When-Then” and Decision Tables.
We chose “Given-When-Then” test scenarios templates, like the Gherkin language one. It is a quite intuitive way of recording rules in plain language as logical statements. While still readable by business analysts, it is also pseudo-code ready to be plugged in. There is not much to remember if you want to adopt it as well:
Other acceptable keywords are “And”, “Or” and “But”. To be fair, there should be no “Or”; if you need “Or”, you should use another “When” statement, but I have cheated. We have extended statements with the exact database field names and values. Here is an example of the business rule I have captured in this pseudo-code:
|Given||Any transaction record (CRM Transactions)|
|When||Record status (REC_STAT) is “Cancelled” (“CAN”)|
|Or||Person ID (PARTY_ID) is 0|
|Then||Complete status is “Incomplete”|
|When||Transaction Type (TRAN_TYP) is “Financial” (“FINS”)|
|And||Payment date (PAYMT_DTE) is not set or default date (01/01/1900)|
|Then||Complete status is “Incomplete”|
|When||any other conditions|
|Then||Complete status is “Complete”|
Another way of capturing rules could be a decision table. Maybe less readable for business, but very snappy looking (especially if you colour it up!). Here is the same rule as above captured in a decision table:
|Transaction ID||Transaction Type||Record Status||Person ID||Payment Date||Transaction Complete Status|
|AAA/1111||not relevant||CAN||not relevant||not relevant||Incomplete|
|SLS/3333||not relevant||<> CAN||0||not relevant||Incomplete|
|FINS/7777||FINS||<> CAN||<> 0||NULL, ‘1900-01-01’||Incomplete|
|FINS/8888||FINS||<> CAN||<> 0||<> NULL, ‘1900-01-01’||Complete|
|AAA/6666||<> FINS||<> CAN||<> 0||not relevant||Complete|
“Not relevant” in a cell means that the value doesn’t matter, even if it is NULL, the condition is fulfilled anyway.
Both options have their pros and cons. Decision table takes longer to document, but are visualised better. Once we had written the first rule both ways, we couldn’t really choose one, so that we chose both. Here is a template we ended up with: Example-business-rule. We have also added some background details on where the rule came from, business area (captured in BEAM session outputs), rules exceptions and other stuff that it is nice to capture somewhere, but that doesn’t fit into any template. In the end, it takes the whole page and feels slightly verbose but we have satisfied our testers, who couldn’t find anything else to question about the rule.
The last bit to mention is team efficiency. Before we adopted this template we struggled with how the rule development cycle was sequenced. Usually, it started with someone getting the definition of the rule from people related to the source system development. It was vague and often require refinement, but we only found that out when tried to develop it in the Data Warehouse and discrepancies popped up with the source front-end due to small logic tweaks they forgot to mention. Only after that, the testing could start, and the rule could have changed significantly by this point.
With the introduction of this template, the team culture has also changed. Now the process starts with a rule being captured in the template (this could be done as a group Q&A session), putting it into the logical statements helps with understanding the gaps which are defined at this stage. After that, writing test scenarios and DW development can both start, tests are ready for execution by the time the data is loaded into the Data Warehouse. Overall waiting time for the team is minimised and everyone can get on with things in a smoother and more organised way.