Billing Systems are not Billing Systems

Telecommunication Billing System ,supposedly a complex system with billing only being a part of it . I have always argued that its not just billing system in itself but made up of various parts such as :-

• Mediation
• Rating
• Billing
• Bill formatting 
• Collections
• Financial journals & accruals
• Payments (which in itself has several sub parts such as direct debit ,credit card , lockbox, cheques ,etc). 

There are also various other  "modules" which are important to understand on its own right :-

• Order management and Workflow 
• Security modules
• Job schedulers 
• Various middleware systems

Telecommunication Billing system gets easy to understand when we look at each parts separately . However, as Aristotle said "the whole is greater than the some of its parts" , an end to end view is surely beneficial for efficient design,operation ,configuration ,re-design and migration exercise of such systems.

Here is well written article on Billing systems by Hatem Moustafa which resonates with keeping it simple in this complex world of billing systems.

Three simple questions

Three simple questions ,from useful  Forbes article

1. What’s the outcome I want? 
2. What stands in my way? 
3. Who has figured it out already? 

Think it has worked for me , specially when successfully changing a light bulb- Simple!

System Thinking World Journal

I am hoping to see more of these articles around system thinking – such a simple method for analyzing complex problems. I would have named this article as “unleashing deeper understanding”. Refer Unleashing Understanding by Author: Gene Bellinger

Credit Card Parsing Algorithm for Unix

I was asked to parse 85 million records of varchar(255) character fields to find credit card numbers in it and hide them to achieve PCI DSS complaince . This is what I did to simplify the stuff !

1) Get a dump of database table (or important columns) to your unix box.
2) Write a "ccengine" parsing script based on Lugn's algorithm.
3) Dump the rows with credit card numbers.
4) Hide the CC# with regexp .
5) Upload back to the database.

Here is a credit card engine script in SED to help developers parse through credit card numbers in Unix which is needed prior to implementing Luhn's algorithm .

Create a file "ccengine.sed" with following content


s/[./\-#$%^&@_*!\ ]4[0-9]\{3\}[./\-#$%^&@_*!\ ]*[0-9]\{4\}[./\-#$%^&@_*!\ ]*[0-9]\{4\}[./\-#$%^&@_*!\ ]*[0-9]\{4\}[./\-#$%^&@_*!\ ]*[0-9]\{3\}/\*\*\*\*-\*\*\*\*-\*\*\*\*-\*\*\*\*-\*\*\*/g
s/[./\-#$%^&@_*!\ ]5[1-5][0-9]\{2\}[./\-#$%^&@_*!\ ]*[0-9]\{4\}[./\-#$%^&@_*!\ ]*[0-9]\{4\}[./\-#$%^&@_*!\ ]*[0-9]\{4\}[./\-#$%^&@_*!\ ]*[0-9]\{3\}/\*\*\*\*-\*\*\*\*-\*\*\*\*-\*\*\*\*/g
s/[./\-#$%^&@_*!\ ]6011[./\-#$%^&@_*!\ ]*[0-9]\{4\}[./\-#$%^&@_*!\ ]*[0-9]\{4\}[./\-#$%^&@_*!\ ]*[0-9]\{4\}[./\-#$%^&@_*!\ ]*[0-9]\{3\}/\*\*\*\*-\*\*\*\*-\*\*\*\*-\*\*\*\*-\*\*\*/g
s/[./\-#$%^&@_*!\ ]4[0-9]\{3\}[./\-#$%^&@_*!\ ]*[0-9]\{4\}[./\-#$%^&@_*!\ ]*[0-9]\{4\}[./\-#$%^&@_*!\ ]*[0-9]\{4\}/\*\*\*\*-\*\*\*\*-\*\*\*\*-\*\*\*\*/g
s/[./\-#$%^&@_*!\ ]5[1-5][0-9]\{2\}[./\-#$%^&@_*!\ ]*[0-9]\{4\}[./\-#$%^&@_*!\ ]*[0-9]\{4\}[./\-#$%^&@_*!\ ]*[0-9]\{4\}/\*\*\*\*-\*\*\*\*-\*\*\*\*-\*\*\*\*/g
s/[./\-#$%^&@_*!\ ]6011[./\-#$%^&@_*!\ ]*[0-9]\{4\}[./\-#$%^&@_*!\ ]*[0-9]\{4\}[./\-#$%^&@_*!\ ]*[0-9]\{4\}/\*\*\*\*-\*\*\*\*-\*\*\*\*-\*\*\*\*/g
s/[./\-#$%^&@_*!\ ]3[47][0-9]\{13\}/\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*/g
s/[./\-#$%^&@_*!\ ]30[0-5][0-9][./\-#$%^&@_*!\ ]*[0-9]\{4\}[./\-#$%^&@_*!\ ]*[0-9]\{4\}[./\-#$%^&@_*!\ ]*[0-9]\{2\}/\*\*\*\*-\*\*\*\*-\*\*\*\*-\*\*\*\*/g
s/[./\-#$%^&@_*!\ ]36[0-9]\{2\}[./\-#$%^&@_*!\ ]*[0-9]\{4\}[./\-#$%^&@_*!\ ]*[0-9]\{4\}[./\-#$%^&@_*!\ ]*[0-9]\{2\}/\*\*\*\*-\*\*\*\*-\*\*\*\*-\*\*\*\*/g
s/[./\-#$%^&@_*!\ ]38[0-9]\{2\}[./\-#$%^&@_*!\ ]*[0-9]\{4\}[./\-#$%^&@_*!\ ]*[0-9]\{4\}[./\-#$%^&@_*!\ ]*[0-9]\{2\}/\*\*\*\*-\*\*\*\*-\*\*\*\*-\*\*\*\*/g
s/[./\-#$%^&@_*!\ ]3[47][0-9]\{2\}[./\-#$%^&@_*!\ ]*[0-9]\{4\}[./\-#$%^&@_*!\ ]*[0-9]\{4\}/\*\*\*\*\*\*\*\*\*\*\*\*\*/g
s/4[0-9]\{3\}[- ]*[0-9]\{4\}[- ]*[0-9]\{4\}[- ]*[0-9]\{4\}[- ]*[0-9]\{3\}/\*\*\*\*-\*\*\*\*-\*\*\*\*-\*\*\*\*-\*\*\*/g
s/5[1-5][0-9]\{2\}[- ]*[0-9]\{4\}[- ]*[0-9]\{4\}[- ]*[0-9]\{4\}[- ]*[0-9]\{3\}/\*\*\*\*-\*\*\*\*-\*\*\*\*-\*\*\*\*-\*\*\*/g
s/6011[- ]*[0-9]\{4\}[- ]*[0-9]\{4\}[- ]*[0-9]\{4\}[- ]*[0-9]\{3\}/\*\*\*\*-\*\*\*\*-\*\*\*\*-\*\*\*\*-\*\*\*/g
s/4[0-9]\{3\}[- ]*[0-9]\{4\}[- ]*[0-9]\{4\}[- ]*[0-9]\{4\}/\*\*\*\*-\*\*\*\*-\*\*\*\*-\*\*\*\*/g
s/5[1-5][0-9]\{2\}[- ]*[0-9]\{4\}[- ]*[0-9]\{4\}[- ]*[0-9]\{4\}/\*\*\*\*-\*\*\*\*-\*\*\*\*-\*\*\*\*/g
s/6011[- ]*[0-9]\{4\}[- ]*[0-9]\{4\}[- ]*[0-9]\{4\}/\*\*\*\*-\*\*\*\*-\*\*\*\*-\*\*\*\*/g
s/3[47][0-9]\{13\}/\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*/g
s/4[0-9]\{3\}[- ]*[0-9]\{4\}[- ]*[0-9]\{4\}[- ]*[0-9]\{4\}[- ]*[0-9]\{3\}/\*\*\*\*-\*\*\*\*-\*\*\*\*-\*\*\*\*-\*\*\*/g
s/5[1-5][0-9]\{2\}[- ]*[0-9]\{4\}[- ]*[0-9]\{4\}[- ]*[0-9]\{4\}[- ]*[0-9]\{3\}/\*\*\*\*-\*\*\*\*-\*\*\*\*-\*\*\*\*-\*\*\*/g
s/6011[- ]*[0-9]\{4\}[- ]*[0-9]\{4\}[- ]*[0-9]\{4\}[- ]*[0-9]\{3\}/\*\*\*\*-\*\*\*\*-\*\*\*\*-\*\*\*\*-\*\*\*/g
s/4[0-9]\{3\}[- ]*[0-9]\{4\}[- ]*[0-9]\{4\}[- ]*[0-9]\{4\}/\*\*\*\*-\*\*\*\*-\*\*\*\*-\*\*\*\*/g
s/5[1-5][0-9]\{2\}[- ]*[0-9]\{4\}[- ]*[0-9]\{4\}[- ]*[0-9]\{4\}/\*\*\*\*-\*\*\*\*-\*\*\*\*-\*\*\*\*/g
s/6011[- ]*[0-9]\{4\}[- ]*[0-9]\{4\}[- ]*[0-9]\{4\}/\*\*\*\*-\*\*\*\*-\*\*\*\*-\*\*\*\*/g
s/3[47][0-9]\{13\}/\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*/g
s/30[0-5][0-9][- ]*[0-9]\{4\}[- ]*[0-9]\{4\}[- ]*[0-9]\{2\}/\*\*\*\*-\*\*\*\*-\*\*\*\*-\*\*\*\*/g
s/36[0-9]\{2\}[- ]*[0-9]\{4\}[- ]*[0-9]\{4\}[- ]*[0-9]\{2\}/\*\*\*\*-\*\*\*\*-\*\*\*\*-\*\*\*\*/g
s/38[0-9]\{2\}[- ]*[0-9]\{4\}[- ]*[0-9]\{4\}[- ]*[0-9]\{2\}/\*\*\*\*-\*\*\*\*-\*\*\*\*-\*\*\*\*/g
s/[^0-9]4[0-9]\{3\}[- ]*[0-9]\{4\}[- ]*[0-9]\{4\}[- ]*[0-9]/\*\*\*\*-\*\*\*\*-\*\*\*\*-\*\*\*\*/g
s/[#- ]4[0-9]\{3\}[- ]*[0-9]\{4\}[- ]*[0-9]\{4\}/\*\*\*\*-\*\*\*\*-\*\*\*\*-\*\*\*\*/g
s/3[47][0-9]{2\}[- ]*[0-9]\{4\}[- ]*[0-9]\{4\}[- ]*[0-9]\{3\}/\*\*\*\*-\*\*\*\*-\*\*\*\*-\*\*\*/g
Say CCENGINE=ccengine.sed
INFILE= is your initial load
OUTFILE = your output with CC numbers.

If you have the dump of database as a bcp or sqlload or just a file , just need to translate the file using following sed command

sed -f $CCENGINE $INFILE >$OUTFILE

bcp or sqlload back dependending on whether you are using Sybase or Oracle.Keep it Simple !

Choice Overload

Life is full of choices and a good life is all about making the right decision . An interesting talk from Sheena Iyengar in TED on decision making during such choice overload is quite intriguing. I am also quite influenced by Jonah Lehrer's Decisive Moment in the way it deciphers how we make decisions based on our past experience.