If you are picking up this topic out of order, I encourage you to go hop over to our first blog on it. You may get there quicker by following this link.
Moving on quickly on this blog, we are discussing how to best implement an "ETL" process to migrate a series of OLTP systems into the same target OLTP system. Traditionally, these ETL tasks happen from a variety of OLTP systems used in a large organization to an OLAP system at the same organization.
Suffice it to say that repeatability and / or reuse is not very apparent when working on setting up "Transformation" logic to convert data. Specially when you are converting data from a variety of OLTP systems to a totally different OLTP system. A colleague of mine always argues, regarding reuse, that for something to be reusable it has to be usable the first time. A person cannot reuse something if it cant be used at all. So the first rule to apply, if you want to get reuse of tools under the "Transform" part of the ETL process, is to try as many things as possible. As new things and new processes or styles for handling and shaping data are applied, it is important to keep an eye out for the ones that are working and yielding good results and quickly discard the ones that are not. Anything that can make you more productive is a good item to emulate when you tackle the next OLTP system in the conversion queue.
Reuse can be an elusive goal, so try to keep it simple, look for patterns that re emerge, specially when you come up or create a tool that helps you be more productive.
Until next time!
No comments:
Post a Comment