Showing posts with label Data Architecture. Show all posts
Showing posts with label Data Architecture. Show all posts
Tuesday, September 08, 2026

Apache Arrow vs Apache Iceberg

›
Many folks get confused between these two similar open standards; as both of them are used for columnar data.  Apache Arrow defines how data...
Tuesday, October 14, 2025

Dataspaces, Gaia-X and Simpl Middleware

›
In today’s digital economy, data is a powerful resource that drives innovation, economic growth, and improved public services. Recognizing t...
Monday, September 22, 2025

Ruminating on Parquet, Delta Lake and Iceberg

›
In this blogpost, I shall try to demystify some of the concepts around Parquet, Delta Lake and Iceberg in three easy learning steps. Step 1:...
Wednesday, April 30, 2025

Ruminating on "Zero ETL"

›
Traditional ETL workflows consist of three separate stages:  Extracting data from source systems Transforming it to fit analytical needs Loa...
Wednesday, March 19, 2025

Ruminating on Data Mesh and Data-as-a-Product

›
In today’s fast-paced, data-driven world, organizations are constantly looking for better ways to handle the massive amounts of data they ge...
Tuesday, January 21, 2025

Ruminating on Standardizing Data

›
In the realm of statistics, we frequently face datasets of varied sizes and units. This might make it difficult to compare variables or use ...
Friday, November 03, 2023

Ruminating on Debezium CDC

›
Debezium is a distributed open source platform for change data capture (CDC). It collects real-time changes to database tables and transmits...
Wednesday, July 20, 2022

Ruminating on Data Lakehouse

›
 In my previous blog posts, we had discussed about Data Lakes and Snowflake architecture .  Since Snowflake combines the abilities of a tra...
Wednesday, June 01, 2022

Ruminating on Data Gravity

›
Data Gravity is a concept that states that as more and more data gets accumulated (analogy of mass increasing), there would be greater attra...
Saturday, September 08, 2018

Batch ETL to Stream Processing

›
Many of our customers are moving their traditional ETL jobs to real-time stream processing. The following article is an excellent read of ...
Wednesday, August 16, 2017

Cool open source tool for ER diagrams

›
Recently one of my colleagues introduced me to a cool Java tool called SchemaSpy . SchemaSpy is a java tool that can be used to create bea...
Monday, May 30, 2016

Ruminating on IoT datastores

›
The most popular data-store choice for storing a high volume of IoT sensor data are NoSQL time-series databases.  The following link con...
Thursday, February 25, 2016

Ruminating on EMPI

›
EMPI (Enterprise Master Patient Index) is a consolidated hub of all patient related information that would act as a single source of truth....
Wednesday, February 12, 2014

Ruminating on Star Schema

›
Sharing a good YouTube video on the basic concepts of a Star Schema. Definitely worth a perusal for anyone wanting a primer on DW schemas. ...
Thursday, September 19, 2013

Transactions in MongoDB

›
In my previous blog post , we had gone through some of the advantages of MongoDB in terms of schema flexibility and performance. The Metlif...
Wednesday, September 04, 2013

NoSQL for Customer Hub MDM

›
The following article on informationweek  is an interesting read on the use of MongoDB NoSQL for building a customer MDM solution. http:/...
Sunday, July 14, 2013

Street address normalization for MDM solutions

›
Quite often we need to verify street addresses or normalize them to check for duplicate customers. The following article gives a good overv...
Thursday, July 11, 2013

Importance of Geocoding for business

›
Geocoding is the process of finding out the geographical coordinates (latitude/longitude) from street address, post code, etc. A lot of org...

Householding and Hierarchy Management

›
Found this rather interesting article on householding concepts in MDM. Many organizations struggle to define business rules to identify cus...
Tuesday, June 11, 2013

Ruminating on Data Masking

›
A lot of organizations are interested in 'Data Masking' and are actively looking out for solutions around the same. IBM and Informa...