Why capture changes?
Capturing changes may be a little new to readers that don't have experience with data engineering or working with enterprise systems, but there are some really good reasons to do so.
Auditing
One of the reasons that's most relevant to smaller companies is auditing. With CDC (and Cainophile), you can easily keep track of everything a user changes in your system, and store it forever. This means every record touch, every login time (assuming you track "last logged in"), every name change, etc will be captured and available for archive or real time monitoring. This is useful for security/forensic purposes, but may also be helpful for debugging, or recovering from a software bug that corrupts data.
Analytics
While having a discrete event log is the preferred way to do event-based analytics, it's not always feasible to instrument your app like this, whether due to time constraints or lack of control. In addition, unless you've built your app using event sourcing, you will be losing changes/events that you didn't consider tracking ahead of time. Using change data capture for analytics allows you to rest easy, knowing that if the business asks a question regarding something that changes in the database, you can answer it.
Indexing
Changes can also be used to maintain a secondary index/cache of your data, such as an Elasticsearch cluster. With Cainophile, you can write some simple integration code that transforms and sends data to Elasticsearch in real-time, or just use it to maintain an ETS table that a Phoenix channel interrogates for real-time updates via Websockets.
<https://bbhoss.io/posts/announcing-cainophile/#why-capture-changes>
Capturing changes may be a little new to readers that don't have experience with data engineering or working with enterprise systems, but there are some really good reasons to do so.
Auditing
One of the reasons that's most relevant to smaller companies is auditing. With CDC (and Cainophile), you can easily keep track of everything a user changes in your system, and store it forever. This means every record touch, every login time (assuming you track "last logged in"), every name change, etc will be captured and available for archive or real time monitoring. This is useful for security/forensic purposes, but may also be helpful for debugging, or recovering from a software bug that corrupts data.
Analytics
While having a discrete event log is the preferred way to do event-based analytics, it's not always feasible to instrument your app like this, whether due to time constraints or lack of control. In addition, unless you've built your app using event sourcing, you will be losing changes/events that you didn't consider tracking ahead of time. Using change data capture for analytics allows you to rest easy, knowing that if the business asks a question regarding something that changes in the database, you can answer it.
Indexing
Changes can also be used to maintain a secondary index/cache of your data, such as an Elasticsearch cluster. With Cainophile, you can write some simple integration code that transforms and sends data to Elasticsearch in real-time, or just use it to maintain an ETS table that a Phoenix channel interrogates for real-time updates via Websockets.
<https://bbhoss.io/posts/announcing-cainophile/#why-capture-changes>
sincerely,