Data Archiving and Purging

The TitleSCAN Web Data Entry Statistics System stores activity data in an SQL table that will be referred to here as the Current Database.   Large amounts of data can accumulate quickly in this database on larger, busy TitleSCAN Web systems.   In order to improve performance and to speed the generation of reports, a TitleSCAN Web administrator may wish to extract older data from the Current Database and move it to a second database that is referred to as the Archive Database.  This process is called Archiving.

The TitleSCAN Web administrator can allow the Archive Database to grow indefinitely, or the administrator can choose to remove the oldest data from the Archive Database in an operation called Purging.  The Purging operation permanently removes data from the Archive Database, which cannot be recovered other than by restoring from disk backup media.

IMPORTANT:  There are three key concepts that must be understood:

Archiving
extracts data from the Current Database and moves it to the Archive Database.

Purging
permanently removes data from the Archive Database.

Reports
can be generated from data in either the Current Database or the Archive Database (although they would normally be generated from data in the Current Database).
 

The Archiving and Purging operations can be performed by clicking on the Archive/Purge  button on the initial page.  When this is done, a screen similar to the one below will be shown.

The information shown at the top portion of the screen shows the current status of the Current and Archive SQL Databases (tables) and the date ranges of the data in those databases.

Date Ranges

All data in the Current and Archive databases are organized in date order.  The Start Date in the screen above shows the earliest date on file for the Operation that is selected (either Archive Data or Purge Data).   If Archive Data is selected, the Start Date shows the earliest date on file in the Current Database.  If Purge Data is selected, the Start Date shows the earliest date on file in the Archive Database.

The Start Date is grayed-out and cannot be changed since the Archive and Purge operations always start with the earliest date through an end date that is specified by the user, which must be a date that is the same as or later than the Start Date but not later than the date of the most recent data in the database.  Consequently, the date ranges of the data in the Current and Archive databases cannot overlap.

Archiving

The Archive Data operation extracts data from the Current Database and moves it to the Archive Database.  This happens in a single SQL transaction, which will be automatically undone (rolled back) in the event that the operation fails prior to a full completion.

To Archive Data from the Current Database to the Archive Database, do the following:

NOTE:  Data archiving can take some time to complete depending upon the amount of data being archived and the current load on the server.   You may find that this operation is best done at off-hours if you are archiving a large amount of data.
 

Purging

The Purge Data operation permanently removes data from the Archive Database.  Once purged, this data cannot be recovered without restoring it from disk backup media.

To Purge Data from the Archive Database, do the following:

NOTE:  Data purging can take some time to complete depending upon the amount of data being purged and the current load on the server.   You may find that this operation is best done at off-hours if you are purging a large amount of data.