VTScada Historical Data Management

A Fast and Compact Database Schema

VTScada’s native Historical database uses a standardized file-based schema that stores data in binary files resulting in very compact storage and fast data read/write access.

Built-in is Better - The VTScada Historian is pre-configured in every application reducing integration time and long-term database maintenance. It is designed to automatically match the number of tags logging data to it. Built-in ODBC, OPC, and Web Services interfaces share data with external business systems.

Log Data from Multiple Sources - Third-party relational databases can also be used as a concurrent synchronized backup to the VTScada Historian. Supported databases include Oracle^®, SQL Server^®, MySQL^®, and SQLite^®. These products must be purchased separately.

Log Data Based on Triggers

• Log on change with user-configurable deadband (default)
• Related event (can also be used for disable/enable)
• Log on time/sample period
• Operator actions (e.g. Entry of a manual value into a numeric data entry field, change of a setpoint, or a control action)

Data is stored directly to the Historian at the time of trigger occurrence. This eliminates any need for separate real-time and historical databases.

Simple Historian Status Monitoring - The Historian Status Monitoring Widget provides instant status of write and storage rates. When write rates exceed storage rates, data is automatically buffered and written in burst mode when the Historian connection is available. VTScada write rates have been tested to 4,000 values per second for a single tag, thus buffering is usually the result of a slow network, underpowered CPU, or slow storage media.

Summary, On-Demand Data

Upon request, the Historian analyzes raw tag data and provides the following time-series summary data, based on a user-definable duration divided into time slices (e.g. one day’s data divided into one hour slices):

• Time-weighted average (analog)
• Minimum (analog)
• Maximum (analog)
• Change in value (analog)
• Value at start (analog)
• Time of Minimum (analog)
• Time of Maximum (analog)
• Totalizer (analog)
• Interpolated (analog)
• Diff between start and end (analog)
• Zero to non-zero transitions (digital)
• Non-zero time (digital)

Database Size, Storage Limiting, and Data Retention Periods - The Historian supports the same number of I/O tags as the licensed SCADA server with which it is integrated. Calculated values logged to the Historian do not count against the total tag count.

The Historian will automatically grow in total storage size to that available on the available drives. Where additional space is required, increasing the space available to a logical drive will automatically make that space available to the Historian.

Due to the efficient size of the Historian’s native binary data file format and the low cost of large storage, the Historian is configured to keep all data by default. However, where storage space is limited, the Historian can be configured to automatically delete data older than X days or keep a specific number of records for each tag. Data is overwritten based on a First-In-First-Out (FIFO) methodology.

Multiple Historian Configuration - Any number of Historians can be created for a single application. Different Historians may be configured to store data for a different period or number of records per tag. The Historians may use similar or different data storage formats. For example Historian #1 may use the file-based storage and Historian #2 may use MS SQL Server or another relational database. Each tag may store data to one Historian.

Redundant Historian Configuration - Since the VTScada Historian is an integrated component of any Runtime or Development Runtime license, any computer running one of these licenses can be configured as a Historian server (e.g. primary, backup, 2^nd backup, 3^rd backup, etc).

Redundant Historians may also use similar or different data storage formats, for example the Primary Historian may use the file-based storage and the Backup Historian may use Oracle or another relational database. Each data point will be stored exactly the same on each redundant Historian. Redundant Historians may be co-located with the Primary Historian or may be geographically separated as long as an IP connection exists between the two.

Bi-directional Synchronization - All redundant Historians will be identical with regard to the same schema and a complete, replicated copy of all data. Timestamps will be matched for each data-point to the millisecond. Should the primary database server fail, associated workstations and Internet clients switch to the next designated database. When it is restored, historical data automatically synchronizes across a local or wide area network at up to 160,000 values per second. This speed is automatically throttled such that real-time communications between SCADA servers is not significantly deteriorated. Any data on any historian that is missing on another will be propagated automatically regardless of how long it has been since the databases have communicated.

Long Term Data Storage Platform (no more archiving) - In the SCADA industry, archiving is typically adopted for long-term historical data storage due, in part, to a) a lack of online drive space, and b) to ensure data was backed up in the event of Historian failure.

The VTScada Historian eliminates the need for data archiving. VTScada’s efficient binary storage eliminates online drive space issues allowing the size of the database to scale as required utilizing the drive space available. New space can be added to the logical drive with the addition of a new physical driver providing unlimited scalability for the Historian. Establishing redundant Historians is a far more robust backup methodology, allowing any number of redundant distributed Historians to be updated in real time.

Secure Historical Data - VTScada’s historical file database cannot be edited within VTScada or by any third party software tool. Not only are the proprietary log files not human readable, VTScada uses a checksum to detect any tampering or corruption.  Note: If you are also logging to a redundant third-party SQL format you will need to secure that databases as per the manufacturer’s instructions.

Viewing Historical Data within VTScada Applications

VTScada includes several methods to access historical data from within an application.

Historical Data Viewer (HDV) Trend View - The HDV provides a continuous view of historical and real-time data on a single plot timeline. This standard VTScada interface includes a pen legend on the bottom and a trend viewing area for wide graphs of both analog and digital values. 

Move the new Marker Line horizontally to see continuously updated values for each plotted analog tag at every selected timestamp. New icons in the Pen Legend allow you to hide individual pens (tags) or edit their appearance.

Simple Trend Generation - You can also configure the HDV as a popup display or as a highly customized trend on any custom page using the new HDV Widget. Generate plot views for specific tags by selecting each tag’s value on a process display, by right-clicking tags in the tag browser, or right-clicking alarms that references the tag in the alarm list.

Historical Data Viewer (HDV) Tabular View - HDV data may also be shown in time-series tabular view. Data may be shown as raw data to the nearest millisecond or as ave, min, or max values.

Additionally, to facilitate time-sliced viewing, a total period may be selected and then further time sliced to display summary data for each time period. For example, 24 hours of data may be shown as 24 separate rows of 1 hour averages for a selected set of tags.

Accessing Historian Data Using External Software

Third-party software products can access VTScada data via the VTScada Connectivity Pack. These are a set of industry standard protocols and methods that provide real-time and/or historical data access to VTScada’s databases.

VTScada OPC Server - Allows OPC Clients to read/write real-time process data from/to VTScada. OPC DA is supported.

VTScada ODBC Server - Support SQL read-only queries to the VTScada Historian using ODBC. Both real-time and historical data is available via this connection.

Upon request, the Historian analyzes raw tag data and provides the following time-series summary data, based on a user-definable duration divided into time slices (ex. one day’s data divided into 1 hour time slices):

• Time-weighted average (analog)
• Minimum (analog)
• Maximum (analog)
• Change in value (analog)
• Value at start (analog)
• Time of Minimum (analog)
• Time of Maximum (analog)
• Totalizer (analog)
• Interpolated (analog)
• Diff between start and end (analog)
• Zero to non-zero transitions (digital)
• Non-zero time (digital)

Simplified connectors for reporting packages including XLReporter^® and DreamReport^® are included.

VTScada Web Services – A SOAP (XML) interface providing third-party business systems read/write access from/to VTScada real-time and historical data.