Premium Data Products Support Corporate
General Support
- Downloads
- How to Download
- Installation
- Downloading / Installation FAQ
> Data File Corruption
- Antivirus Programs
- Submit Problem Report
- Premium Data Converter
- Smart Charts Copier
- The MetaStock Data Format
- Virtual Environments
Sales Support
- Order Form
- Refunds Policy
- My Account
- Forgot Password
- Price
- Free Trial
- History Only Purchases
- Installing on a Second Machine
- How to Upgrade an ASX Subscription
- Privacy Policy
- Licence Agreement
Premium Data/Stock Market Support
- Premium Data FAQ
- ASX FAQ
DataTools/Futures Support
- DataTools FAQ
- DataTools Service Information
- Understanding Futures Data
- Futures Coverage Notes
- Futures Database Notes
- Futures Contract Detail (Spreadsheet)
- Futures - Continuous Contracts
Forex Support
- Premium Forex FAQ
- Premium Forex Service Information
Charting Programs & Our Data
- Switching To Us From Another Vendor
- Advanced GET FAQ
- AmiBroker FAQ
- Dynamic Trader FAQ
- FCharts FAQ
- Fibotrader FAQ
- Gannalyst FAQ
- Guppytraders Essentials FAQ
- HotTrader FAQ
- Market Analyst FAQ
- MetaStock™ FAQ
- MTPredictor FAQ
- Omnitrader FAQ
- Stock Neuromaster FAQ
- Trading Blox FAQ
- TradeGuider FAQ
- TradeStation FAQ
Market Resources
- Alternative Market Glossary
- Links to Other Sites
Notices
- Release of the 'Norgate Data Updater' platform

Data File Corruption



Data file corruption can be indicated by error messages appearing during an update or by the inappropriate appearance of one or more charts. For instance, if the file order in a folder becomes jumbled with respect to its "master" file, the data for one security will start to be appended to the history for another. This anomaly should show up on a price chart even though the file is being updated successfully.

Some known causes of "MetaStock" format data file corruption are discussed below. Go to the second part of the discussion for ways of recovering.

Possible causes of data file corruption

Simultaneous accessing of data files

If you have two or more charting applications, make sure you do not use them to view data at the same time, or file corruption may ensue. Also, make sure that no charting or analysis application that can access the data is open when you are performing a data update.

Interference from Real-time Anti-Virus Scanning

Careful analysis of repeated file corruption events has shown that the problem could be attributed to real-time anti-virus scanning. When real-time scanning was disabled, daily updates proceeded without error. When scanning was resumed, file corruption invariably recurred.

During an update, numerous files need to be opened and closed. Real-time anti-virus scanning attempts to check all of these files at approximately the same time as the update is dealing with them. If the timing is slightly out, then the two applications are attempting to access the files at the same time and corruption occurs.

If you experience repeated instances of file corruption, it is strongly recommended that you exclude your data folders from real-time anti-virus scanning.

Interference from Cloud-based file hosting systems

Cloud-based file hosting systems (such as DropBox, Google Drive, Microsoft OneDrive) periodically lock files when they are synchronizing them to the Cloud. If data files are locked when a data update is performed, corruption may result.

Power failure during a data update

A power failure during an update, or whilst data files are being accessed by a charting or analysis application, may result in data file corruption. Consider installing an Uninterruptible Power Supply (UPS) as a guard against all data problems caused by loss of power.

Computer component issues

Computers that spontaneously reboot usually have hardware component issues. A failing hard drive is quite likely to cause file corruption in general.

Recovering from data file corruption

1.   Restore a back-up copy of the data, or -
2.   Install a fresh instance of the data.

One way to insure against potential corruption is to perform regular, incremental backups of your data folders with a Backup utility. Failing that, Windows also has some built-in file protection which you may be able to use as a fall-back.

Windows built-in file protection

What to Restore

If you restore an entire database, the corresponding updating program will "know" where to update from.

For Premium Data/Stocks - restore the entire "Stocks" folder (or selectively restore the "ASX", "US" or "World Indices" folders).
For DataTools/Futures - restore the entire "Futures" folder.
For Premium Forex/Forex - restore the entire "Forex" folder.

Installing a fresh Data History

It may be that no sound backup can be found to restore, or only a very old one.

In these situations, you can always recover quite quickly by installing a fresh history (by obtaining the relevant history installer from our Downloads area). Notes on history re-installation are available here.

History installers are always available to current subscribers who purchased the corresponding history at some stage.