The famous WinFile.mqh file from MTIntelligence.com

WinFile.mqh

//+------------------------------------------------------------------+
//|                              Copyright © 2009 MTIntelligence.com |
//|                                    http://www.mtintelligence.com |
//+------------------------------------------------------------------+
#property copyright "Copyright © 2009 MTIntelligence.com"
#property link      "http://www.mtintelligence.com"


// *************************************************************************************
//
// OVERVIEW
//
// Library which imports the CreateFile, ReadFile etc functions from the Win32 API
// and provides wrappers making them easier to use.
//
// N.B. Should be installed in the experts\include folder, and requires 
// "Allow DLL Imports" to be turned on.
//
// Much the same as http://codebase.mql4.com/4086 in terms of allowing access to
// the full hard disk (whereas the built-in MQL file functions restrict access to
// only the experts/files and tester/files directories).
//
// Differences/improvements from http://codebase.mql4.com/4086 are as follows:
//
//    *  Provides a read function which uses a 200-byte buffer, whereas
//       http://codebase.mql4.com/4086 seems to read the file 1 byte at a time.
//    *  Provides a ReadLineFromFile() function which can be used to read
//       CRLF-terminated (or pipe-terminated etc) lines, rather than having to
//       read an entire file into memory and then split it.
//    *  Provides functions for reading and writing an array of doubles.
//    *  Provides a wrapper function for opening a new file in append mode.
//    *  Provides wrapper functions with parameters for opening files in shared mode.
//    *  The _lcreat() functions etc used in http://codebase.mql4.com/4086 are
//       deprecated by Microsoft, and ideally shouldn't be used.
//
//
// FUNCTIONS
//
// As with the native MQL file functions, you must get a file handle before doing
// read or write operations. You can either use the underlying CreateFileA()
// function in the API, or one of the wrappers which this library provides:
//
//    OpenNewFileForWriting()          Opens a file for writing, and overwrites it if it already exists
//    OpenExistingFileForWriting()     Opens a file for writing, returning an error if it does not already exist
//    OpenExistingFileForReading()     Opens a file for reading, returning an error if it does not already exist
//
// All of these return an integer file handle. Success of the call can be
// checked using IsValidFileHandle(). Please note that the file handles from
// CreateFileA() are *not* interchangeable with MQL's own file functions,
// or with the handles used in http://codebase.mql4.com/4086
//
// Once a file has been opened, it *must* subsequently be closed. If not, the
// file will remain open until MT4 is closed. You can either use the CloseHandle()
// function in the API, or the following wrapper:
//
//    CloseFile()                      Closes an open file (can safely be called on handles which are already closed)
//
// You can write text to a file using the WriteToFile() function:
//
//    WriteToFile()                    Writes a block of text to a file
//
// You can then either read all the text from a file in one go, 
// or you can read the file contents in "lines" (which can be pipe-delimited
// etc rather than CRLF-delimited lines):
//
//    ReadWholeFile()                  Returns a string containing the full contents of a file 
//    ReadLineFromFile()               Reads from a file up to the next terminator (e.g. | or CRLF)
//
// You can also read and write arrays of doubles (with any number of dimensions):
//
//    WriteDoubleArrayToFile()         Writes an array of doubles to a file 
//    ReadDoubleArrayFromFile()        Reads an array of doubles from a file 
//
// After reading/writing you can move the file pointer around in a way analogous to
// MQL's own FileSeek() function. You can either use the SetFilePointer() API
// call directly, or you can use one of the following. (N.B. Appending to
// a file using OpenExistingFileForWriting() simply opens a file for writing and
// then uses SetFilePointer() to move to its end).
//
//    MoveToFileStart()                Move the pointer to the start of the file 
//    MoveToFileEnd()                  Move the pointer to the end of the file 
//
// Finally, the library provides a few service functions:
//
//    DeleteFile()                     Deletes a file (simple wrapper around DeleteFileA() in the API)
//    DoesFileExist()                  Checks whether a file with the given name exists
//    StringSplit()                    Splits a delimited line into an array of sub-strings 
//
//
// NOTES ON NASTY WORKAROUND:
//
// MQL only allows each API function to be imported once. You cannot import the same 
// function twice, with different aliases and different parameter lists. This leads 
// to a problem because, in order to read/write both strings and arrays, you would 
// ideally have two definitions of the imported ReadFile() and WriteFile() functions, 
// one pair taking string parameters and the other pair taking array parameters.
//
// As a result, this code has to use a complicated and nasty workaround. Both ReadFile()
// and WriteFile() are declared as taking an integer parameter, which is the address
// in memory of the input/output buffer. The code then gets the addresses of strings
// and arrays as an int using the following tricks:
//
//    *  The address of a string is obtained using MulDiv() - a well-known trick from VB6 days 
//    *  When writing an array, the code allocates a block of memory using LocalAlloc(),
//       recording the returned address as an int, and then copies the array into the
//       new memory using RtlMoveMemory(). (The memory from LocalAlloc() is then
//       obviously freed again before returning.)
//    *  When reading into an array, the code gets the address of the array by passing
//       the array to GlobalLock(). This simply passes back the address of the
//       memory passed in, which is stored as an int.
//
// *************************************************************************************





// *************************************************************************************
// Constants used by the file functions in the Windows API
// *************************************************************************************

#define GENERIC_READ            0x80000000
#define GENERIC_WRITE           0x40000000

#define FILE_SHARE_READ         1
#define FILE_SHARE_WRITE        2

#define CREATE_NEW              1
#define CREATE_ALWAYS           2
#define OPEN_ALWAYS             4
#define OPEN_EXISTING           3
#define TRUNCATE_EXISTING       5

#define FILE_BEGIN              0
#define FILE_CURRENT            1
#define FILE_END                2

#define INVALID_HANDLE_VALUE    -1



// *************************************************************************************
// Import of functions from the Windows SDK. Note the need in MQL to use an array when
// passing integers by reference to a DLL (i.e. LPDWORD parameters)
// *************************************************************************************

#import "kernel32.dll"
   int CreateFileA(string Filename, int AccessMode, int ShareMode, int PassAsZero, int CreationMode, int FlagsAndAttributes, int AlsoPassAsZero);
   int ReadFile(int FileHandle, int BufferPtr, int BufferLength, int & BytesRead[], int PassAsZero);
   int WriteFile(int FileHandle, int BufferPtr, int BufferLength, int & BytesWritten[], int PassAsZero);
   int SetFilePointer(int FileHandle, int Distance, int PassAsZero, int FromPosition);
   int GetFileSize(int FileHandle, int PassAsZero);
   int CloseHandle(int FileHandle);

   bool DeleteFileA(string Filename);

   // See notes above for the explanation of the following...
   
   // Used for converting the address of a string into an integer
   int MulDiv(string X, int N1, int N2);

   // Used for temporary conversion of an array into a block of memory, which
   // can then be passed as an integer to ReadFile
   int LocalAlloc(int Flags, int Bytes);
   int RtlMoveMemory(int DestPtr, double & Array[], int Length);
   int LocalFree(int lMem);

   // Used for converting the address of an array to an integer 
   int GlobalLock(double & Array[]);
   bool GlobalUnlock(int hMem);
#import




// *************************************************************************************
// Determines whether a file exists. N.B. A file can exist without being openable, if
// it is already in use by a caller who has not specified FILE_SHARE_READ or 
// FILE_SHARE_WRITE
// *************************************************************************************

bool DoesFileExist(string FileName)
{
   int FileHandle = CreateFileA(FileName, 0, 0, 0, OPEN_EXISTING, 0, 0);
   if (IsValidFileHandle(FileHandle))
   {
      CloseHandle(FileHandle);
      return (true);
   } else {
      return (false);
   }
}


// *************************************************************************************
// Opens a file for writing, and overwrites its contents (setting its length to zero) if
// it already exists. The return value is the file handle for use in subsequent calls, 
// or INVALID_HANDLE_VALUE if the operation fails. This is a simple wrapper around CreateFileA, 
// and overwrites the file if it already exists by specifying CREATE_ALWAYS. The call will 
// fail if the file is already in use by something else.
// *************************************************************************************

int OpenNewFileForWriting(string FileName, bool ShareForReading = false)
{
   int ShareMode = 0;
   if (ShareForReading) ShareMode = FILE_SHARE_READ;
   return (CreateFileA(FileName, GENERIC_WRITE, ShareMode, 0, CREATE_ALWAYS, 0, 0));
}


// *************************************************************************************
// Opens a existing file for writing and, by default, opens it for appending rather
// than overwriting data already in the file. The return value is the file handle for 
// use in subsequent calls, or INVALID_HANDLE_VALUE if the operation fails. 
// This is a simple wrapper around CreateFileA, using OPEN_ALWAYS so that the file 
// is opened if it already exists, or created if not already in existence.
// The call will fail if the file has already been opened for writing by somebody else. 
// *************************************************************************************

int OpenExistingFileForWriting(string FileName, bool Append = true, bool ShareForReading = false)
{
   int ShareMode = 0;
   if (ShareForReading) ShareMode = FILE_SHARE_READ;
   int FileHandle = CreateFileA(FileName, GENERIC_WRITE, ShareMode, 0, OPEN_ALWAYS, 0, 0);
   if (IsValidFileHandle(FileHandle) && Append)
   {
      SetFilePointer(FileHandle, 0, 0, FILE_END);
   }
   
   return (FileHandle);  
}


// *************************************************************************************
// Opens a existing file for reading and, by default, opens it for appending rather
// than overwriting data already in teh file. The return value is the file handle for 
// use in subsequent calls, or INVALID_HANDLE_VALUE if the operation fails. 
// This is a simple wrapper around CreateFileA, using OPEN_EXISTING so that the call
// fails if the file does not already exist. By default, the optional parameters allow 
// other callers to read from the file, but not to write to it. The function will
// fail if somebody else has already opened the file for writing without specifying
// FILE_SHARE_READ.
// *************************************************************************************

int OpenExistingFileForReading(string FileName)
{
   int FileHandle = CreateFileA(FileName, GENERIC_READ|GENERIC_WRITE, 0, 0, OPEN_EXISTING, 0, 0);
   return (FileHandle);
}


// *************************************************************************************
// Checks to see if a file handle is valid.
// *************************************************************************************

bool IsValidFileHandle(int FileHandle)
{
   return (FileHandle != INVALID_HANDLE_VALUE);
}


// *************************************************************************************
// Writes a string to a file handle. Returns True if the data is written in its entirety.
// Can return False if the data was only partially written. However, a False return 
// value much more commonly indicates that the file handle is invalid - i.e. has been
// opened for reading rather than writing. Can be called multiple times to append
// blocks of data to a file.
// *************************************************************************************

bool WriteToFile(int FileHandle, string DataToWrite)
{
   // Receives the number of bytes written to the file. Note that MQL can only pass 
   // arrays as by-reference parameters to DLLs
   int BytesWritten[1] = {0};

   // Get the length of the string 
   int szData = StringLen(DataToWrite);

   // Do the write 
   WriteFile(FileHandle, MulDiv(DataToWrite, 1, 1), szData, BytesWritten, 0);
   
   // Return true if the number of bytes written matches the expected number 
   return (BytesWritten[0] == szData);   
}


// *************************************************************************************
// Reads a file's entire contents into a string. Can return a blank string either if
// the file is empty, or if the read failed - because the file handle is invalid, or
// because the file has been opened for writing rather than reading.
// *************************************************************************************

string ReadWholeFile(int FileHandle)
{
   // Move to the start of the file 
   SetFilePointer(FileHandle, 0, 0, FILE_BEGIN);
   
   // String which holds the combined file 
   string strCombinedFile = "";
   
   // Keep reading from the file until reads fail because we've reached the end (or 
   // because the file handle is not valid for reading)
   bool bContinueRead = true;
   
   while (bContinueRead)
   {
      // Receives the number of bytes read from the file. Note that MQL can only pass 
      // arrays as by-reference parameters to DLLs
      int BytesRead[1] = {0};
      
      // 200-byte buffer...
      string ReadBuffer = "01234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789";
      int BufferLength = StringLen(ReadBuffer);
      
      // Do a read of up to 200 bytes 
      ReadFile(FileHandle, MulDiv(ReadBuffer, 1, 1), BufferLength, BytesRead, 0);
      
      // Check whether any data has been read...
      if (BytesRead[0] != 0)
      {
         // Add the data which has been read to the combined string 
         strCombinedFile = StringConcatenate(strCombinedFile, StringSubstr(ReadBuffer, 0, BytesRead[0]));
         bContinueRead = true;
      } else {
         // Read failed. Must be at the end of the file (or the file handle is not valid for reading)
         bContinueRead = false;
      }
   }

   return (strCombinedFile);
}


// *************************************************************************************
// Reads a line from a file. The return value can be blank if the end of the file 
// has been reached, or if the file handle is simply not valid for reading. The 
// line-end terminator to look for can be specified using the optional second parameter.
// This can be set to "\r" rather than "\r\n" to read CR rather than CRLF terminated lines.
// Can also be set to e.g. "|" to read files which are pipe-delimited rather than 
// CRLF-delimited
// *************************************************************************************

string ReadLineFromFile(int FileHandle, string Terminator = "\r\n")
{
   // Holds the line which is eventually returned to the caller 
   string Line = "";

   // Keep track of the file pointer before we start doing any reading 
   int InitialFilePointer = SetFilePointer(FileHandle, 0, 0, FILE_CURRENT);
   
   // Keep reading from the file until we get the end of the line, or the end of the file 
   bool bContinueRead = true;
   while (bContinueRead)
   {
      // Receives the number of bytes read from the file. Note that MQL can only pass 
      // arrays as by-reference parameters to DLLs
      int BytesRead[1] = {0};

      // 200-byte buffer...
      string ReadBuffer = "01234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789";
      int BufferLength = StringLen(ReadBuffer);

      // Do a read of up to 200 bytes 
      ReadFile(FileHandle, MulDiv(ReadBuffer, 1, 1), BufferLength, BytesRead, 0);

      // Check whether any data has been read...
      if (BytesRead[0] != 0)
      {
         // Add the new data to the line we've built so far
         Line = StringConcatenate(Line, StringSubstr(ReadBuffer, 0, BytesRead[0]));

         // Does the line now contain the specified terminator?
         int pFindTerminator = StringFind(Line, Terminator);
         if (pFindTerminator != -1)
         {
            // The line does contain the specified terminator. Remove it from the data we're 
            // going to pass back to the caller
            Line = StringSubstr(Line, 0, pFindTerminator);
            
            // We've almost certainly read too much data - i.e. the latest 200 byte block 
            // intrudes into the next line. Need to adjust the file pointer to the start
            // of the next line. This must be the file pointer before we started reading, plus
            // the length of the line we've read, plus the length of the terminator 
            SetFilePointer(FileHandle, InitialFilePointer + StringLen(Line) + StringLen(Terminator), 0, FILE_BEGIN);
         
            // Stop reading 
            bContinueRead = false;
         } else {
            // The line read so far does not yet contain the specified terminator 
            bContinueRead = true;
         }
         
      } else {
         // Either at the end of the file, or the file handle is not valid for reading 
         bContinueRead = false;
      }
   }

   return (Line);
}


// *************************************************************************************
// Checks to see if the file's pointer is currently at the end of the file. Can be
// used e.g. with repeated calls to ReadLineFromFile() to keep reading until the
// end of the file is reached 
// *************************************************************************************

bool IsFileAtEnd(int FileHandle)
{
   int CurrentFilePointer = SetFilePointer(FileHandle, 0, 0, FILE_CURRENT);
   return (CurrentFilePointer >= GetFileSize(FileHandle, 0));
}


// *************************************************************************************
// Writes an array of doubles to a file. Returns true if the entire array is successfully 
// written. Can return false if the data is only partially written but, more normally,
// a false return value indicates that the file handle is not valid for writing.
// *************************************************************************************

bool WriteDoubleArrayToFile(int FileHandle, double & Array[], int Precision = 6)
{
   // Get the total number of elements in the array 
   int sz = 1;
   for (int iDim = 0; iDim < ArrayDimension(Array); iDim++)
   {
      sz *= ArrayRange(Array, iDim);
   }

   // Quit now if the array is empty 
   if (!sz) return (false);

   // Get the size of the array in bytes   
   int szBytes = sz * 8;
   
   // Allocate a block of memory and copy the array into that. 
   // (This step is necessary because we have to pass an integer parameter to ReadFile -
   // see the notes above)
   int lMem = LocalAlloc(0, szBytes);
   RtlMoveMemory(lMem, Array, szBytes);

   // Receives the number of bytes written to the file. Note that MQL can only pass 
   // arrays as by-reference parameters to DLLs
   int BytesWritten[1] = {0};

   // Do the write 
   WriteFile(FileHandle, lMem, szBytes, BytesWritten, 0);
      
   // Free the temporary memory   
   LocalFree(lMem);
   
   // Indicate whether the write succeeded in full   
   return (szBytes == BytesWritten[0]);
}


// *************************************************************************************
// Reads a double array from a file. The dimensions of the array **MUST** be set 
// before calling the function. It returns true if the entire array was read from disk.
// False indicates either that there was insufficient data in the file, or that the
// file handle is simply not valid for reading. If the function returns false then
// the contents of the array are effectively random.
// *************************************************************************************

bool ReadDoubleArrayFromFile(int FileHandle, double & Array[])
{
   // Get the total number of elements in the array 
   int sz = 1;
   for (int iDim = 0; iDim < ArrayDimension(Array); iDim++)
   {
      sz *= ArrayRange(Array, iDim);
   }

   // Quit if the array has no elements
   if (!sz) return (false);

   // Get the size of the double array in bytes     
   int szBytes = sz * 8;

   // Nasty workaround (see notes above). Get the address in memory of the array 
   int pMem = GlobalLock(Array);

   // Receives the number of bytes written to the file. Note that MQL can only pass 
   // arrays as by-reference parameters to DLLs
   int BytesRead[1] = {0};

   // Do the read from the file 
   ReadFile(FileHandle, pMem, szBytes, BytesRead, 0);

   // Undo the temporary memory lock which was required to get the address of the array    
   GlobalUnlock(pMem);

   // See if the entire expected amount of data was returned 
   return (szBytes == BytesRead[0]);
}


// *************************************************************************************
// Simple wrappers around SetFilePointer(), moving to the start and end of a file 
// *************************************************************************************

bool MoveToFileStart(int FileHandle)
{
   return (SetFilePointer(FileHandle, 0, 0, FILE_BEGIN) != -1);
}

bool MoveToFileEnd(int FileHandle)
{
   return (SetFilePointer(FileHandle, 0, 0, FILE_END) != -1);
}


// *************************************************************************************
// Simple renaming wrapper around CloseHandle(), making its name more intuitive
// *************************************************************************************

bool CloseFile(int FileHandle)
{
   CloseHandle(FileHandle);
}

// *************************************************************************************
// Simple renaming wrapper around DeleteFileA(), making its name more intuitive
// *************************************************************************************

bool DeleteFile(string FileName)
{
   return (DeleteFileA(FileName));
}


// *************************************************************************************
// Service function for splitting a string into an array of values. For example,
// it can be used to take a comma-separated line (read using ReadLineFromFile() etc)
// and break it into an array of its comma-delimited parts.
// Example usage is:
//      string Results[];
//      StringSplit("abc,def,ghi", ",", Results);
// *************************************************************************************

void StringSplit(string InputString, string Separator, string & ResultArray[])
{
   int lenSeparator = StringLen(Separator), NewArraySize;
   while (InputString != "") 
   {
      int p = StringFind(InputString, Separator);
      if (p == -1) {
         NewArraySize = ArraySize(ResultArray) + 1;
         ArrayResize(ResultArray, NewArraySize);      
         ResultArray[NewArraySize - 1] = InputString;
         InputString = "";
      } else {
         NewArraySize = ArraySize(ResultArray) + 1;
         ArrayResize(ResultArray, NewArraySize);      
         ResultArray[NewArraySize] = StringSubstr(InputString, 0, p);
         InputString = StringSubstr(InputString, p + lenSeparator);
         if (InputString == "")
         {
            ArrayResize(ResultArray, NewArraySize + 1);      
            ResultArray[NewArraySize] = "";
         }

      }     
   }
}

can it use for read data and make a stategy that only have good results in backtest and different results in real live account?