This year at GSOC I applied to make a similar tool as ccache, but for mono. I worked in my repository entirely, later the mono team will fetch my repository and integrate it with their features. There are 3 main branches in the repository:
- Where the tool is located (master)
- Where are the unit tests made in MSTest (UnitTests)
- Where is the nuget package. (NugetPackage)
Link to the repository: https://github.com/dcalance/GSOC17
The tool had to have the following features:
- Able to prevent repeated compilation of a program.
- Reports it’s own failures with error messages.
- Returns the error code that the underlying compiler returns (important for use with Make).
- Does not fail to recompile files that do change.
- Captures resources and other non-code files passed to compiler as well.
At the end these were the features implemented:
- Able to prevent repeated compilation of a program.
- Reports it’s own failures with error messages.
- Returns the error code that the underlying compiler returns (important for use with Make).
- Does not fail to recompile files that do change.
- Captures resources and other non-code files passed to compiler as well.
- Configuration file that allows to set a different compiler and such things as default extension, cache location, arguments to be ignored.
- Storage of the warnings at the compilation time, and printing them at a repeated compilation.
- Installation scripts for windows and unix.
- Nuget package.
The process of making the project was the following:
- Reading documentation about ccache and mcs.
- Make a method that will be able to execute a cmd/bash process with passed arguments.
- Make the first working prototype that is able to store the cache and load it at a repeated compilation.
- Make a configuration file that will replace all the hardcoded values.
- Handle the possible error that can happen at argument passing.
- Making a new parser to parse the compiler line.
- Adding a more powerful argument parser by using OptionSet from Mono.Options.
- Adding support for response files and modules.
- Creating install files for windows and linux.
The tool is a class library that has a bunch of classes:
- CSCache - is the main class that recieves the compiler with arguments in constructor and has a method
void Cache()that caches the input arguments if there is no error. - Config - class that handles the config. It generates config if it doesn't exist and stores the values read from config file in properties.
- ConsoleTools - a set of static methods that are used for executing and outputing error.
- FilesTools - a set of static methods that are used for working with files.
- LibArgs - a set of static methods that execute options passed to the tool.
- MD5Tools - a set of static methods that are for generating hashes.
The tool is minimalistic since the main objective is to be integrated in bigger environments such as monodevelop and it should work fast, otherwise there will be no use of the tool.
The classes that are with static members can be reused, that was the main propouse to make them static and public. For example:
Contains the class ConsoleTools with the following methods and fields:
static bool IsUnix- This field can determine if your OS is either unix or windows.static string Execute(string cmdLine, out int errCode)- This method allows execution of a command either in bash or cmd. The execution is based on the enviromnent from where the application was executed.- string cmdLine - The command line passed to the bash/cmd.
- out int errCode - out parameter that returns the exit code of the executed command line.
- return - string that contains the concatenation of standard error and standard output.
static void Error(string msg, int errCode)- Exits the program and returns specified code.- string msg - Error message that will be displayed to user.
- int errCode - Exit code of the application.
Contains the class FilesTools and methods to work with files:
static string[] GetRecurseFiles(string pattern)- Gets all files that are fitting the current pattern.- string pattern - pattern that will be evaluated.
- return - an array of all the files found that correspond to the pattern.
Contains the class MD5Tools and methods for hashing:
static byte[] GenerateFilesCache(List<string> inputF)- Generates a single MD5 hash from all the input files. Uses the following methods from the same class : static byte[] MakeMD5File(string filename), static byte[] CombineHashes(List<byte[]> input).- List inputF - a list of input files location. The method throws unhandled error if the file does not exist.
- return - a byte array that contains the hash that results from combining the hashes of each seaparte file.
static byte[] CombineHashes(List<byte[]> input)- Generates a single MD5 hash from a list of hashes.- List<byte[]> input - a list of hashes of the same size.
- return - a byte array that contains the hash of the input hashes.
static byte[] MakeMD5String(string input)- Generates MD5 hash from the input string.- string input - Input string.
- return - a byte array that is the result of hashing the current string.
static byte[] MakeMD5File(string filename)- Generates MD5 hash from the content of the file.- string filename - The name of the input file.
- return - a byte array that is the result of hashing the contents of the file.
Contains the class ParseTools with methods for parsing:
static string[] ParseArguments(string commandLine)- parses the input string as a command line argument.- string commandLine - the input string that will be parsed.
- return - an array that contains the parsed arguments, all empty elements are removed before return.
static string generateOption(string[] argArr)- parses an array of strings and generates a compatible option for OptionSet from Mono.Options.- string[] argArr - The input array that will be parsed.
- return - a string that will be of form accepted as option in OptionSet. (Example:
o|out|output)
string[] ParseResponseFile(string file)- parses a response file into an array of strings. It uses the method static string[] ParseArguments(string commandLine).- string file - The name of the file that will be parsed.
- return - An array of strings that contains all arguments parsed, if the file does not exist the returned array will be with 0 elements.
static string[] ParseArgArrayWithResponse(string[] input)- Parses a command line that contains response files and generates a final command line that is composed from original arguments and the ones from response files in correct order. This method is using static string[] ParseResponseFile(string file).- string[] input - parsed command line arguments that contain response files. (Response files must be of form: @response.txt)
- return - an array that contains the original arguments with parsed arguments from response file in original order.
static string[] ParseComaSemicolon(string input)- Parses arguments that are delimited by comma or/and semicolon.- string input - string that is delimited with commas and/or semicolons.
- return - array of strings with parsed elements.
There is a bug regarding windows bash that I wasn't able to fix. The problem with it appears when you try to create a windows bash process and pass it a compiler line with arguments. The bug is that the paths that have dashes (example: folder1\input1.cs) does not have dashes anymore and compiler throws error that input files is not found. The problem is not at my level of implementation but either at process class or in the way the windows bash processes the input arguments. This bug may appear at random times.
This project was an interesting experience for me since it was entirely new, I've never worked at compiler level. Now I can understand more things about how compilers work, how arguments are passed and processed, how the output is generated. This project was great for learning because it was not very difficult and at the same time it had specific things that you can learn from it. The mentor was amazing and helped me in difficult situation at the same time leaving the entire thought process to me.