https://github.com/sohamdixit81/Distribute-llvm-lit.git
-
Links to commits that have been merged:
Bugzilla https://bugs.llvm.org/show_bug.cgi?id=41152
Phabricator (Improve dumping of empty location expressions) https://reviews.llvm.org/D103502
To run the test cases in a distributed way, because as the test cases get sent to the execution machine to run by the distributed system then it will take less time to execute. To reduce the execution time of the test. We have used the HTCondor framework just as an example. you may use any framework of your choice. Expected results: An easy to use harness as described above. Some evidence that given a distributed system, a user can expect the speed up in execution, if they are using that harness.
The LLVM lit test suites consist of thousands of small independent tests. Due to the large number of tests, it can take a long time to run the full suite, even on a high-specification computer. Builds are already distributable across multiple computers available on the same network, using software such as distcc or icecream, so running tests on a single machine becomes a potential bottleneck. One way to speed up running of the tests could be to distribute test execution across many computers. Lit provides a test sharding mechanism, which allows multiple computers to run parts of the same testsuite in tandem, but this currently assumes access to a single common filesystem, which may not be possible in all cases and a knowledge of which machines the suite can currently be run on. This project’s goal is to update the existing lit harness (or write a wrapper around it) to allow distribution of the tests in this way, with the idea that developers can write their own interface between the harness and the distribution system of their choice. This harness may need to be able to identify test dependencies such as input files and executables, send the tests to the distribution system (possibly in batches), and receive, collate and report the results to the user, in a similar manner to how lit already does.
map.py script- it takes the input from the user which folder you want and maintains the same tree structure. same as build folder and llvm project folder
location.py- it runs the test cases
transfer.sub- Its example of the HTcondor framework. what it does is, it takes the executable as location.py and output folder as transfer_the_file. And its run, the tests are run successfully.
1.The scripts dependent.py which will be checking the dependencies of the test case and will be transferring only that specific file as dependencies to executing node. Hence it will run faster, it will need less time to execute and as well as less space so by this we will get a faster result in less time.
2. Just the integration remaining of all scripts means integration is pending. After this our project goal will be satisfied and if any future changes are needed we will do it!!!!!
3. We are trying to make scripts more generic and according to the changes we will let you know about how to Run, Format needed. All these changes will be given in the Git Repository Readme.md file