BTP - a system for analyzing the performance of Web applications. The idea is this: a web application measures the execution time of some interesting pieces of code (for example, queries to the database / cache / external services), and this time tells BTP. BTP handles the data and displays statistics on the interface. At the same time used to measure the normal code in your programming language, without requiring the use of any native extensions, so while we use the BTP from PHP, but nevertheless, BTP can be used with a pleasure programming language.
Links to all parts of the project:
- Http://github.com/mambaru/btp-daemon/ - Demon - Http://github.com/mambaru/btp-webui/ - Web-based interface - Http://github.com/mambaru/btp-api/ - api
[Reference to all the servers memkesha] (http://i.imgur.com/kiY0H.png)
See more pictures: http://imgur.com/a/idFF6
- [Kyoto Cabinet] [1] - [Intel Threading Building Blocks] [2] - GCC 4.5 + - Boost - CMake
In Gentoo, you need to install dev-db/kyotocabinet, dev-cpp/tbb, dev-libs/boost, dev-util/cmake In Debian: libtbb2, libtbb-dev, libboost-dev, libboost-thread-dev libboost-program-options-dev, cmake from source and have kyotocabinet http://fallabs.com/kyotocabinet/pkg/ http://threadingbuildingblocks.org/file.php?fid=77
git clone git :/ / github.com / mambaru / btp-daemon.git cd btp-daemon make all . / Release / daemon - help
git clone git :/ / github.com / mambaru / btp-webui.git
Next, you need to put this directory so that it was visible web server. If the daemon and web interface are on the same server, nothing more needs to be done. If the web interface and daemon are on different servers, you need to edit and enter the address config.js demon.
BTP consists of two parts - a daemon and a web interface. They are independent from each other, and if necessary you can use a few demons (dev / production) with a single web interface. As a communications protocol used by JSON-RPC. Counters can be passed as over TCP, and in the UDP, requests for data is best done over TCP.
For example:
user @ host $ telnet 127.0.0.1 22400 ^] telnet> Connection closed.
This request informs BTP, which was launched by the script test.php, which is addressed to the service mysql, and specifically to the server db7, and connect the operation was due to a time and took about 3ms, and select - 3 times, 3.5, 2 and 6.8 ms.
After that, the interface can be seen as follows:
- / # Service / - appeared in the list of mysql - / # Script / - appeared in the list of test.php - / # Service / mysql / - in the "servers, which serve service mysql" server appeared db7 and less visible operations select and connect. It shows all the stress taken into account, which goes to the server - / # Script / test.php / - in the "Services that are used in test.php" appeared mysql. If you click on mysql, get into .. - / # Script / test.php / mysql / - what operations the service mysql and how much is used from a script test.php
Now make a second request:
user @ host $ telnet 127.0.0.1 22400
^] telnet> Connection closed.
- / # Service / - no change - / # Script / - in the list appeared test2.php - / # Service / mysql / - there was a second set of dots to the right along the time axis. otherwise no change - / # Script / test.php / - no change - / # Script / test.php / mysql / - with no change (the second point does not appear, because they are shown on this page searches only the test.php) - / # Script/test2.php/mysql / - a set of points corresponding to the new set of / # service / mysql /
After some time (up to 30 minutes) on the page / # service / mysql / a list of scripts that are most actively using service mysql.
git clone git :/ / github.com / mambaru / btp-webui.git
The pattern of something like this. For example, you have the code:
function get_something_from_database ($ parameters) {
$ Data = Database :: getConnection () -> query ('select * from table where some_id = 12');
$ Result = array ();
foreach ($ data as $ row) {
$ Result [] = ....
}
return $ result;
}/ / Option 1 - the explicit removal of the counter
function get_something_from_database ($ parameters) {
$ Counter = new Stat_Btp_Counter (Stat_Btp_Request :: getLast (), array (
'Srv' => 'db7',
'Service' => 'mysql',
'Op' => 'select',
));
$ Data = Database :: getConnection () -> query ('select * from table where some_id = 12');
$ Counter-> stop ();
$ Result = array ();
foreach ($ data as $ row) {
$ Result [] = ....;
}
return $ result;
}
/ / Option 2 - using the destructor
function get_something_from_database ($ parameters) {
$ Counter = new Stat_Btp_Counter (Stat_Btp_Request :: getLast (), array (
'Srv' => 'db7',
'Service' => 'mysql',
'Op' => 'select',
));
$ Data = Database :: getConnection () -> query ('select * from table where some_id = 12');
$ Result = array ();
foreach ($ data as $ row) {
$ Result [] = ....;
}
return $ result;
}In the embodiment 1 will be measured only while the Database :: getConnection () -> query ..., in option 2 the total time is measured from the second line of the function to exit. It should be noted that the current implementation of counters collected and sent in small groups. For web-scripts and some cron-script this behavior is desirable, but if you like your cron-script most of the time in sleep, it is better to send a counter force before going to sleep: Stat_Btp_Request :: getLast () -> close ()
It is necessary, particularly if you want to make their implementation schedules. To test calls can be used cmd.php, such as
Recording calls in the notation to be used as follows:
get_list ({service: "?"})
Recording calls on the command line:
php cmd.php get_list host = ip_of_btp_daemon service =?
Record the call in the PHP-code:
$ Conn = new JsonRpc_Connection (array ('host' => '127 .0.0.1 ',' port '=> 22400));
print_r ($ conn-> request ('get_list', array ('service' => '?')) -> get ());
Delete ({service:" value ", op:" value "}) => bool
data deletion operation
Delete ({service:" value ", server:" value "}) => bool
Remove the server data
Delete ({service:" value "}) => bool
deletion of data service
Delete ({script:" value "}) => bool
delete data script
Get_list_advanced ({service: '?', Limit: 10, sortby: 'count'})
service list, sorted by count (possible sort: count - the number of requests, total - which is equal to the total time count * avg, perc80 - value percentile)
Get_list_advanced ({service: 'value', op: 'value', script: '?', Limit: 10, sortby: 'count'})
list of scripts that use the transaction server, sorted by count (possible sort: count - the number of requests, total - the total time which is equal to count * avg, perc80 - value percentile)
Get_list_advanced ({service: 'value', op: 'value', server: '?', Limit: 10, sortby: 'count'})
a list of servers that provide transaction server, sorted by count (possible sort: count - the number of requests, total - the total time which is equal to count * avg, perc80 - value percentile)
Get_list ({server: .., server: .., op: .., script: ..})
one of the four keys to be "?", the rest can be left out / empty strings, or specify a value. However, not all combinations make sense, for instance {op: "?", Service: "val"} - the list of operations on the service val, {op: "?", Server: "val"} - does not make sense (as op binds the service and not the server), so the server will be ignored
Put ({script: 'index.php', items: {service_name: {server_name: {op_name: [ts1, ts2, .. tsn]}}})
adding statistics. transferred to the script name, and counters for services / servers / Operations
In the "Mamba" BTP treats about 2 .. 3 million meters per second. Line from the top:
3185 user 20 0 17.8g 14g 2.1g R 186 61.8 14072:14 btp_release </ code>
Memory comes in the buffer cache, and storage of raw data prior to calculation. At a lower load, memory usage will be less.
Daemon was developed in [the company "Mamba"] [4] and is licensed under GNU GPL ver. Two. Web-based interface was developed in [the company "Mamba"] [4] and is licensed under the MIT.
Developer - [Ilia Shapovalov] [5]. In the daemon uses the following libraries:
- Faslib - GPL2. (C) [Vladimir Migashko, the company "Mamba"] [7]
- Kyoto cabinet - GPL3 + Foss License Exception. (C) [FAL Labs] [6]
- Intel tbb - GPL2. (C) Intel
- Boost: Boost License.
Libraries used in the web-interface, see the subfolder the web interface.
[1]: http://fallabs.com/kyotocabinet/
[2]: http://threadingbuildingblocks.org/ver.php?fid=182
[3]: git :/ / github.com / mambaru / btp-webui.git
[4]: http://corp.mamba.ru/
[5]: http://github.com/shepik/
[6]: http://fallabs.com/
[7]: http://code.google.com/p/faslib/