monbang/Configuring Nginx + LAMP server

## Configuring Nginx + LAMP server
Configuring Nginx + LAMP server at home. Part 1: Setting up the frontend - backend

Hello. I recently asked a question about creating a series of articles. Here is the first article.

In this series of articles, you will learn how to correctly configure the LAMP server, but "hosting is only more powerful."
We will use the following stack: nginx - apache-mpm-itk - mod_php - mysql - linux / debian.

I will cover the following topics:
Frontend setup - backend
Calculation of server capabilities, setting up mysql and backend
Story about experience based on intel s3420gp

I can confidently say that setting up a LAMP server is not limited to 6-10 commands for installing and uncommenting certain lines in the configuration files.
Example: by default, nginx does not allow uploading a request body larger than 1M to the server. If you do not configure this parameter, an error 414 (Request-URI Too Large) will occur when you try to add a small series of photos.
Apache has the exact opposite: its request body is unlimited by default. This makes it possible to do dirty tricks.

In this article, we will get to know the entire setup thoroughly. In the article you can find the configuration files prepared by me. Being a pedant, my configuration files are always grouped by type, for example: “performance”, “content generation”, “error pages”, “compression”, “other settings”, “general settings”. It seems to me that the readability of these files becomes much better if they are grouped.

We will learn about simple attacks and how to defend against them. I must say right away that with the basic configuration of the frontend in the face of nginx - backend apache still remains vulnerable.

I am pretty sure that I can’t fit everything in one article. Welcome to cat.


- Preface:
It so happened that recently our server began to crash. For some reason, he fell from evening to night, and during the day he worked quite for himself. I can’t say that there was a huge load on it. During the falls, I discovered incomprehensibly huge jumps in the allocation of RAM on apache, more than 700 megs per process, although PHP had a maximum of 256M. The server went into a swap, and then crashed. The server initially had 8G RAM, after which they installed 16G.

Before these crashes, the server worked for a whole year and did not know any problems. He had a terrible configuration made on his knees, because we were already driven from the hosting. Here is its configuration, all from the Debian repositories:
Apache2.2.16_mpm_itk + php5.3.0 hung on the Internet for frontend and backend at the same time, without protection against possible attacks in general. I managed to carry out all the attacks mentioned on the Habré =).
Mysql5.1 was configured in a basic configuration with suboptimal use of RAM and all that.

From that moment, I had to study everything very well. By the way, after everything was fine-tuned, the amount of unnecessary verbiage in the virtual host configurations decreased dramatically!

Let's start the setup

- Repositories are different:
The first problem that arose was the software update. As you know, debian repositories have quite outdated software. They say they are well tested, but they are really old! By the way, I myself was surprised when I found this selection of software.
Now I take the LAMP repositories from here www.dotdeb.org
Here is the setup guide for www.dotdeb.org/instructions
For those who are very dry for the first time (but after all, it was such once and there was nobody to help but Google):
- Install debian immediately put a daw to put the SSH server (nothing else !!!), then we find out the IP server for the VGA monitor no longer sit down, you can turn the server into a data center.
- Connect via SSH, I advise putty under win.
- nano /etc/apt/sources.list is inserted there according to the instructions with dotdeb. (note: inserting text into the console from the windows buffer is done with the right mouse button)
- we perform other things according to the instructions
- we execute a large block of commands that I wrote below
- we work through MC like white people
- then everything depends on you! =)

Note, here is my hardware configuration: Intel s3420gp, xeon x3450, 16GB ECC RDIMM, 3 * 1Tb 3.5 "SATA" WD BE "(2x-raid1 (25G /, 20G swap, 100G / var, ~ 800G / home) + 1single (1Tb / mnt / unsafe)). For beginners: I terribly stepped in my time that I installed Black Edition discs - I had to install Raid Edition.

So. Now we just divided the disks, installed the operating system from the debian minidisk (190Mb) and made the repository settings. Now let's continue. During operations, you will have questions from dpkg, you need to answer them.
apt-get update
apt-get upgrade
apt-get install nginx apache2 apache2-mpm-itk php5 php5-apc php-pear php5-dev php5-gd mysql-server mysql-client php5-mysql postfix mc -y
apt-get install libapache2-mod-rpaf -y
echo all done!

nginx - frontend, apache2-mpm-itk - backend, mod_php5.3 - language, mysql5.5 - database, postfix - mailing from PHP.

- We make up the architecture of the system:
It is always known that without a visual picture it is more difficult to configure the system than if it were. At first, I myself couldn’t imagine it and drew it only after reading more than half of the main documentation on two servers and a little about Apache with mysql. Now I drew it for readers right away, so that it would be easier and more convenient for them to study the configuration. I think that according to this picture it will be quite simple to analyze the narrow parts of the system in the future, which will positively affect the support of the system and, possibly, its scaling. (Picture clickable) I will explain all parts of the picture in the next article. In this article we will reveal the frontend - backend settings. Apache protection.


Frontend: nginx setup

We will configure nginx in proxy mode. There are several reasons for this:
Apache server protection against attacks
Compress and render static content on a lightweight server
Saving server brains
Ease of implementation
Apache mod_php works ( not much || worse ) than PHP FastCGI, while mod_php configuration is more clear and standard

- Configs to the studio!
Here are my configuration files that I use for the proxy server
yadi.sk/d/w0SNSFIM0nyk1
What you got from the repositories should be mercilessly replaced with these files, as well as add the .docx, .pptx, .xlsx extensions to the mime opposite the corresponding mime types. Let's take a look at the device configurations:
/nginx.conf - главный конфигурационный файл - собственно с него и начинается загрузка. В нем я оставил общие настройки сервера и импорт директорий.
/proxy_params - конфигурационный файл, для настройки nginx сервера в режима proxy. Там собраны все настройки касательно проксирования сгруппированные по группам: Базовые настройки, Защита от killapache.pl, Размер буферов, Кеширование, Другое.
/conf.d - директория в которой я сгруппировал конфигурации по каждому модулю. В частности я сгруппировал error-docs - страницы ошибок, ngx_http_core_module - базовые настройки сервера, ngx_http_gzip_module - настройки сжатия.

I intentionally do not explain the meaning of the settings, as there is Russian documentation . However, I want to point out some of them and why I chose such parameters. I would like to think to readers what settings are needed for their purposes, especially since I no longer need to search for them; I have grouped them all.
I increased client_max_body_size to 64M so that it would be possible to upload various multimedia to the server - corny photos.
client_body_buffer_size - I increased the standard buffer to 32k because by default it will obviously not be enough. Often you have to process "big data (10-20k)" at the input. In general, this parameter should be determined from what code will be executed on the server. If you are for example Twitter, then you do not need to spend more than 1k (however, you still need to put 8k to drop it on the 64x memory page of the system). If you’re a habrahabr, I would put 8k (less than the default value), because they write comments there often going beyond 1k, but on average less than 8k (for a snap, I could be wrong). What happens if this parameter overflows, read the documentation.
large_client_header_buffers - I reduced it by aligning with what the rest of the system can accept - apache + php.
worker_processes - set worker processes by the number of cores / 2.
worker_priority - set above all so that the rest of the backend does not slow down the return of generated content.
server_tokens off; - do not shine with what you have =), you can get into trouble.
proxy_read_timeout - I increased it to 300 + 20 seconds. This is slightly more than the timeout apache, which occurs after 300 + 10 seconds. This is done so that the timeout would come "from the depths" of the backend, and not break off for unknown reasons with the frontend. By default, this parameter is 60 seconds, which is sometimes not enough for heavy computing. I note that php in this configuration can run up to 300 seconds, before the start of the timeout thread.
Please pay attention to the proxy_set_header directives in the proxy_params file: they are used to set headers for the proxied server. In particular, the proxied server does not see what IP address it addressed, because it believes that local 127: 0: 0: 1 is accessing it
I am proxying to the local port 127.0.0.1:88, I honestly tried to find sockets or crutches based on them, but it didn’t work out = (


Backend: apache setup

Now let's set up apache. Here is another pack of grouped configuration files: yadi.sk/d/ZqsisoDl0nzrl
In the Apache files, I signed each parameter by copying it from the documentation, which would be more convenient to configure. Again, I am sending you to the documentation for customization for your needs and telling you about key things:
ports.conf file: listen to ports 88 and 443.
additionally install libapache2-mod-rpaf (already done above). It serves to decrypt the headers sent to us by the proxy server. Yes, the very headers that we set with the proxy_set_header directive.
DeflateCompressionLevel 1 (file /mods-enabled/deflate.conf) in defline I set the compression ratio = 1, I decided not to press much. In any case, if you have extra hands on the server, then why not 9 ?.
I activated the mod_headers module and in the conf.d / security file I cut out some headers for security, for hosts hanging on port 443. Details in configs. I don’t know why I left this port not proxied, but it’s a fact. Just something didn’t reach my hands, or I was afraid, and then my hands didn’t reach. In any case, ordinary mortals are forbidden to contact him. By the way, with Apache version> = 2.2.21, this problem was solved by special tuning at the kernel level.
ServerTokens Prod, ServerSignature Off - again, do not shine with tasty places.
In the secutiry file in the directory section, I described as many different settings as possible, so that there would be less verbiage in the files of the virtual hosts.
TimeOut 310 - already wrote why 310 seconds.
RLimitMEM is an interesting thing, I advise you to read. Allows you to limit the memory that Apache consumes as a whole. Other R * parameters are also interesting.
DefaultType application / octet-stream - if we don’t know what we are giving, then let them be binaries.
AddDefaultCharset Off - it is better not to turn it on if you are not sure that you have everything in one encoding


With what you need to pay attention, I ended up. Now let's move on to the mpm-itk module selected for hosting tasks.
The fact is that this module is convenient in that every time a new server is accessed, a new process is created and it switches to a specific user, say www-ru-example. That is, this user is "locked" in his directory and no scripts can get anywhere, given that you have correctly configured the operating system. I note that many default configuration files in debian are open for reading to everyone !!! ..
The story of this MPM is interesting. The fact is that it was made on the basis of mpm prefork, which means that all settings for prefork apply to itk as well. Accordingly, in my configuration file you can track this.
Please pay attention to MaxClients 150, these are the same maximum 150 users that may be when accessing the backend. Also, please pay attention to MaxRequestsPerChild. By default, it is zero. It is advised to install it at least limited - this reduces the possibility of memory leak.

Another important thing in mpm-itk is nice value. I set this value to -2. I did just that, because as soon as the database returned the result, PHP should instantly form and submit it to the end. Please note the nice-value hierarchy.
nginx = -5, apache = -2, mysql = 0. This is done in order to generate content as quickly as possible and give it to the user. The operating system should supplant non-priority processes for later.

I would like to say a few words about the basic Apache protection.
There are several types of attacks that any system administrator should know: killapache.pl, slow post, slow lori. All of these attacks are very simple to open apache. Mod_headers or nginx saves from killapache.pl, where you can close the problem by banning certain headers. They say that killapache.pl is a problem of the protocol itself. slow post, slow lori are identical attacks, one is done when sending a large POST with a very slow channel, the other is done by sending the generated content to the client in a very slow channel. These attacks are not scary for the strong, muscular and sinewy nginx server we used to hide behind. For Apache, it is mortally similar, for example, the PHP sandbox is cleaned after the server has given all the data, now imagine how much memory can be consumed.

In the end, I want to say that in the configuration files I sent just the files. Do not forget to create simulinki in particular for mods-enabled directories from mods-available, sites- *, etc.

Thanks for reading, I hope I liked my selection of configs. I will try to cover many other things in other topics. For example, setting up the backend (php - mysql) and calculating server capabilities. If the first and second article is interesting, then I can roll out 2 more articles: “user accounting system”, “experience regarding the choice of entry-level hardware”, “miscellaneous about working on the server”. In the final, I can develop a set of utilities for quickly configuring the server for the articles I have specified.
	Configuring Nginx + LAMP server at home. Part 1: Setting up the frontend - backend

	Hello. I recently asked a question about creating a series of articles. Here is the first article.

	In this series of articles, you will learn how to correctly configure the LAMP server, but "hosting is only more powerful."
	We will use the following stack: nginx - apache-mpm-itk - mod_php - mysql - linux / debian.

	I will cover the following topics:
	Frontend setup - backend
	Calculation of server capabilities, setting up mysql and backend
	Story about experience based on intel s3420gp

	I can confidently say that setting up a LAMP server is not limited to 6-10 commands for installing and uncommenting certain lines in the configuration files.
	Example: by default, nginx does not allow uploading a request body larger than 1M to the server. If you do not configure this parameter, an error 414 (Request-URI Too Large) will occur when you try to add a small series of photos.
	Apache has the exact opposite: its request body is unlimited by default. This makes it possible to do dirty tricks.

	In this article, we will get to know the entire setup thoroughly. In the article you can find the configuration files prepared by me. Being a pedant, my configuration files are always grouped by type, for example: “performance”, “content generation”, “error pages”, “compression”, “other settings”, “general settings”. It seems to me that the readability of these files becomes much better if they are grouped.

	We will learn about simple attacks and how to defend against them. I must say right away that with the basic configuration of the frontend in the face of nginx - backend apache still remains vulnerable.

	I am pretty sure that I can’t fit everything in one article. Welcome to cat.


	- Preface:
	It so happened that recently our server began to crash. For some reason, he fell from evening to night, and during the day he worked quite for himself. I can’t say that there was a huge load on it. During the falls, I discovered incomprehensibly huge jumps in the allocation of RAM on apache, more than 700 megs per process, although PHP had a maximum of 256M. The server went into a swap, and then crashed. The server initially had 8G RAM, after which they installed 16G.

	Before these crashes, the server worked for a whole year and did not know any problems. He had a terrible configuration made on his knees, because we were already driven from the hosting. Here is its configuration, all from the Debian repositories:
	Apache2.2.16_mpm_itk + php5.3.0 hung on the Internet for frontend and backend at the same time, without protection against possible attacks in general. I managed to carry out all the attacks mentioned on the Habré =).
	Mysql5.1 was configured in a basic configuration with suboptimal use of RAM and all that.

	From that moment, I had to study everything very well. By the way, after everything was fine-tuned, the amount of unnecessary verbiage in the virtual host configurations decreased dramatically!

	Let's start the setup

	- Repositories are different:
	The first problem that arose was the software update. As you know, debian repositories have quite outdated software. They say they are well tested, but they are really old! By the way, I myself was surprised when I found this selection of software.
	Now I take the LAMP repositories from here www.dotdeb.org
	Here is the setup guide for www.dotdeb.org/instructions
	For those who are very dry for the first time (but after all, it was such once and there was nobody to help but Google):
	- Install debian immediately put a daw to put the SSH server (nothing else !!!), then we find out the IP server for the VGA monitor no longer sit down, you can turn the server into a data center.
	- Connect via SSH, I advise putty under win.
	- nano /etc/apt/sources.list is inserted there according to the instructions with dotdeb. (note: inserting text into the console from the windows buffer is done with the right mouse button)
	- we perform other things according to the instructions
	- we execute a large block of commands that I wrote below
	- we work through MC like white people
	- then everything depends on you! =)

	Note, here is my hardware configuration: Intel s3420gp, xeon x3450, 16GB ECC RDIMM, 3 * 1Tb 3.5 "SATA" WD BE "(2x-raid1 (25G /, 20G swap, 100G / var, ~ 800G / home) + 1single (1Tb / mnt / unsafe)). For beginners: I terribly stepped in my time that I installed Black Edition discs - I had to install Raid Edition.

	So. Now we just divided the disks, installed the operating system from the debian minidisk (190Mb) and made the repository settings. Now let's continue. During operations, you will have questions from dpkg, you need to answer them.
	apt-get update
	apt-get upgrade
	apt-get install nginx apache2 apache2-mpm-itk php5 php5-apc php-pear php5-dev php5-gd mysql-server mysql-client php5-mysql postfix mc -y
	apt-get install libapache2-mod-rpaf -y
	echo all done!

	nginx - frontend, apache2-mpm-itk - backend, mod_php5.3 - language, mysql5.5 - database, postfix - mailing from PHP.

	- We make up the architecture of the system:
	It is always known that without a visual picture it is more difficult to configure the system than if it were. At first, I myself couldn’t imagine it and drew it only after reading more than half of the main documentation on two servers and a little about Apache with mysql. Now I drew it for readers right away, so that it would be easier and more convenient for them to study the configuration. I think that according to this picture it will be quite simple to analyze the narrow parts of the system in the future, which will positively affect the support of the system and, possibly, its scaling. (Picture clickable) I will explain all parts of the picture in the next article. In this article we will reveal the frontend - backend settings. Apache protection.



	Frontend: nginx setup

	We will configure nginx in proxy mode. There are several reasons for this:
	Apache server protection against attacks
	Compress and render static content on a lightweight server
	Saving server brains
	Ease of implementation
	Apache mod_php works ( not much \|\| worse ) than PHP FastCGI, while mod_php configuration is more clear and standard

	- Configs to the studio!
	Here are my configuration files that I use for the proxy server
	yadi.sk/d/w0SNSFIM0nyk1
	What you got from the repositories should be mercilessly replaced with these files, as well as add the .docx, .pptx, .xlsx extensions to the mime opposite the corresponding mime types. Let's take a look at the device configurations:
	/nginx.conf - главный конфигурационный файл - собственно с него и начинается загрузка. В нем я оставил общие настройки сервера и импорт директорий.
	/proxy_params - конфигурационный файл, для настройки nginx сервера в режима proxy. Там собраны все настройки касательно проксирования сгруппированные по группам: Базовые настройки, Защита от killapache.pl, Размер буферов, Кеширование, Другое.
	/conf.d - директория в которой я сгруппировал конфигурации по каждому модулю. В частности я сгруппировал error-docs - страницы ошибок, ngx_http_core_module - базовые настройки сервера, ngx_http_gzip_module - настройки сжатия.

	I intentionally do not explain the meaning of the settings, as there is Russian documentation . However, I want to point out some of them and why I chose such parameters. I would like to think to readers what settings are needed for their purposes, especially since I no longer need to search for them; I have grouped them all.
	I increased client_max_body_size to 64M so that it would be possible to upload various multimedia to the server - corny photos.
	client_body_buffer_size - I increased the standard buffer to 32k because by default it will obviously not be enough. Often you have to process "big data (10-20k)" at the input. In general, this parameter should be determined from what code will be executed on the server. If you are for example Twitter, then you do not need to spend more than 1k (however, you still need to put 8k to drop it on the 64x memory page of the system). If you’re a habrahabr, I would put 8k (less than the default value), because they write comments there often going beyond 1k, but on average less than 8k (for a snap, I could be wrong). What happens if this parameter overflows, read the documentation.
	large_client_header_buffers - I reduced it by aligning with what the rest of the system can accept - apache + php.
	worker_processes - set worker processes by the number of cores / 2.
	worker_priority - set above all so that the rest of the backend does not slow down the return of generated content.
	server_tokens off; - do not shine with what you have =), you can get into trouble.
	proxy_read_timeout - I increased it to 300 + 20 seconds. This is slightly more than the timeout apache, which occurs after 300 + 10 seconds. This is done so that the timeout would come "from the depths" of the backend, and not break off for unknown reasons with the frontend. By default, this parameter is 60 seconds, which is sometimes not enough for heavy computing. I note that php in this configuration can run up to 300 seconds, before the start of the timeout thread.
	Please pay attention to the proxy_set_header directives in the proxy_params file: they are used to set headers for the proxied server. In particular, the proxied server does not see what IP address it addressed, because it believes that local 127: 0: 0: 1 is accessing it
	I am proxying to the local port 127.0.0.1:88, I honestly tried to find sockets or crutches based on them, but it didn’t work out = (


	Backend: apache setup

	Now let's set up apache. Here is another pack of grouped configuration files: yadi.sk/d/ZqsisoDl0nzrl
	In the Apache files, I signed each parameter by copying it from the documentation, which would be more convenient to configure. Again, I am sending you to the documentation for customization for your needs and telling you about key things:
	ports.conf file: listen to ports 88 and 443.
	additionally install libapache2-mod-rpaf (already done above). It serves to decrypt the headers sent to us by the proxy server. Yes, the very headers that we set with the proxy_set_header directive.
	DeflateCompressionLevel 1 (file /mods-enabled/deflate.conf) in defline I set the compression ratio = 1, I decided not to press much. In any case, if you have extra hands on the server, then why not 9 ?.
	I activated the mod_headers module and in the conf.d / security file I cut out some headers for security, for hosts hanging on port 443. Details in configs. I don’t know why I left this port not proxied, but it’s a fact. Just something didn’t reach my hands, or I was afraid, and then my hands didn’t reach. In any case, ordinary mortals are forbidden to contact him. By the way, with Apache version> = 2.2.21, this problem was solved by special tuning at the kernel level.
	ServerTokens Prod, ServerSignature Off - again, do not shine with tasty places.
	In the secutiry file in the directory section, I described as many different settings as possible, so that there would be less verbiage in the files of the virtual hosts.
	TimeOut 310 - already wrote why 310 seconds.
	RLimitMEM is an interesting thing, I advise you to read. Allows you to limit the memory that Apache consumes as a whole. Other R * parameters are also interesting.
	DefaultType application / octet-stream - if we don’t know what we are giving, then let them be binaries.
	AddDefaultCharset Off - it is better not to turn it on if you are not sure that you have everything in one encoding


	With what you need to pay attention, I ended up. Now let's move on to the mpm-itk module selected for hosting tasks.
	The fact is that this module is convenient in that every time a new server is accessed, a new process is created and it switches to a specific user, say www-ru-example. That is, this user is "locked" in his directory and no scripts can get anywhere, given that you have correctly configured the operating system. I note that many default configuration files in debian are open for reading to everyone !!! ..
	The story of this MPM is interesting. The fact is that it was made on the basis of mpm prefork, which means that all settings for prefork apply to itk as well. Accordingly, in my configuration file you can track this.
	Please pay attention to MaxClients 150, these are the same maximum 150 users that may be when accessing the backend. Also, please pay attention to MaxRequestsPerChild. By default, it is zero. It is advised to install it at least limited - this reduces the possibility of memory leak.

	Another important thing in mpm-itk is nice value. I set this value to -2. I did just that, because as soon as the database returned the result, PHP should instantly form and submit it to the end. Please note the nice-value hierarchy.
	nginx = -5, apache = -2, mysql = 0. This is done in order to generate content as quickly as possible and give it to the user. The operating system should supplant non-priority processes for later.

	I would like to say a few words about the basic Apache protection.
	There are several types of attacks that any system administrator should know: killapache.pl, slow post, slow lori. All of these attacks are very simple to open apache. Mod_headers or nginx saves from killapache.pl, where you can close the problem by banning certain headers. They say that killapache.pl is a problem of the protocol itself. slow post, slow lori are identical attacks, one is done when sending a large POST with a very slow channel, the other is done by sending the generated content to the client in a very slow channel. These attacks are not scary for the strong, muscular and sinewy nginx server we used to hide behind. For Apache, it is mortally similar, for example, the PHP sandbox is cleaned after the server has given all the data, now imagine how much memory can be consumed.

	In the end, I want to say that in the configuration files I sent just the files. Do not forget to create simulinki in particular for mods-enabled directories from mods-available, sites- *, etc.

	Thanks for reading, I hope I liked my selection of configs. I will try to cover many other things in other topics. For example, setting up the backend (php - mysql) and calculating server capabilities. If the first and second article is interesting, then I can roll out 2 more articles: “user accounting system”, “experience regarding the choice of entry-level hardware”, “miscellaneous about working on the server”. In the final, I can develop a set of utilities for quickly configuring the server for the articles I have specified.