Scrapy post request with parameters example

scrapy_sample.md

Scrapy post request example

We can use scrapy.FormRequest send post request with parameters.

import scrapy

class ExampleSpider(scrapy):
    name = 'ExampleSpider'
    allowed_domains = ['example.com']

    def start_requests(self):
        params = {
            'parameter1': 'value1',
            'parameter2': 'value2',
        }
        yield scrapy.FormRequest('api.example.com', callback=self.parse,
                                 method='POST', formdata=params)

    def parse(self, response):
        pass

Hello, how does the incoming post information from the outside，

how to send header with form request?

how to send header with form request?

FormRequest is a subclass of Request and so you can use the headers argument, like this:
yield scrapy.FormRequest('api.example.com', callback=self.parse, method='POST', formdata=params, headers={'key': 'value'})

how to get response ? want to store in the file.

Hey, guys. Here is the problems I met. I wanna to send a post request, which has both form and params. Like this:

urls = 'https://www.xxx.com/'
params = {"args1":"aaa","args2":"bbb"}
body ={"key":"value"}

As mentioned, we may find the example in the explorer like this:

https://www.xxx.com/?args1=aaa&args2=bbb, with post method and body {"key":"value"}

how should i write in the scrapy?

This code works perfectly.
Thanks @arun-shaji

Hey, guys. Here is the problems I met. I wanna to send a post request, which has both form and params. Like this:

urls = 'https://www.xxx.com/' params = {"args1":"aaa","args2":"bbb"} body ={"key":"value"}

As mentioned, we may find the example in the explorer like this:

https://www.xxx.com/?args1=aaa&args2=bbb, with post method and body {"key":"value"}

how should i write in the scrapy?

Well..this is still actual topic 😕