Skip to content

Instantly share code, notes, and snippets.

@arun-shaji
Last active December 12, 2023 08:12
Show Gist options
  • Save arun-shaji/895df5216ff6905f70228cb1ab228636 to your computer and use it in GitHub Desktop.
Save arun-shaji/895df5216ff6905f70228cb1ab228636 to your computer and use it in GitHub Desktop.
Scrapy post request with parameters example

Scrapy post request example

We can use scrapy.FormRequest send post request with parameters.

import scrapy

class ExampleSpider(scrapy):
    name = 'ExampleSpider'
    allowed_domains = ['example.com']

    def start_requests(self):
        params = {
            'parameter1': 'value1',
            'parameter2': 'value2',
        }
        yield scrapy.FormRequest('api.example.com', callback=self.parse,
                                 method='POST', formdata=params)

    def parse(self, response):
        pass
@jis0324
Copy link

jis0324 commented Oct 4, 2021

This code works perfectly.
Thanks @arun-shaji

@pol4xer
Copy link

pol4xer commented Dec 12, 2023

Hey, guys. Here is the problems I met. I wanna to send a post request, which has both form and params. Like this:

urls = 'https://www.xxx.com/' params = {"args1":"aaa","args2":"bbb"} body ={"key":"value"}

As mentioned, we may find the example in the explorer like this:

https://www.xxx.com/?args1=aaa&args2=bbb, with post method and body {"key":"value"}

how should i write in the scrapy?

Well..this is still actual topic 😕

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment