Skip to content

Instantly share code, notes, and snippets.

@ziadoz
Created January 11, 2024 17:33
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
Star You must be signed in to star a gist
Save ziadoz/2801223aa34db96e950634a6ef46c66d to your computer and use it in GitHub Desktop.
Scrape Twitter using PHP and Symfony Panther
{
"require-dev": {
"symfony/panther": "^2.1",
"dbrekelmans/bdi": "^1.1"
}
}
<?php
require 'vendor/autoload.php';
use Symfony\Component\Panther\Client;
ini_set('zend.exception_string_param_max_len', 100_000);
foreach (glob('screenshots/*.png') as $screenshot) {
unlink($screenshot);
}
function env($env) {
return ($val = getenv($env)) !== false ? $val : throw new Exception('Missing ' . $env . ' environment variable');
}
$credentials = [
'username' => env('TWITTER_USERNAME'),
'password' => env('TWITTER_PASSWORD'),
];
$client = Client::createChromeClient(
arguments: ['--user-agent=Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36'],
baseUri: 'https://twitter.com',
);
// Login
$client->request('GET', '/i/flow/login');
$crawler = $client->getCrawler();
$client->waitForVisibility('[autocomplete=username]');
$crawler->filter('[autocomplete=username]')->sendKeys($credentials['username']);
$crawler->filterXPath('//span[contains(text(), "Next")]/parent::span/parent::div/parent::div')->click();
$client->waitForVisibility('[autocomplete=current-password]');
$crawler->filter('[autocomplete=current-password]')->sendKeys($credentials['password']);
$crawler->filter('[data-testid="LoginForm_Login_Button"]')->click();
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment