Thread LWP::UserAgent - URL-Splitting löst Boterkennung aus? (7 answers)
Opened by whatever_u_want at 2020-09-16 14:04

GwenDragon
 2020-09-16 17:54
#192508 #192508
User since
2005-01-17
14754 Artikel
Admin1
[Homepage]
user image
Lass mal ausgeben was gesendet+empfangen wird.

Ein anschauliches Beispiel:
Code (perl): (dl )
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
use strict;
use warnings;
use 5.010;
use LWP::UserAgent;

my $request_uri = '';
my $ua          = LWP::UserAgent->new;
$ua->protocols_allowed( [ 'http', 'https' ] );
$ua->ssl_opts( verify_hostnames => 1 );

$ua->add_handler( "request_send",  sub { shift->dump; return } );
$ua->add_handler( "response_done", sub { shift->dump; return } );

$ua->agent("Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/85.0.4183.106 Safari/537.36");

$ua->proxy(['http', 'https'], 'http://localhost:8001/');


$request_uri = 'https://example.org/';

my $response = $ua->get($request_uri);
print $response->decoded_content;


Ist es so, dass TOR Proxy socks nutzt?
Dann brauchste noch LWP::Protocol::socks.
Und wohl eher:
$ua->proxy([qw(http https)] => 'socks://localhost:9150');
Last edited: 2020-09-16 18:08:42 +0200 (CEST)

View full thread LWP::UserAgent - URL-Splitting löst Boterkennung aus?