PHP cURL 如何添加用户代理值或克服服务器阻塞 cURL 请求?

声明:本页面是StackOverFlow热门问题的中英对照翻译,遵循CC BY-SA 4.0协议,如果您需要使用它,必须同样遵循CC BY-SA许可,注明原文地址和作者信息,同时你必须将它归于原作者(不是我):StackOverFlow 原文地址: http://stackoverflow.com/questions/17801094/
Warning: these are provided under cc-by-sa 4.0 license. You are free to use/share it, But you must attribute it to the original authors (not me): StackOverFlow

提示:将鼠标放在中文语句上可以显示对应的英文。显示中英文
时间:2020-08-25 16:29:06  来源:igfitidea点击:

PHP cURL how to add the User Agent value OR overcome the Servers blocking cURL requests?

phpcurlhttp-headersuser-agent

提问by u775856

I am transferring an Object Array. I have a cURL client (submitter) on own Serverand listening script on other's Server, which one is not under my control. Then i think there, they are blocking the incoming cURL requestsbecause when i test with the normal HTML <form>, it is working. But not via cURL anyway.

我正在传输一个对象数组。我有一个在自己的服务器客户端卷曲(提交)听力脚本其他的服务器上,哪一次不是我的控制之下。然后我想,他们正在阻止传入的 cURL 请求,因为当我使用普通 HTML 进行测试时<form>,它正在工作。但无论如何都不能通过 cURL。

So i think they have done some restriction to cURL.

所以我认为他们对 cURL 做了一些限制。

Then my questions here are:

那么我的问题是:

  1. Can a Server restrict/block the cURL incoming requests?
  2. If so, can i trick/change the HTTP Header (User Agent) in my initiating cURL script?
  3. Or is there any other possible stories?
  1. 服务器可以限制/阻止 cURL 传入请求吗?
  2. 如果是这样,我可以在启动 cURL 脚本中欺骗/更改 HTTP 标头(用户代理)吗?
  3. 或者还有其他可能的故事吗?

Thanks!

谢谢!

采纳答案by TroyCheng

  1. In the server side, we can block some requests by recognize the header fields(including refer, cookie, user-agent and so on) in http request, the ip address, access frequency. And in most case, requests generated by machine usually has something different than human requests,for example, no refer & cookie, or with higher access frequency, we can write some rules to deny these requests.

  2. According to 1, you can try your best to simulate real requests by filling the header fields, using random and slower frequency, using more ip addresses. (sounds like attack)

  3. Generally, using lower frequency and do not make heavy load for their server, follow their access rules, they will seldom block your requests.

  1. 在服务器端,我们可以通过识别http请求中的头域(包括refer、cookie、user-agent等)、ip地址、访问频率等来拦截一些请求。而且在大多数情况下,机器产生的请求通常与人的请求有所不同,例如,没有引用和cookie,或者访问频率较高,我们可以编写一些规则来拒绝这些请求。

  2. 根据1,你可以通过填充header字段,使用随机和较慢的频率,使用更多的ip地址来尽量模拟真实的请求。(听起来像是攻击)

  3. 一般来说,使用较低的频率,不要让他们的服务器负载过重,遵循他们的访问规则,他们很少会阻止您的请求。

回答by Black0CodeR

IF you are still facing the problem then do the following.

如果您仍然面临问题,请执行以下操作。

1.

1.

$config['useragent'] = 'Mozilla/5.0 (Windows NT 6.2; WOW64; rv:17.0) Gecko/20100101 Firefox/17.0';

curl_setopt($curl, CURLOPT_USERAGENT, $config['useragent']);
curl_setopt($curl, CURLOPT_REFERER, 'https://www.domain.com/');

2.

2.

$dir                   = dirname(__FILE__);
$config['cookie_file'] = $dir . '/cookies/' . md5($_SERVER['REMOTE_ADDR']) . '.txt';

curl_setopt($curl, CURLOPT_COOKIEFILE, $config['cookie_file']);
curl_setopt($curl, CURLOPT_COOKIEJAR, $config['cookie_file']);

NOTE:You need a COOKIES folder in directory.

注意:您需要在目录中有一个 COOKIES 文件夹。

3.

3.

curl_setopt($curl, CURLOPT_FOLLOWLOCATION, true);
curl_setopt($curl, CURLOPT_RETURNTRANSFER, true);

If doing these don't solve the problem then Give the Sample Input/Output/Error/etc. So, that more precise solution can be provided.

如果这样做不能解决问题,则提供示例输入/输出/错误等。因此,可以提供更精确的解决方案。

回答by srain

  $agent = 'Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1)';
  $curl=curl_init();
  curl_setopt($curl, CURLOPT_USERAGENT, $agent);

回答by AxelPAL

Server cannot block only cURL requests because they are just HTTP requests. So changing User Agent of your cURL can solve your problem, as server will think you are connecting through browser presented in UA.

服务器不能只阻止 cURL 请求,因为它们只是 HTTP 请求。因此,更改 cURL 的用户代理可以解决您的问题,因为服务器会认为您是通过 UA 中显示的浏览器进行连接。