15
Янв
2017

Как получить ответ через POST с помощью CURL?

Здравствуйте! Нужно получить данные праздников, которые приходят с сервера через Ajax. Сайт https://www.investing.com/holiday-calendar/ Задача: спарсить календарь. Я спарсил только ту часть, которая видна при загрузке, а вот получить на весь год не получается.

Нужно нажать на календарь, выбрать дату. После этого, нужно проскроллить страницу вниз, чтобы отправился post запрос на сервер.

Я сохраняю куки, задаю те же параметры (см скриншоты), но ответ не могу получить.

ini_set('error_reporting', E_ALL);
ini_set('display_errors', 1);
ini_set('display_startup_errors', 1);
define('DIR', __DIR__);

$user_agent = isset($_SERVER['HTTP_USER_AGENT']) ? $_SERVER['HTTP_USER_AGENT'] : 'Traider';
$cookie = dirname(__FILE__).DIRECTORY_SEPARATOR.'init_cookie.txt';
$f = fopen('init_deb.txt', 'w');
$ch = curl_init();

$getUrl = 'https://www.investing.com/holiday-calendar/';
$postUrl = 'https://www.investing.com/holiday-calendar/Service/getCalendarFilteredData';

$dateFrom='2017-01-14';
$dateTo='2017-12-31';
$limit_from = 0;
$params = [
    'dateFrom' => $dateFrom,
    'dateTo' => $dateTo,
    'county' => '',
    'limit_from' => $limit_from
];

curl_setopt($ch, CURLOPT_URL, $postUrl);
curl_setopt($ch, CURLOPT_POSTFIELDS, $params);
curl_setopt($ch, CURLOPT_HEADER, 1);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);
curl_setopt($ch, CURLOPT_COOKIEFILE, $cookie);
curl_setopt($ch, CURLOPT_COOKIEJAR, $cookie);
curl_setopt($ch, CURLOPT_USERAGENT, $user_agent);
curl_setopt($ch, CURLOPT_VERBOSE, true);
curl_setopt($ch, CURLOPT_STDERR, $f);

$response = curl_exec($ch);
fclose($f);
curl_close($ch);

echo $response;

image image

После отправки post запроса получаю главную страницу. image

Сначала делаю get чтобы получить куки, заголовки, затем post, меняя только $getUrl на $postUrl. Куки сохраняются в файл, а вот заголовки - не знаю. Пробовал сохранить заголовки в файл, а потом добавлять через CURLOPT_HTTPHEADER , но безуспешно. Результат пост запроса

* Trying 185.94.84.100...
* Connected to www.investing.com (185.94.84.100) port 443 (#0)
* ALPN, offering http/1.1
* Cipher selection: ALL:!EXPORT:!EXPORT40:!EXPORT56:!aNULL:!LOW:!RC4:@STRENGTH
* successfully set certificate verify locations:
* CAfile: c:/openserver/modules/php/PHP-7-x64/cacert.pem
CApath: none
* NPN, negotiated HTTP1.1
* SSL connection using TLSv1.2 / ECDHE-RSA-AES256-GCM-SHA384
* ALPN, server did not agree to a protocol
* Server certificate:
* subject: OU=Domain Control Validated; CN=*.investing.com
* start date: Jan 18 09:48:38 2016 GMT
* expire date: Jan 18 09:48:38 2019 GMT
* subjectAltName: host "www.investing.com" matched cert's "*.investing.com"
* issuer: C=US; ST=Arizona; L=Scottsdale; O=GoDaddy.com, Inc.; OU=certs.godaddy.com/repository/; CN=Go Daddy Secure Certificate Authority - G2
* SSL certificate verify ok.
> POST /holiday-calendar/Service/getCalendarFilteredData HTTP/1.1
Host: www.investing.com
User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36 OPR/42.0.2393.94
Accept: */*
Cookie: PHPSESSID=ec8j7t662mqidip4chg5fl9vq6; StickySession=id.18267366337.742www.investing.com; geoC=RU; gtmFired=OK; nyxDorf=YGcyYDVkYSNkNGFrM34wMzFlPmMxKDc3YmNuag%3D%3D; adBlockerNewUserDomains=1484480915
Content-Length: 453
Expect: 100-continue
Content-Type: multipart/form-data; boundary=------------------------5773b84c8c410fd5

< HTTP/1.1 100 Continue
< HTTP/1.1 301 Moved Permanently
< Server: nginx/1.10.2
< Date: Sun, 15 Jan 2017 11:55:25 GMT
< Content-Type: text/html; charset=utf-8
< Content-Length: 0
< Connection: keep-alive
< X-Powered-By: PHP/5.4.43
< Expires: Thu, 19 Nov 1981 08:52:00 GMT
< Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0
< Pragma: no-cache
* Added cookie ses_id="deleted" for domain www.investing.com, path /, expire 1
< Set-Cookie: ses_id=deleted; expires=Thu, 01-Jan-1970 00:00:01 GMT; path=/; domain=www.investing.com
< Location: https://www.investing.com/
< Vary: User-Agent
< Accept-Ranges: bytes
< X-Varnish: 1244608783
< Age: 0
< Via: 1.1 varnish
< Content-Security-Policy: upgrade-insecure-requests
* HTTP error before end of send, stop sending
< 
* Closing connection 0
* Issue another request to this URL: 'https://www.investing.com/'
* Switch from POST to GET
* Hostname www.investing.com was found in DNS cache
* Trying 185.94.84.100...
* Connected to www.investing.com (185.94.84.100) port 443 (#1)
* ALPN, offering http/1.1
* Cipher selection: ALL:!EXPORT:!EXPORT40:!EXPORT56:!aNULL:!LOW:!RC4:@STRENGTH
* successfully set certificate verify locations:
* CAfile: c:/openserver/modules/php/PHP-7-x64/cacert.pem
CApath: none
* SSL re-using session ID
* NPN, negotiated HTTP1.1
* SSL connection using TLSv1.2 / ECDHE-RSA-AES256-GCM-SHA384
* ALPN, server did not agree to a protocol
* old SSL session ID is stale, removing
* Server certificate:
* subject: OU=Domain Control Validated; CN=*.investing.com
* start date: Jan 18 09:48:38 2016 GMT
* expire date: Jan 18 09:48:38 2019 GMT
* subjectAltName: host "www.investing.com" matched cert's "*.investing.com"
* issuer: C=US; ST=Arizona; L=Scottsdale; O=GoDaddy.com, Inc.; OU=certs.godaddy.com/repository/; CN=Go Daddy Secure Certificate Authority - G2
* SSL certificate verify ok.
> GET / HTTP/1.1
Host: www.investing.com
User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36 OPR/42.0.2393.94
Accept: */*
Cookie: PHPSESSID=ec8j7t662mqidip4chg5fl9vq6; StickySession=id.18267366337.742www.investing.com; geoC=RU; gtmFired=OK; nyxDorf=YGcyYDVkYSNkNGFrM34wMzFlPmMxKDc3YmNuag%3D%3D; adBlockerNewUserDomains=1484480915

< HTTP/1.1 200 OK
< Server: nginx/1.10.2
< Date: Sun, 15 Jan 2017 11:55:26 GMT
< Content-Type: text/html; charset=utf-8
< Content-Length: 305967
< Connection: keep-alive
< X-Powered-By: PHP/5.4.43
< Expires: Thu, 19 Nov 1981 08:52:00 GMT
< Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0
< Pragma: no-cache
* Added cookie ses_id="deleted" for domain www.investing.com, path /, expire 1
< Set-Cookie: ses_id=deleted; expires=Thu, 01-Jan-1970 00:00:01 GMT; path=/; domain=www.investing.com
* Replaced cookie nyxDorf="OT4xY2Y3NnQ0ZDw2NXg1Nj9rYj8wKTU1PD01MQ%3D%3D" for domain www.investing.com, path /, expire 1484484926
< Set-Cookie: nyxDorf=OT4xY2Y3NnQ0ZDw2NXg1Nj9rYj8wKTU1PD01MQ%3D%3D; expires=Sun, 15-Jan-2017 12:55:26 GMT; path=/
< Vary: Accept-Encoding,User-Agent
< Accept-Ranges: bytes
< X-Varnish: 1244608919
< Age: 0
< Via: 1.1 varnish
< Content-Security-Policy: upgrade-insecure-requests
< 
* Connection #1 to host www.investing.com left intact

Помогите разобраться как отправить post и получить правильный ответ.

Источник: https://ru.stackoverflow.com/questions/614927/%D0%9A%D0%B0%D0%BA-%D0%BF%D0%BE%D0%BB%D1%83%D1%87%D0%B8%D1%82%D1%8C-%D0%BE%D1%82%D0%B2%D0%B5%D1%82-%D1%87%D0%B5%D1%80%D0%B5%D0%B7-post-%D1%81-%D0%BF%D0%BE%D0%BC%D0%BE%D1%89%D1%8C%D1%8E-curl

Тебе может это понравится...

Добавить комментарий