Laravel 504 网关超时
声明：本页面是StackOverFlow热门问题的中英对照翻译，遵循CC BY-SA 4.0协议，如果您需要使用它，必须同样遵循CC BY-SA许可，注明原文地址和作者信息，同时你必须将它归于原作者（不是我）：StackOverFlow
原文地址: http://stackoverflow.com/questions/48056742/
Warning: these are provided under cc-by-sa 4.0 license. You are free to use/share it, But you must attribute it to the original authors (not me):
StackOverFlow
Laravel 504 gateway timeout
提问by kinjal jethva
I have a script in Laravel platform which process thousands of records , and it's timing out issue. I want to avoid update settings on server like php.ini file.
我在 Laravel 平台上有一个处理数千条记录的脚本，它是超时问题。我想避免像 php.ini 文件那样更新服务器上的设置。
Is there any way to chunk all records in small pieces and use them in further processing until all records will be used.
有没有办法将所有记录分成小块，并在进一步处理中使用它们，直到使用所有记录。
I have thousands of records which are getting Pricefrom another server.
我有数千条记录从另一台服务器获取价格。
So,
所以，
1-10 records works fine.
1-10 条记录工作正常。
1-100 records works fine.
1-100 条记录工作正常。
More than 500 records script stuck and gives error like below.
超过 500 条记录脚本卡住并给出如下错误。
405 nginx server not allowed
405 不允许 nginx 服务器
My Route is like :- example.com/updateAllKeywords
我的路线就像：- example.com/updateAllKeywords
Below code is getting keyword to fetch keywords'price from eBay API
下面的代码是获取关键字以从 eBay API 获取关键字的价格
$keywordCollection = Keywords::all();
Keywords::chunk(100, function ($keywordCollection) use($request,$keywordCollection,$defaultStartDate,$ignoredItems,$response) {
foreach ($keywordCollection->toArray() as $keyword_result) {
$response = $this->apirequest($request, $keyword_result['id'], $keyword_result, $defaultStartDate, $ignoredItems);
}
});
This is api request function:
这是 api 请求函数：
/**
* Call Finding API for Active List items
*
* @return API Response
*/
public function apirequest(Request $request, $id, $keyword_result, $defaultStartDate, $ignoredItems) {
try {
/* Request */
$request = new Types\FindItemsAdvancedRequest();
/* 1) default 100 items being displayed without pagination */
$request->paginationInput = new Types\PaginationInput();
$request->paginationInput->entriesPerPage = Config::get('constants.paginationoptions.entries_per_page');
/* 2) ItemFilter for three conditions NEW ,NEWOTHER,USED */
$itemFilterForConditions = new Types\ItemFilter();
$itemFilterForConditions->name = 'Condition';
$itemFilterForConditions->value = ['1000', '1500', '3000'];
/* 3) ItemFilter for hide duplicate items */
$itemFilterduplicate = new Types\ItemFilter();
$itemFilterduplicate->name = 'HideDuplicateItems';
$itemFilterduplicate->value[] = 'true';
/* 4) pull items before given date / days */
$itemFilterStartDateFrom = new Types\ItemFilter();
$itemFilterStartDateFrom->name = 'StartTimeFrom';
$itemFilterStartDateFrom->value[] = date('Y-m-d\TH:i:s\Z', mktime(0, 0, 0, date('m'), date('d') - $defaultStartDate['defaultStartDate'], date('y')));
/* Exclude Keywords/MPN by set Priced Percentage */
if (!empty($keyword_result['price']) && ($defaultStartDate['exclusionPercentage'] > 0)) {
$percentageAmount = $keyword_result['price'] * ($defaultStartDate['exclusionPercentage'] / 100);
$percentageAmount = floor($percentageAmount).substr($percentageAmount-floor($percentageAmount),1,2+1);
$itemFilterPriced = new Types\ItemFilter();
$itemFilterPriced->name = 'MinPrice';
$itemFilterPriced->value[] = $percentageAmount;
}
$itemFilter = (isset($itemFilterPriced)) ? [$itemFilterForConditions, $itemFilterduplicate, $itemFilterStartDateFrom, $itemFilterPriced] : [$itemFilterForConditions, $itemFilterduplicate, $itemFilterStartDateFrom];
$request->itemFilter = $itemFilter;
/* 5) Request KEYWORD/MPN to filter Items Active Lists */
$searchKeyword = (!empty($keyword_result['exclusions'])) ? $keyword_result['keyword'] . " -" . "(" . trim($keyword_result['exclusions']) . ")" : $keyword_result['keyword'];
$request->keywords = $searchKeyword;
/* 6) Request For sorting Items by Lowest To Highest Itemprice Price + Shipping Price */
$request->sortOrder = 'PricePlusShippingLowest';
/* 7) Request For Keyword Search Including the Item Descriptions,Title and Subtitle */
$request->descriptionSearch = ($defaultStartDate['advancedExclusions'] == 1) ? true : false;
$service = $this->index();
/* Response returned by $request */
$response = $this->apiresponse($request, $service, $id, $keyword_result['keyword'], $ignoredItems);
return $response;
} catch (\Exception $e) {
echo json_encode(array('ack' => 'Failure', 'msg' => $e->getMessage()));
exit;
}
}
public function apiresponse($request, $service, $id, $keyword, $ignoredItems) {
try {
$response = $service->findItemsAdvanced($request);
/* fetch all ignored Items */
$newItems = array();
$newOtherItems = array();
$usedItems = array();
$errors = "";
if ($response->ack == "Success") {
$results = $response->searchResult->item;
ob_start();
foreach ($results as $item) {
if ($item->listingInfo->buyItNowAvailable == '1') {
$itemPrice = $item->listingInfo->buyItNowPrice->value;
} elseif (($item->listingInfo->listingType == 'Auction') && ($item->sellingStatus->currentPrice->value == 0)) {
$itemResponse = $this->getItemDetails($item->itemId);
$itemPrice = isset($itemResponse) ? $itemResponse : 0;
} else {
$itemPrice = $item->sellingStatus->currentPrice->value;
}
/* check for ignored items */
if (!in_array($item->itemId, $ignoredItems)) {
/* check for NEW Condition (conditionId = 1000) */
if ($item->condition->conditionId == '1000') {
$newItems[] = array(
'keywordId' => $id,
'conditionId' => (string) $item->condition->conditionId,
'itemId' => $item->itemId,
'title' => $item->title,
'price' => $itemPrice,
'itemURL' => $item->viewItemURL,
'galleryURL' => isset($item->galleryURL) ? $item->galleryURL : '',
);
}
/* check for NEW OTHER Condition (conditionId = 1500) */ elseif ($item->condition->conditionId == '1500') {
$newOtherItems[] = array(
'keywordId' => $id,
'conditionId' => (string) $item->condition->conditionId,
'itemId' => $item->itemId,
'title' => $item->title,
'price' => $itemPrice,
'itemURL' => $item->viewItemURL,
'galleryURL' => isset($item->galleryURL) ? $item->galleryURL : '',
);
}
/* check for USED Condition (conditionId = 3000) */ elseif ($item->condition->conditionId == '3000') {
$usedItems[] = array(
'keywordId' => $id,
'conditionId' => (string) $item->condition->conditionId,
'itemId' => $item->itemId,
'title' => $item->title,
'price' => $itemPrice,
'itemURL' => $item->viewItemURL,
'galleryURL' => isset($item->galleryURL) ? $item->galleryURL : '',
);
}
}
ob_flush();
}
$newItems = array_slice($newItems, 0, 20);
$newOtherItems = array_slice($newOtherItems, 0, 20);
$usedItems = array_slice($usedItems, 0, 20);
$insertItems = array_merge($newItems, $newOtherItems, $usedItems);
/* clean up items */
Items::where('keywordId', $id)->delete();
/* save pulled items */
$items = Items::insert($insertItems);
} elseif ($response->ack == "Failure") {
foreach ($response->errorMessage->error as $error) {
$errors .= "%s: %s\n\n" . $error->severity === Enums\ErrorSeverity::C_ERROR ? '<b>Error</b> ' : '<b>Warning</b> ' . $error->message;
}
}
/* save API log fot the keyword */
$this->setapilog($request, $response, $keyword);
if (!empty($errors)) {
echo json_encode(array('ack' => 'Failure', 'msg' => $errors));
exit;
} else {
return true;
}
} catch (\Exception $e) {
echo json_encode(array('ack' => 'Failure', 'msg' => $e->getMessage()));
exit;
}
}
Any information on this would be greatly appreciated. Thanks!
任何有关这方面的信息将不胜感激。谢谢！
回答by Tuhin Bepari
@kinjal jethva
@kinjal 杰思瓦
Problem:
问题：
You are using Database Query into foreach loop. You are calling your apirequest() methods into foreach loop.
您正在将数据库查询用于 foreach 循环。您正在将 apirequest() 方法调用到 foreach 循环中。
When yourr $keywords array have 500 records.
当您的 $keywords 数组有 500 条记录时。
You are calling
你在打电话
$keyword_result => 500 times.
$defaultStartDate = 500 times
In total you are calling your 1000 database query in one single http request.
总的来说，您在一个 http 请求中调用了 1000 个数据库查询。
Solution:
解决方案：
Calling $defaultStartDate inside your apirequest() function is totally useless. Instead call it outside and pass value as a parameter. So it will call once.
在 apirequest() 函数中调用 $defaultStartDate 是完全没用的。而是在外部调用它并将值作为参数传递。所以它会调用一次。
$keyword_result can be called better way.
$keyword_result 可以称为更好的方式。
$defaultStartDate = Settings::select('defaultStartDate',
'exclusionPercentage', 'advancedExclusions')->where('currentEnv', 1)->get()-
>first()->toArray();
$keywordValues = array_keys($keywords);
$keywordCollection = Keywords::whereIn('id', $keywordValues)-
>select('keyword', 'exclusions', 'price')->first()->toArray();
foreach ($keywordCollection as $keyword_result) {
$response = $this->apirequest($request, $keyword_result->id,
$keyword_result, $defaultStartDate);
}
/**
* Call Finding API for Active List items
*
* @return API Response
*/
function apirequest(Request $request, $id, $keyword_result, $defaultStartDate)
{
// your code here.
}
This is just a structure of code you can use.
这只是您可以使用的代码结构。
回答by afolabiabass
Create this has a job Laravel QueuesPut your function in the handle method.
Create this has a job Laravel Queues把你的函数放在handle方法中。
$response = $this->apiRequest(...);
Dispatch your job from say your controller
从说你的控制器分派你的工作
foreach ($keywordCollection as $key => $keywordResult) {
ProcessJob::dispatch($keywordResult)->delay($key * 2);
}
This way you are implementing exponential backoff such that as your array increases you increase your wait time to make the next api request.
通过这种方式，您正在实施指数退避，以便随着阵列的增加，您会增加发出下一个 api 请求的等待时间。
