When we work with some API interface, it often have Rate-Limit, usually certain number of http-requests per unit of time (see for example Twitter API Rate Limit).
This helper can help you efficiently handle these restrictions.
Helper code:
/**
* Helper for work with some API.
*/
class ApiHelper
{
/**
* Wraps specified callback function and retries of execution in the case when callback throws specified exception.
* Used Exponential BackOff algorithm for evaluating of timeouts between retries of callback execution.
*
* Example:
* <code>
* $sendHttpRequest = function ($pageNum) use ($httpClient){
* $httpClient->sendGet('http://example.com?pageNum=' . $pageNum);
* };
*
* $wrappedFn = ApiHelper::wrapUsingExponentialBackOff($sendHttpRequest, 5000, '\TooManyRequestsHttpException');
*
* // We perform 100 requests. But remote server has RateLimit.
* for ($i = 1; $i <= 100; $i++) {
* // $sendHttpRequest($i); // request without wrapper
* $wrappedFn($i); // request with wrapper
* // this wrapper retry callback execution in the case of \TooManyRequestsHttpException exception.
* }
* </code>
*
* @param \Closure $callbackFn Callback function.
* @param int $maxSleepMs Max value of timeout between callback execution.
* @param string $rateLimitExceptionClassName Exception that should throw callback on rate limit.
* @param int $maxRetries Max number of retries.
* @param \Closure $onTooManyRequest Callback function that performed on each RateLimit exception.
*
* @return \Closure
*/
public static function wrapUsingExponentialBackOff(\Closure $callbackFn, $maxSleepMs, $rateLimitExceptionClassName, $maxRetries = 10, \Closure $onTooManyRequest = null)
{
$wrapFn = function () use ($callbackFn, $maxSleepMs, $rateLimitExceptionClassName, $maxRetries, $onTooManyRequest) {
$currentTry = 0;
$retriesRemain = $maxRetries;
while ($retriesRemain-- > 0) {
try {
$currentTry++;
return call_user_func_array($callbackFn, func_get_args());
} catch (\Exception $exception) {
// retries only throttling errors
if ($exception instanceof $rateLimitExceptionClassName) {
$backOffDuration = static::_getSleepDuration($currentTry, 10, $maxSleepMs);
if ($onTooManyRequest) {
$onTooManyRequest($exception, $backOffDuration, $retriesRemain, $currentTry);
}
usleep($backOffDuration * 1000);
} else {
throw $exception;
}
}
}
throw new \Exception('ExponentialBackOff: Too many retries of function call [maxRetries=' . $maxRetries . ']');
};
return $wrapFn;
}
protected static function _getSleepDuration($currentTry, $minSleepMs, $maxSleepMs)
{
$currentSleepMs = (int)($minSleepMs * pow(2, $currentTry));
return min($currentSleepMs, $maxSleepMs);
}
}
How it use
ApiHelper::wrapUsingExponentialBackOff()
method$sendHttpRequest = function ($pageNum) use ($httpClient) {
$httpClient->sendGet('http://example.com?pageNum=' . $pageNum);
};
$wrappedFn = ApiHelper::wrapUsingExponentialBackOff($sendHttpRequest, 5000, '\TooManyRequestsHttpException');
// We perform 100 requests. But remote server has RateLimit.
for ($i = 1; $i <= 100; $i++) {
// $sendHttpRequest($i); // request without wrapper
$wrappedFn($i); // request with wrapper
// this wrapper retry callback execution in the case of \TooManyRequestsHttpException exception.
}
How it works
This helper uses "Exponential backoff" algorithm.
The main idea of this algorithm is to reduce the rate at which you are executing an operation by introducing delays. Therefore when you are "backing off" you are waiting for a period of time before attempting to execute the operation again.
Wrapped callback function $wrappedFn
will retry to execute specified original callback function $sendHttpRequest
if it throws exception.
Чтобы увидеть комментарии, нужно быть участником сообщества
Регистрация