php 如何限制我网站的 API 用户？

How do I throttle my site's API users?

提问by scotts

The legitimate users of my site occasionally hammer the server with API requests that cause undesirable results. I want to institute a limit of no more than say one API call every 5 seconds or n calls per minute (haven't figured out the exact limit yet). I could obviously log every API call in a DB and do the calculation on every request to see if they're over the limit, but all this extra overhead on EVERY request would be defeating the purpose. What are other less resource-intensive methods I could use to institute a limit? I'm using PHP/Apache/Linux, for what it's worth.

我网站的合法用户偶尔会用 API 请求攻击服务器，从而导致不良结果。我想设置一个限制，不超过每 5 秒一次 API 调用或每分钟 n 次调用（还没有弄清楚确切的限制）。我显然可以将每个 API 调用记录到数据库中，并对每个请求进行计算以查看它们是否超过限制，但是每个请求的所有这些额外开销都将违背目的。我可以使用哪些其他资源密集度较低的方法来制定限制？我正在使用 PHP/Apache/Linux，这是值得的。

回答by scotts

Ok, there's no way to do what I asked without anywrites to the server, but I can at least eliminate logging every single request. One way is by using the "leaky bucket" throttling method, where it only keeps track of the last request ($last_api_request) and a ratio of the number of requests/limit for the time frame ($minute_throttle). The leaky bucket never resets its counter (unlike the Twitter API's throttle which resets every hour), but if the bucket becomes full (user reached the limit), they must wait nseconds for the bucket to empty a little before they can make another request. In other words it's like a rolling limit: if there are previous requests within the time frame, they are slowly leaking out of the bucket; it only restricts you if you fill the bucket.

好的，如果不对服务器进行任何写入，就无法执行我所要求的操作，但是我至少可以消除记录每个请求的情况。一种方法是使用“漏桶”节流方法，它只跟踪最后一个请求 ( $last_api_request) 和时间范围内请求数/限制的比率 ( $minute_throttle)。漏桶永远不会重置它的计数器（不像 Twitter API 的节流阀每小时重置一次），但是如果桶变满（用户达到限制），他们必须等待n几秒钟让桶清空，然后才能发出另一个请求。换句话说，它就像一个滚动限制：如果在时间范围内有先前的请求，它们就会慢慢地从桶中泄漏出来；它只会在你填满桶时限制你。

This code snippet will calculate a new $minute_throttlevalue on every request. I specified the minutein $minute_throttlebecause you can add throttles for any time period, such as hourly, daily, etc... although more than one will quickly start to make it confusing for the users.

此代码段将计算$minute_throttle每个请求的新值。我指定了分钟，$minute_throttle因为您可以为任何时间段添加节流阀，例如每小时、每天等……尽管不止一个会很快开始让用户感到困惑。

$minute = 60;
$minute_limit = 100; # users are limited to 100 requests/minute
$last_api_request = $this->get_last_api_request(); # get from the DB; in epoch seconds
$last_api_diff = time() - $last_api_request; # in seconds
$minute_throttle = $this->get_throttle_minute(); # get from the DB
if ( is_null( $minute_limit ) ) {
    $new_minute_throttle = 0;
} else {
    $new_minute_throttle = $minute_throttle - $last_api_diff;
    $new_minute_throttle = $new_minute_throttle < 0 ? 0 : $new_minute_throttle;
    $new_minute_throttle += $minute / $minute_limit;
    $minute_hits_remaining = floor( ( $minute - $new_minute_throttle ) * $minute_limit / $minute  );
    # can output this value with the request if desired:
    $minute_hits_remaining = $minute_hits_remaining >= 0 ? $minute_hits_remaining : 0;
}

if ( $new_minute_throttle > $minute ) {
    $wait = ceil( $new_minute_throttle - $minute );
    usleep( 250000 );
    throw new My_Exception ( 'The one-minute API limit of ' . $minute_limit 
        . ' requests has been exceeded. Please wait ' . $wait . ' seconds before attempting again.' );
}
# Save the values back to the database.
$this->save_last_api_request( time() );
$this->save_throttle_minute( $new_minute_throttle );

回答by Markus Malkusch

You can control the rate with the token bucket algorithm, which is comparable to the leaky bucket algorithm. Note that you will have to share the state of the bucket (i.e. the amount of tokens) over processes (or whatever scope you want to control). So you might want to think about locking to avoid race conditions.

您可以使用令牌桶算法来控制速率，这与漏桶算法相当。请注意，您必须通过进程（或您想要控制的任何范围）共享存储桶的状态（即令牌的数量）。因此，您可能需要考虑锁定以避免竞争条件。

The good news: I did all of that for you: bandwidth-throttle/token-bucket

好消息：我为你做了所有这些：带宽限制/令牌桶

use bandwidthThrottle\tokenBucket\Rate;
use bandwidthThrottle\tokenBucket\TokenBucket;
use bandwidthThrottle\tokenBucket\storage\FileStorage;

$storage = new FileStorage(__DIR__ . "/api.bucket");
$rate    = new Rate(10, Rate::SECOND);
$bucket  = new TokenBucket(10, $rate, $storage);
$bucket->bootstrap(10);

if (!$bucket->consume(1, $seconds)) {
    http_response_code(429);
    header(sprintf("Retry-After: %d", floor($seconds)));
    exit();
}

回答by Kedar Joshi

I don't know if this thread is still alive or not but I would suggest to keep these statistics in memory cache like memcached. This will reduce the overhead of logging the request to the DB but still serve the purpose.

我不知道这个线程是否还活着，但我建议将这些统计信息保存在像 memcached 这样的内存缓存中。这将减少将请求记录到数据库的开销，但仍然可以达到目的。

回答by Lasse V. Karlsen

Simplest solution would be to just give each API key a limited number of requests per 24 hours, and reset them at some known, fixed, time.

最简单的解决方案是每 24 小时为每个 API 密钥提供有限数量的请求，并在某个已知的固定时间重置它们。

If they exhaust their API requests (ie. the counter reaches zero, or the limit, depending on the direction you're counting), stop serving them data until you reset their counter.

如果他们耗尽了他们的 API 请求（即计数器达到零或限制，取决于您计数的方向），请停止向他们提供数据，直到您重置他们的计数器。

This way, it will be in theirbest interest to not hammer you with requests.

这样，不向您提出要求符合他们的最佳利益。

回答by steve

In addition to implementation from scratch you you can also take a look at API infrastructure like 3scale (http://www.3scale.net) which does rate limiting as well as a bunch of other stuff (analytics etc.). There's a PHP plugin for it: https://github.com/3scale/3scale_ws_api_for_php.

除了从头开始实现之外，您还可以查看 API 基础设施，如 3scale ( http://www.3scale.net)，它进行速率限制以及其他一些东西（分析等）。有一个 PHP 插件：https: //github.com/3scale/3scale_ws_api_for_php。

You can also stick something like Varnish infront of the API and do the API rate limiting like that.

你也可以在 API 前面贴上类似 Varnish 的东西，并像这样进行 API 速率限制。

回答by zombat

You say that "all thos extra overhead on EVERY request would be defeating the purpose", but I'm not sure that's correct. Isn't the purpose to prevent hammering of your server? This is probably the way I would implement it, as it really only requires a quick read/write. You could even farm out the API server checks to a different DB/disk if you were worried about the performance.

你说“每个请求的所有额外开销都会违背目的”，但我不确定这是正确的。目的不是防止您的服务器受到攻击吗？这可能是我实现它的方式，因为它实际上只需要快速读/写。如果您担心性能，您甚至可以将 API 服务器检查转移到不同的数据库/磁盘。

However, if you want alternatives, you should check out mod_cband, a third-party apache module designed to assist in bandwidth throttling. Despite being primarily for bandwidth limiting, it can throttle based on requests-per-second as well. I've never used it, so I'm not sure what kind of results you'd get. There was another module called mod-throttle as well, but that project appears to be closed now, and was never released for anything above the Apache 1.3 series.

但是，如果您想要替代方案，您应该查看mod_cband，这是一个旨在帮助带宽节流的第三方 apache 模块。尽管主要用于带宽限制，但它也可以根据每秒请求数进行节流。我从来没有用过它，所以我不确定你会得到什么样的结果。还有另一个名为 mod-throttle 的模块，但该项目现在似乎已关闭，并且从未针对 Apache 1.3 系列以上的任何内容发布。