Added github integration
This commit is contained in:
@@ -0,0 +1 @@
|
||||
pip
|
||||
@@ -0,0 +1,165 @@
|
||||
GNU LESSER GENERAL PUBLIC LICENSE
|
||||
Version 3, 29 June 2007
|
||||
|
||||
Copyright (C) 2007 Free Software Foundation, Inc. <https://fsf.org/>
|
||||
Everyone is permitted to copy and distribute verbatim copies
|
||||
of this license document, but changing it is not allowed.
|
||||
|
||||
|
||||
This version of the GNU Lesser General Public License incorporates
|
||||
the terms and conditions of version 3 of the GNU General Public
|
||||
License, supplemented by the additional permissions listed below.
|
||||
|
||||
0. Additional Definitions.
|
||||
|
||||
As used herein, "this License" refers to version 3 of the GNU Lesser
|
||||
General Public License, and the "GNU GPL" refers to version 3 of the GNU
|
||||
General Public License.
|
||||
|
||||
"The Library" refers to a covered work governed by this License,
|
||||
other than an Application or a Combined Work as defined below.
|
||||
|
||||
An "Application" is any work that makes use of an interface provided
|
||||
by the Library, but which is not otherwise based on the Library.
|
||||
Defining a subclass of a class defined by the Library is deemed a mode
|
||||
of using an interface provided by the Library.
|
||||
|
||||
A "Combined Work" is a work produced by combining or linking an
|
||||
Application with the Library. The particular version of the Library
|
||||
with which the Combined Work was made is also called the "Linked
|
||||
Version".
|
||||
|
||||
The "Minimal Corresponding Source" for a Combined Work means the
|
||||
Corresponding Source for the Combined Work, excluding any source code
|
||||
for portions of the Combined Work that, considered in isolation, are
|
||||
based on the Application, and not on the Linked Version.
|
||||
|
||||
The "Corresponding Application Code" for a Combined Work means the
|
||||
object code and/or source code for the Application, including any data
|
||||
and utility programs needed for reproducing the Combined Work from the
|
||||
Application, but excluding the System Libraries of the Combined Work.
|
||||
|
||||
1. Exception to Section 3 of the GNU GPL.
|
||||
|
||||
You may convey a covered work under sections 3 and 4 of this License
|
||||
without being bound by section 3 of the GNU GPL.
|
||||
|
||||
2. Conveying Modified Versions.
|
||||
|
||||
If you modify a copy of the Library, and, in your modifications, a
|
||||
facility refers to a function or data to be supplied by an Application
|
||||
that uses the facility (other than as an argument passed when the
|
||||
facility is invoked), then you may convey a copy of the modified
|
||||
version:
|
||||
|
||||
a) under this License, provided that you make a good faith effort to
|
||||
ensure that, in the event an Application does not supply the
|
||||
function or data, the facility still operates, and performs
|
||||
whatever part of its purpose remains meaningful, or
|
||||
|
||||
b) under the GNU GPL, with none of the additional permissions of
|
||||
this License applicable to that copy.
|
||||
|
||||
3. Object Code Incorporating Material from Library Header Files.
|
||||
|
||||
The object code form of an Application may incorporate material from
|
||||
a header file that is part of the Library. You may convey such object
|
||||
code under terms of your choice, provided that, if the incorporated
|
||||
material is not limited to numerical parameters, data structure
|
||||
layouts and accessors, or small macros, inline functions and templates
|
||||
(ten or fewer lines in length), you do both of the following:
|
||||
|
||||
a) Give prominent notice with each copy of the object code that the
|
||||
Library is used in it and that the Library and its use are
|
||||
covered by this License.
|
||||
|
||||
b) Accompany the object code with a copy of the GNU GPL and this license
|
||||
document.
|
||||
|
||||
4. Combined Works.
|
||||
|
||||
You may convey a Combined Work under terms of your choice that,
|
||||
taken together, effectively do not restrict modification of the
|
||||
portions of the Library contained in the Combined Work and reverse
|
||||
engineering for debugging such modifications, if you also do each of
|
||||
the following:
|
||||
|
||||
a) Give prominent notice with each copy of the Combined Work that
|
||||
the Library is used in it and that the Library and its use are
|
||||
covered by this License.
|
||||
|
||||
b) Accompany the Combined Work with a copy of the GNU GPL and this license
|
||||
document.
|
||||
|
||||
c) For a Combined Work that displays copyright notices during
|
||||
execution, include the copyright notice for the Library among
|
||||
these notices, as well as a reference directing the user to the
|
||||
copies of the GNU GPL and this license document.
|
||||
|
||||
d) Do one of the following:
|
||||
|
||||
0) Convey the Minimal Corresponding Source under the terms of this
|
||||
License, and the Corresponding Application Code in a form
|
||||
suitable for, and under terms that permit, the user to
|
||||
recombine or relink the Application with a modified version of
|
||||
the Linked Version to produce a modified Combined Work, in the
|
||||
manner specified by section 6 of the GNU GPL for conveying
|
||||
Corresponding Source.
|
||||
|
||||
1) Use a suitable shared library mechanism for linking with the
|
||||
Library. A suitable mechanism is one that (a) uses at run time
|
||||
a copy of the Library already present on the user's computer
|
||||
system, and (b) will operate properly with a modified version
|
||||
of the Library that is interface-compatible with the Linked
|
||||
Version.
|
||||
|
||||
e) Provide Installation Information, but only if you would otherwise
|
||||
be required to provide such information under section 6 of the
|
||||
GNU GPL, and only to the extent that such information is
|
||||
necessary to install and execute a modified version of the
|
||||
Combined Work produced by recombining or relinking the
|
||||
Application with a modified version of the Linked Version. (If
|
||||
you use option 4d0, the Installation Information must accompany
|
||||
the Minimal Corresponding Source and Corresponding Application
|
||||
Code. If you use option 4d1, you must provide the Installation
|
||||
Information in the manner specified by section 6 of the GNU GPL
|
||||
for conveying Corresponding Source.)
|
||||
|
||||
5. Combined Libraries.
|
||||
|
||||
You may place library facilities that are a work based on the
|
||||
Library side by side in a single library together with other library
|
||||
facilities that are not Applications and are not covered by this
|
||||
License, and convey such a combined library under terms of your
|
||||
choice, if you do both of the following:
|
||||
|
||||
a) Accompany the combined library with a copy of the same work based
|
||||
on the Library, uncombined with any other library facilities,
|
||||
conveyed under the terms of this License.
|
||||
|
||||
b) Give prominent notice with the combined library that part of it
|
||||
is a work based on the Library, and explaining where to find the
|
||||
accompanying uncombined form of the same work.
|
||||
|
||||
6. Revised Versions of the GNU Lesser General Public License.
|
||||
|
||||
The Free Software Foundation may publish revised and/or new versions
|
||||
of the GNU Lesser General Public License from time to time. Such new
|
||||
versions will be similar in spirit to the present version, but may
|
||||
differ in detail to address new problems or concerns.
|
||||
|
||||
Each version is given a distinguishing version number. If the
|
||||
Library as you received it specifies that a certain numbered version
|
||||
of the GNU Lesser General Public License "or any later version"
|
||||
applies to it, you have the option of following the terms and
|
||||
conditions either of that published version or of any later version
|
||||
published by the Free Software Foundation. If the Library as you
|
||||
received it does not specify a version number of the GNU Lesser
|
||||
General Public License, you may choose any version of the GNU Lesser
|
||||
General Public License ever published by the Free Software Foundation.
|
||||
|
||||
If the Library as you received it specifies that a proxy can decide
|
||||
whether future versions of the GNU Lesser General Public License shall
|
||||
apply, that proxy's public statement of acceptance of any version is
|
||||
permanent authorization for you to choose that version for the
|
||||
Library.
|
||||
@@ -0,0 +1,338 @@
|
||||
Metadata-Version: 2.1
|
||||
Name: limiter
|
||||
Version: 0.5.0
|
||||
Summary: ⏲️ Easy rate limiting for Python. Rate limiting async and thread-safe decorators and context managers that use a token bucket algorithm.
|
||||
Home-page: https://github.com/alexdelorenzo/limiter
|
||||
Author: Alex DeLorenzo
|
||||
License: LGPL-3.0
|
||||
Keywords: rate-limit,rate,limit,token,bucket,token-bucket,token_bucket,tokenbucket,decorator,contextmanager,asynchronous,threadsafe,synchronous
|
||||
Requires-Python: >=3.10
|
||||
Description-Content-Type: text/markdown
|
||||
License-File: LICENSE
|
||||
Requires-Dist: strenum <0.5.0,>=0.4.7
|
||||
Requires-Dist: token-bucket <0.4.0,>=0.3.0
|
||||
|
||||
# ⏲️ Easy rate limiting for Python
|
||||
|
||||
`limiter` makes it easy to add [rate limiting](https://en.wikipedia.org/wiki/Rate_limiting) to Python projects, using
|
||||
a [token bucket](https://en.wikipedia.org/wiki/Token_bucket) algorithm. `limiter` can provide Python projects and
|
||||
scripts with:
|
||||
|
||||
- Rate limiting thread-safe [decorators](https://www.python.org/dev/peps/pep-0318/)
|
||||
- Rate limiting async decorators
|
||||
- Rate limiting thread-safe [context managers](https://www.python.org/dev/peps/pep-0343/)
|
||||
- Rate
|
||||
limiting [async context managers](https://www.python.org/dev/peps/pep-0492/#asynchronous-context-managers-and-async-with)
|
||||
|
||||
Here are some features and benefits of using `limiter`:
|
||||
|
||||
- Easily control burst and average request rates
|
||||
- It
|
||||
is [thread-safe, with no need for a timer thread](https://en.wikipedia.org/wiki/Generic_cell_rate_algorithm#Comparison_with_the_token_bucket)
|
||||
- It adds [jitter](https://aws.amazon.com/blogs/architecture/exponential-backoff-and-jitter/) to help with contention
|
||||
- It has a simple API that takes advantage of Python's features, idioms
|
||||
and [type hinting](https://www.python.org/dev/peps/pep-0483/)
|
||||
|
||||
## Example
|
||||
|
||||
Here's an example of using a limiter as a decorator and context manager:
|
||||
|
||||
```python
|
||||
from aiohttp import ClientSession
|
||||
from limiter import Limiter
|
||||
|
||||
|
||||
limit_downloads = Limiter(rate=2, capacity=5, consume=2)
|
||||
|
||||
|
||||
@limit_downloads
|
||||
async def download_image(url: str) -> bytes:
|
||||
async with ClientSession() as session, session.get(url) as response:
|
||||
return await response.read()
|
||||
|
||||
|
||||
async def download_page(url: str) -> str:
|
||||
async with (
|
||||
ClientSession() as session,
|
||||
limit_downloads,
|
||||
session.get(url) as response
|
||||
):
|
||||
return await response.text()
|
||||
```
|
||||
|
||||
## Usage
|
||||
|
||||
You can define limiters and use them dynamically across your project.
|
||||
|
||||
**Note**: If you're using Python version `3.9.x` or below, check
|
||||
out [the documentation for version `0.2.0` of `limiter` here](https://github.com/alexdelorenzo/limiter/blob/master/README-0.2.0.md).
|
||||
|
||||
### `Limiter` instances
|
||||
|
||||
`Limiter` instances take `rate`, `capacity` and `consume` arguments.
|
||||
|
||||
- `rate` is the token replenishment rate per second. Tokens are automatically added every second.
|
||||
- `consume` is the amount of tokens consumed from the token bucket upon successfully taking tokens from the bucket.
|
||||
- `capacity` is the total amount of tokens the token bucket can hold. Token replenishment stops when this capacity is
|
||||
reached.
|
||||
|
||||
### Limiting blocks of code
|
||||
|
||||
`limiter` can rate limit all Python callables, and limiters can be used as context managers.
|
||||
|
||||
You can define a limiter with a set refresh `rate` and total token `capacity`. You can set the amount of tokens to
|
||||
consume dynamically with `consume`, and the `bucket` parameter sets the bucket to consume tokens from:
|
||||
|
||||
```python3
|
||||
from limiter import Limiter
|
||||
|
||||
|
||||
REFRESH_RATE: int = 2
|
||||
BURST_RATE: int = 3
|
||||
MSG_BUCKET: str = 'messages'
|
||||
|
||||
limiter: Limiter = Limiter(rate=REFRESH_RATE, capacity=BURST_RATE)
|
||||
limit_msgs: Limiter = limiter(bucket=MSG_BUCKET)
|
||||
|
||||
|
||||
@limiter
|
||||
def download_page(url: str) -> bytes:
|
||||
...
|
||||
|
||||
|
||||
@limiter(consume=2)
|
||||
async def download_page(url: str) -> bytes:
|
||||
...
|
||||
|
||||
|
||||
def send_page(page: bytes):
|
||||
with limiter(consume=1.5, bucket=MSG_BUCKET):
|
||||
...
|
||||
|
||||
|
||||
async def send_page(page: bytes):
|
||||
async with limit_msgs:
|
||||
...
|
||||
|
||||
|
||||
@limit_msgs(consume=3)
|
||||
def send_email(to: str):
|
||||
...
|
||||
|
||||
|
||||
async def send_email(to: str):
|
||||
async with limiter(bucket=MSG_BUCKET):
|
||||
...
|
||||
```
|
||||
|
||||
In the example above, both `limiter` and `limit_msgs` share the same limiter. The only difference is that `limit_msgs`
|
||||
will take tokens from the `MSG_BUCKET` bucket by default.
|
||||
|
||||
```python3
|
||||
assert limiter.limiter is limit_msgs.limiter
|
||||
assert limiter.bucket != limit_msgs.bucket
|
||||
assert limiter != limit_msgs
|
||||
```
|
||||
|
||||
### Creating new limiters
|
||||
|
||||
You can reuse existing limiters in your code, and you can create new limiters from the parameters of an existing limiter
|
||||
using the `new()` method.
|
||||
|
||||
Or, you can define a new limiter entirely:
|
||||
|
||||
```python
|
||||
# you can reuse existing limiters
|
||||
limit_downloads: Limiter = limiter(consume=2)
|
||||
|
||||
# you can use the settings from an existing limiter in a new limiter
|
||||
limit_downloads: Limiter = limiter.new(consume=2)
|
||||
|
||||
# or you can simply define a new limiter
|
||||
limit_downloads: Limiter = Limiter(REFRESH_RATE, BURST_RATE, consume=2)
|
||||
|
||||
|
||||
@limit_downloads
|
||||
def download_page(url: str) -> bytes:
|
||||
...
|
||||
|
||||
|
||||
@limit_downloads
|
||||
async def download_page(url: str) -> bytes:
|
||||
...
|
||||
|
||||
|
||||
def download_image(url: str) -> bytes:
|
||||
with limit_downloads:
|
||||
...
|
||||
|
||||
|
||||
async def download_image(url: str) -> bytes:
|
||||
async with limit_downloads:
|
||||
...
|
||||
```
|
||||
|
||||
Let's look at the difference between reusing an existing limiter, and creating new limiters with the `new()` method:
|
||||
|
||||
```python3
|
||||
limiter_a: Limiter = limiter(consume=2)
|
||||
limiter_b: Limiter = limiter.new(consume=2)
|
||||
limiter_c: Limiter = Limiter(REFRESH_RATE, BURST_RATE, consume=2)
|
||||
|
||||
assert limiter_a != limiter
|
||||
assert limiter_a != limiter_b != limiter_c
|
||||
|
||||
assert limiter_a != limiter_b
|
||||
assert limiter_a.limiter is limiter.limiter
|
||||
assert limiter_a.limiter is not limiter_b.limiter
|
||||
|
||||
assert limiter_a.attrs == limiter_b.attrs == limiter_c.attrs
|
||||
```
|
||||
|
||||
The only things that are equivalent between the three new limiters above are the limiters' attributes, like
|
||||
the `rate`, `capacity`, and `consume` attributes.
|
||||
|
||||
### Creating anonymous, or single-use, limiters
|
||||
|
||||
You don't have to assign `Limiter` objects to variables. Anonymous limiters don't share a token bucket like named
|
||||
limiters can. They work well when you don't have a reason to share a limiter between two or more blocks of code, and
|
||||
when a limiter has a single or independent purpose.
|
||||
|
||||
`limiter`, after version `v0.3.0`, ships with a `limit` type alias for `Limiter`:
|
||||
|
||||
```python3
|
||||
from limiter import limit
|
||||
|
||||
|
||||
@limit(capacity=2, consume=2)
|
||||
async def send_message():
|
||||
...
|
||||
|
||||
|
||||
async def upload_image():
|
||||
async with limit(capacity=3) as limiter:
|
||||
...
|
||||
```
|
||||
|
||||
The above is equivalent to the below:
|
||||
|
||||
```python3
|
||||
from limiter import Limiter
|
||||
|
||||
|
||||
@Limiter(capacity=2, consume=2)
|
||||
async def send_message():
|
||||
...
|
||||
|
||||
|
||||
async def upload_image():
|
||||
async with Limiter(capacity=3) as limiter:
|
||||
...
|
||||
```
|
||||
|
||||
Both `limit` and `Limiter` are the same object:
|
||||
|
||||
```python3
|
||||
assert limit is Limiter
|
||||
```
|
||||
|
||||
### Jitter
|
||||
|
||||
A `Limiter`'s `jitter` argument adds jitter to help with contention.
|
||||
|
||||
The value is in `units`, which is milliseconds by default, and can be any of these:
|
||||
|
||||
- `False`, to add no jitter. This is the default.
|
||||
- `True`, to add a random amount of jitter.
|
||||
- A number, to add a fixed amount of jitter.
|
||||
- A `range` object, to add a random amount of jitter within the range.
|
||||
- A `tuple` of two numbers, `start` and `stop`, to add a random amount of jitter between the two numbers.
|
||||
- A `tuple` of three numbers: `start`, `stop` and `step`, to add jitter like you would with `range`.
|
||||
|
||||
For example, if you want to use a random amount of jitter between `0` and `100` milliseconds:
|
||||
|
||||
```python3
|
||||
limiter = Limiter(rate=2, capacity=5, consume=2, jitter=(0, 100))
|
||||
limiter = Limiter(rate=2, capacity=5, consume=2, jitter=(0, 100, 1))
|
||||
limiter = Limiter(rate=2, capacity=5, consume=2, jitter=range(0, 100))
|
||||
limiter = Limiter(rate=2, capacity=5, consume=2, jitter=range(0, 100, 1))
|
||||
```
|
||||
|
||||
All of the above are equivalent to each other in function.
|
||||
|
||||
You can also supply values for `jitter` when using decorators or context-managers:
|
||||
|
||||
```python3
|
||||
limiter = Limiter(rate=2, capacity=5, consume=2)
|
||||
|
||||
|
||||
@limiter(jitter=range(0, 100))
|
||||
def download_page(url: str) -> bytes:
|
||||
...
|
||||
|
||||
|
||||
async def download_page(url: str) -> bytes:
|
||||
async with limiter(jitter=(0, 100)):
|
||||
...
|
||||
```
|
||||
|
||||
You can use the above to override default values of `jitter` in a `Limiter` instance.
|
||||
|
||||
|
||||
To add a small amount of random jitter, supply `True` as the value:
|
||||
```python3
|
||||
limiter = Limiter(rate=2, capacity=5, consume=2, jitter=True)
|
||||
|
||||
# or
|
||||
|
||||
@limiter(jitter=True)
|
||||
def download_page(url: str) -> bytes:
|
||||
...
|
||||
```
|
||||
|
||||
To turn off jitter in a `Limiter` configured with jitter, you can supply `False` as the value:
|
||||
|
||||
```python3
|
||||
limiter = Limiter(rate=2, capacity=5, consume=2, jitter=range(10))
|
||||
|
||||
|
||||
@limiter(jitter=False)
|
||||
def download_page(url: str) -> bytes:
|
||||
...
|
||||
|
||||
|
||||
async def download_page(url: str) -> bytes:
|
||||
async with limiter(jitter=False):
|
||||
...
|
||||
```
|
||||
|
||||
Or create a new limiter with jitter turned off:
|
||||
|
||||
```python3
|
||||
limiter: Limiter = limiter.new(jitter=False)
|
||||
```
|
||||
|
||||
### Units
|
||||
|
||||
`units` is a number representing the amount of units in one second. The default value is `1000` for 1,000 milliseconds in one second.
|
||||
|
||||
Similar to `jitter`, `units` can be supplied at all the same call sites and constructors that `jitter` is accepted.
|
||||
|
||||
If you want to use a different unit than milliseconds, supply a different value for `units`.
|
||||
|
||||
## Installation
|
||||
|
||||
### Requirements
|
||||
|
||||
- Python 3.10+ for versions `0.3.0` and up
|
||||
- [Python 3.7+ for versions below `0.3.0`](https://github.com/alexdelorenzo/limiter/blob/master/README-0.2.0.md)
|
||||
|
||||
### Install via PyPI
|
||||
|
||||
```bash
|
||||
$ python3 -m pip install limiter
|
||||
```
|
||||
|
||||
## License
|
||||
|
||||
See [`LICENSE`](/LICENSE). If you'd like to use this project with a different license, please get in touch.
|
||||
@@ -0,0 +1,14 @@
|
||||
limiter-0.5.0.dist-info/INSTALLER,sha256=zuuue4knoyJ-UwPPXg8fezS7VCrXJQrAP7zeNuwvFQg,4
|
||||
limiter-0.5.0.dist-info/LICENSE,sha256=46mU2C5kSwOnkqkw9XQAJlhBL2JAf1_uCD8lVcXyMRg,7652
|
||||
limiter-0.5.0.dist-info/METADATA,sha256=MwITStcmPl2MY7At0JDbwhWhPk723uiQB-WrTbZN_FY,9660
|
||||
limiter-0.5.0.dist-info/RECORD,,
|
||||
limiter-0.5.0.dist-info/REQUESTED,sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU,0
|
||||
limiter-0.5.0.dist-info/WHEEL,sha256=-G_t0oGuE7UD0DrSpVZnq1hHMBV9DD2XkS5v7XpmTnk,110
|
||||
limiter-0.5.0.dist-info/top_level.txt,sha256=rqG3yt65wf_ybeha_4dH4LNZI8b__T9xih_yH7H1VA8,8
|
||||
limiter-0.5.0.dist-info/zip-safe,sha256=AbpHGcgLb-kRsJGnwFEktk7uzpZOCcBY74-YBdrKVGs,1
|
||||
limiter/__init__.py,sha256=UQbgzz-eV_Mv_62-9Hjeq6SqYi4K1kMHjy_iOEK4LwQ,136
|
||||
limiter/__pycache__/__init__.cpython-311.pyc,,
|
||||
limiter/__pycache__/base.cpython-311.pyc,,
|
||||
limiter/__pycache__/limiter.cpython-311.pyc,,
|
||||
limiter/base.py,sha256=qSyc9PQT1Mg3ssteyYmmKOOTt7E2Yo3qoVTQsk8ZJkM,2339
|
||||
limiter/limiter.py,sha256=oI8wsqHttd4Ja43PuqIBm5J5n5MqAwN4XaTY2E2WrhY,6681
|
||||
@@ -0,0 +1,6 @@
|
||||
Wheel-Version: 1.0
|
||||
Generator: bdist_wheel (0.42.0)
|
||||
Root-Is-Purelib: true
|
||||
Tag: py2-none-any
|
||||
Tag: py3-none-any
|
||||
|
||||
@@ -0,0 +1 @@
|
||||
limiter
|
||||
@@ -0,0 +1 @@
|
||||
|
||||
Reference in New Issue
Block a user