Usage
The entrypoint to pyqwest is Client for asyncio applications
and SyncClient for synchronous applications.
Clients are lightweight - while we generally expect you'll initialize them once for your application,
it should generally be fine to even create them per-operation. There is no close type of method on
a client because they share an application-scoped default transport, which is the actual connection
pool. This should feel familiar to those coming from Go's net/http.
With a client, you can use methods corresponding to the HTTP methods, or execute, to issue a request
and get back a full response.
Multipart forms
To send a multipart/form-data request, for example to upload files, pass a
Multipart object as the request content with Client, or a
SyncMultipart with SyncClient. Parts can be provided
as bytes or str for simple form fields, or as a Part /
SyncPart to set a filename or part headers such as a
content type. A part's content can also be an iterator of bytes to stream it. The
multipart boundary is generated automatically when constructing the request and the
content-type header is set to match it.
from pyqwest import Multipart, Part
async def file_chunks():
yield b"file "
yield b"content"
response = await client.post(
"https://httpbingo.org/post",
content=Multipart({
"field": "value",
"file": Part(file_chunks(), filename="hello.txt", headers={"content-type": "text/plain"}),
}),
)
print(response.text())
from pyqwest import SyncMultipart, SyncPart
def file_chunks():
yield b"file "
yield b"content"
response = client.post(
"https://httpbingo.org/post",
content=SyncMultipart({
"field": "value",
"file": SyncPart(file_chunks(), filename="hello.txt", headers={"content-type": "text/plain"}),
}),
)
print(response.text())
Transport
The default transport is setup to behave closely to a web browser, using standard root certificates
and having timeouts, TCP keepalive, etc configured in a reasonable way, borrowing from the defaults
of the Go net/http package. You may need a custom transport though to configure TLS settings or
timeouts, in which case you create an HTTPTransport or SyncHTTPTransport. Unlike clients, transports
are heavy, with connection pools. Generally you should only create one per application and ensure it
is closed.
TLS
The default transport will use standard root certificates that can access sites served via
https in the same way as a browser would. For internal use cases, you may use certificates
issued by a custom certificate authority. You can initialize an HTTPTransport
or SyncHTTPTransport with a CA certificate for this case.
If using mTLS with client certificates, just add tls_cert and tls_key similarly.
Middleware
HTTP middleware are themselves just Transport implementations that accept another
Transport to wrap it. By having the same signature, they can operate on any part of the
request/response lifecycle.
pyqwest includes the following middleware.
Retry
Retry middleware automatically reissues requests on errors. The default behavior is to
retry known-safe errors, which include connection errors and transient error responses for
GET, HEAD, PUT, and DELETE. Whether a request or response is retryable can be customized by
subclassing the middleware class and implementing should_retry_request or
should_retry_response, for example to match against request.url.
should_retry_request may return False to disable retries, True to retry if
should_retry_response also returns True, or a RetryMode explicitly.
RetryMode.BUFFERED retains streamed request content in memory as it is sent so it can be
replayed. This can increase peak memory usage by the full size of the body even when the
first attempt succeeds. RetryMode.UNBUFFERED retains no streamed content and makes streamed
requests only retry connection errors. Content provided as bytes is already replayable, so
it follows the normal response retry policy in either mode.
from pyqwest import Client, HTTPTransport, Request
from pyqwest.middleware.retry import RetryMode, RetryTransport
class MyRetryTransport(RetryTransport):
def should_retry_request(self, request: Request) -> bool | RetryMode:
if request.url.endswith("/unsafe-method"):
return False
return RetryMode.UNBUFFERED
client = Client(transport=MyRetryTransport(HTTPTransport()))
await client.get(
"http://localhost/safe-method"
) # will retry on transient errors
await client.get("http://localhost/unsafe-method") # will not retry
from pyqwest import SyncClient, SyncHTTPTransport, SyncRequest
from pyqwest.middleware.retry import RetryMode, SyncRetryTransport
class MyRetryTransport(SyncRetryTransport):
def should_retry_request(self, request: SyncRequest) -> bool | RetryMode:
if request.url.endswith("/unsafe-method"):
return False
return RetryMode.UNBUFFERED
client = SyncClient(transport=MyRetryTransport(SyncHTTPTransport()))
client.get("http://localhost/safe-method") # will retry on transient errors
client.get("http://localhost/unsafe-method") # will not retry
Proxies
The transport can be configured to send all requests through a proxy by passing its URL.
The URL scheme may be http, https, socks5, or socks5h. Credentials in the URL
will be used for proxy authentication.
For more control, pass a Proxy object instead of a URL. It supports basic
authentication without embedding credentials in the URL, extra headers to send
to the proxy, restricting the proxy to http or https requests, and a
no_proxy exclusion list of hosts that should connect directly.
from pyqwest import Proxy
proxy = Proxy(
"http://localhost:8030",
auth=("user", "pass"),
headers={"x-tenant": "my-tenant"},
no_proxy="localhost, internal.example.com",
scheme="https",
)
A sequence of proxies can also be passed to apply multiple routing rules. The
first proxy matching a request is used. An empty sequence, like None,
configures no explicit proxy, in which case proxy environment variables such
as HTTP_PROXY still apply.
Timeouts
The transport can be configured with timeouts for overall operations, connect, and reads.
The overall operation timeout can also be configured per-call to override the transport's
setting by passing timeout for sync clients or using asyncio.wait_for or
asyncio.timeout for async. Connect and read timeout cannot be configured per-call.
Redirects
By default, transports follow redirects automatically up to a maximum of 10, which can be
configured with max_redirects.
To instead receive 301 or 302 responses directly, set follow_redirects=False.
A request that exceeds max_redirects fails with pyqwest.TooManyRedirects.
When using pyqwest as an httpx transport, you may prefer to disable follow_redirects
so HTTPX handles them as usual, notably filling response.history.
Logging
pyqwest integrates with Python's standard logging
module using two loggers, both emitting at DEBUG level:
pyqwest.access— one summary line per HTTP request, in the same format as httpx, emitted once response headers are received.pyqwest— granular request lifecycle records, e.g. when a request is sent or fails without a response.
Unlike httpx, no records are emitted at INFO level, so enabling INFO for your whole
application will not also log every HTTP request. Opt in explicitly instead:
import logging
logging.basicConfig(
format="%(levelname)s [%(asctime)s] %(name)s - %(message)s",
datefmt="%Y-%m-%d %H:%M:%S",
)
logging.getLogger("pyqwest.access").setLevel(logging.DEBUG)
response = client.get("https://pyqwest.dev")
Will send log output such as:
Setting the pyqwest logger to DEBUG instead enables the full request lifecycle,
including the access records through standard logger inheritance. The two never
duplicate each other, and an explicit level on pyqwest.access always takes
precedence, so the access log can be enabled on its own or silenced while keeping
the granular records.
The log records are emitted directly from the Rust HTTP transports with no overhead
when neither logger is enabled for DEBUG. Requests that fail without a response,
such as connection errors, appear only on the pyqwest logger, keeping the access
log to requests with responses like httpx.