HTTP & TLS

This page documents the HTTP client implementation, including TLS configuration, connection pooling, and request handling. The HTTP layer provides the foundation for all LLM API communication.

HttpClient Structure

The HTTP client wraps a Hyper client with TLS support:

#[derive(Clone)]
pub struct HttpClient {
    client: HttpsClient,
}

type HttpsClient = Client<
    hyper_rustls::HttpsConnector<
        hyper_util::client::legacy::connect::HttpConnector
    >,
    Full<Bytes>
>;

Component Stack

TLS Configuration

TLS is configured using native system roots:

impl HttpClient {
    pub fn new() -> Result<Self, LlmError> {
        let https = HttpsConnectorBuilder::new()
            .with_native_roots()
            .map_err(|e| {
                LlmError::new(
                    "TLS_INIT_FAILED",
                    format!("failed to load native TLS roots: {}", e),
                )
            })?
            .https_or_http()
            .enable_http1()
            .build();

        let client = Client::builder(TokioExecutor::new()).build(https);
        Ok(Self { client })
    }
}

TLS Features

Native Roots

The with_native_roots() method loads CA certificates from:

• Linux: /etc/ssl/certs/ or system trust store
• macOS: Keychain Access
• Windows: Certificate Store

This ensures compatibility with enterprise proxy configurations.

Connection Pooling

Hyper’s client automatically manages connection pooling:

// Connections are reused for the same host
let response1 = client.post("https://api.anthropic.com/v1/messages", ...).await?;
let response2 = client.post("https://api.anthropic.com/v1/messages", ...).await?;
// Second request may reuse first connection

Pool Behavior

• Connections are kept alive by default
• Idle connections are closed after timeout
• Pool size scales with concurrent requests

POST Requests

The primary method for API calls:

pub async fn post(
    &self,
    uri: &str,
    headers: &[(&str, &str)],
    body: &str,
) -> Result<String, LlmError> {
    let parsed_uri: hyper::Uri = uri
        .parse()
        .map_err(|e| LlmError::new("HTTP_INVALID_URI", format!("{}", e)))?;

    let mut builder = Request::builder()
        .method(Method::POST)
        .uri(parsed_uri.clone());

    for (key, value) in headers {
        builder = builder.header(*key, *value);
    }

    let request = builder
        .body(Full::new(Bytes::from(body.to_string())))
        .map_err(|e| LlmError::new("HTTP_REQUEST_BUILD", format!("{}", e)))?;

    let res = self
        .client
        .request(request)
        .await
        .map_err(|e| LlmError::new("HTTP_REQUEST_FAILED", format!("{}", e)))?;

    let status = res.status();

    let response_body = res
        .collect()
        .await
        .map_err(|e| LlmError::new("HTTP_BODY_READ", format!("{}", e)))?
        .to_bytes();

    let response_text = String::from_utf8(response_body.to_vec())
        .map_err(|e| LlmError::new("HTTP_INVALID_UTF8", format!("{}", e)))?;

    Ok(response_text)
}

Streaming POST

For streaming responses, the client returns a byte stream:

pub async fn post_stream(
    &self,
    uri: &str,
    headers: &[(&str, &str)],
    body: &str,
) -> Result<Pin<Box<dyn Stream<Item = Result<Bytes, LlmError>> + Send>>, LlmError> {
    // Build and send request (same as post())
    let res = self.client.request(request).await?;
    let status = res.status();

    // Return stream of body chunks
    let response_body = res.into_body();
    let byte_stream = stream! {
        use http_body_util::BodyExt;
        let mut body = response_body;
        while let Some(frame_result) = body.frame().await {
            match frame_result {
                Ok(frame) => {
                    if let Some(data) = frame.data_ref() {
                        yield Ok(data.clone());
                    }
                }
                Err(e) => {
                    yield Err(LlmError::new("HTTP_STREAM_ERROR", format!("{}", e)));
                    break;
                }
            }
        }
    };

    Ok(Box::pin(byte_stream))
}

Header Management

Headers are passed as tuples for flexibility:

// Anthropic headers
let headers = vec![
    ("Content-Type", "application/json"),
    ("x-api-key", api_key.as_str()),
    ("anthropic-version", "2023-06-01"),
];

// OpenAI headers
let headers = vec![
    ("Content-Type", "application/json"),
    ("Authorization", format!("Bearer {}", api_key).as_str()),
];

Error Handling

HTTP errors are wrapped in LlmError:

Timeout Configuration

Timeouts can be configured at the request level:

use tokio::time::timeout;

let response = timeout(
    Duration::from_secs(30),
    client.post(uri, &headers, &body)
).await??;

For streaming, timeouts apply to connection establishment:

let stream = timeout(
    Duration::from_secs(10),
    client.post_stream(uri, &headers, &body)
).await??;

Request/Response Flow

┌─────────────────────────────────────────────────────────────────┐
│                    Application                                   │
│  client.post(uri, headers, body)                                │
└─────────────────────────────────┬───────────────────────────────┘
                                  │
                                  ▼
┌─────────────────────────────────────────────────────────────────┐
│                    Request Building                              │
│  - Parse URI                                                     │
│  - Add headers                                                   │
│  - Set body                                                      │
└─────────────────────────────────┬───────────────────────────────┘
                                  │
                                  ▼
┌─────────────────────────────────────────────────────────────────┐
│                    Hyper Client                                  │
│  - Connection pooling                                            │
│  - HTTP/1.1 protocol                                             │
└─────────────────────────────────┬───────────────────────────────┘
                                  │
                                  ▼
┌─────────────────────────────────────────────────────────────────┐
│                    HttpsConnector                                │
│  - TLS handshake                                                 │
│  - Certificate verification                                      │
└─────────────────────────────────┬───────────────────────────────┘
                                  │
                                  ▼
┌─────────────────────────────────────────────────────────────────┐
│                    Network                                       │
│  - DNS resolution                                                │
│  - TCP connection                                                │
│  - Data transfer                                                 │
└─────────────────────────────────────────────────────────────────┘

Thread Safety

The client is designed for concurrent use:

#[derive(Clone)]
pub struct HttpClient {
    client: HttpsClient,  // Hyper client is Clone + Send + Sync
}

Multiple tasks can share the same client:

let client = HttpClient::new()?;
let client_clone = client.clone();

let handle1 = tokio::spawn(async move {
    client.post(url1, &headers, &body1).await
});

let handle2 = tokio::spawn(async move {
    client_clone.post(url2, &headers, &body2).await
});

Dependencies

Next Steps

• Retry Logic - Automatic retry handling
• Streaming - SSE stream processing
• Client Overview - LLMClient structure