27. Pattern: Response Shaping and Inclusion

The Response Shaping and Inclusion pattern empowers API clients to dictate the exact shape, depth, and breadth of the payload returned by the server. By specifying which related resources to expand or which specific fields to return, clients can optimize network bandwidth and reduce memory overhead, eliminating both over-fetching and under-fetching in a single request.

This pattern typically exposes two mechanisms for the client to declare their intent: an include query parameter, or the standard Prefer HTTP header. Furthermore, it utilizes dot notation to allow granular field-level selection within those included resources.

27.1. Overview

APIs often struggle to serve diverse clients with a single fixed response model. A mobile app over a 3G network may only need an order’s status and the product names, while an internal administrative dashboard needs the full order details, customer profiles, and complete product catalogs.

The Response Shaping pattern resolves this tension by allowing clients to request exactly what they need:

Full Resource Inclusion: Requesting the entire related resource to be embedded in the response (e.g., include=product).
Granular Field Selection (Dot Notation): Requesting only specific fields from that related resource (e.g., include=product.id,product.name).

Clients can pass these shaping instructions using one of two standard methods:

The Query Parameter (?include=...): Highly visible, easy to use in browsers, and ideal for standard GET requests.
The Prefer Header (Prefer: include="..."): Keeps URLs clean, avoids URI length limits, and conforms to RFC 7240 for declaring client preferences.

27.2. When to Use Response Shaping

Use Response Shaping when:

Serving diverse client platforms from a single API.
- Mobile clients can shape the response down to minimal bytes, while desktop web apps can request deeply nested relational data.
Aggregating data to prevent N+1 requests.
- Instead of fetching a list of orders and then individually fetching the product details for each order, a client can request /orders?include=product to get everything in one round-trip.
Relational data is expensive to serialize.
- If a resource has heavy relationships (e.g., a post with thousands of comments), those should be excluded by default and only embedded when the client explicitly uses include=comments.

27.3. When NOT to Use Response Shaping

Avoid Response Shaping when:

The API strictly serves a single, fixed-purpose client.
- If only one UI consumes the API and its data needs never vary, hardcoding the response shape is simpler to implement and test.
Caching is a primary performance driver at the CDN level.
- Allowing highly variable query parameters or headers fragments the cache space. Every unique combination of include fields creates a distinct cache key, drastically reducing cache hit rates.
Backend infrastructure cannot optimize the queries.
- If the database layer fetches the entire graph into memory regardless of what the client requests, shaping the response at the API serialization layer only saves network bandwidth, not database IO or server memory.

27.4. What the Pattern Looks Like

Below are examples of how clients can request shaped payloads using both the query parameter and the Prefer header.

1. Option A: Using the `include` Query Parameter (Full Resource)

The client requests an order and asks the server to embed the entire related product object.

Request

GET /orders/ord_992?include=product HTTP/1.1
Host: api.example.com

Response

The server returns the order with the full product object embedded.

HTTP/1.1 200 OK
Content-Type: application/json

{
  "id": "ord_992",
  "status": "shipped",
  "product": {
    "id": "prd_104",
    "name": "Precision Optical Mouse",
    "description": "Ergonomic 4000 DPI wireless mouse.",
    "price": 34.50,
    "weight_grams": 120
  }
}

2. Option B: Using the `Prefer` Header with Dot Notation (Field Selection)

To keep URLs clean or when requesting a highly specific subset of fields, the client uses the Prefer header combined with dot notation to select only the id and name of the product.

Request

GET /orders/ord_992 HTTP/1.1
Host: api.example.com
Prefer: include="product.id,product.name"

Response

The server must acknowledge the applied preference using the Preference-Applied header and return only the requested fields.

HTTP/1.1 200 OK
Content-Type: application/json
Preference-Applied: include="product.id,product.name"
Vary: Prefer

{
  "id": "ord_992",
  "status": "shipped",
  "product": {
    "id": "prd_104",
    "name": "Precision Optical Mouse"
  }
}

27.5. Anti-Patterns to Avoid

Unbounded nesting depth:
- Allowing clients to request include=product.manufacturer.address.country.region opens the API to malicious or accidental denial-of-service via complex database joins. Strictly limit the allowed depth (typically 1 or 2 levels).
Forgetting the Vary header:
- If you use the Prefer header to shape responses, failing to include Vary: Prefer in the response will cause intermediate caching proxies to serve sparse, field-selected responses to clients who later request the full object.
Silently failing on invalid fields:
- If a client requests include=product.nme (a typo), the server should return a 400 Bad Request rather than silently ignoring the instruction and returning an unshaped or unexpected response.
Defaulting to “include everything”:
- Never default to embedding all relational data if the include parameter is omitted. Omitted parameters should result in the lightest possible baseline representation.

27.6. OpenAPI Example

A complete OpenAPI 3.0.3 specification illustrating how to document both the query parameter and the Prefer header for response shaping.

openapi: 3.0.3
info:
  title: Orders API - Response Shaping Example
  version: 1.0.0
servers:
  - url: https://api.example.com

paths:
  /orders/{orderId}:
    get:
      summary: Retrieve an order with optional inclusion
      description: Returns an order. Clients can shape the response to include full related resources or specific fields using dot notation.
      tags: [Orders]
      parameters:
        - in: path
          name: orderId
          required: true
          schema:
            type: string
            example: "ord_992"
        - in: query
          name: include
          required: false
          schema:
            type: string
            example: "product.id,product.name"
          description: Comma-separated list of relationships or dot-notation fields to include (e.g., `product` or `product.id,product.name`).
        - in: header
          name: Prefer
          required: false
          schema:
            type: string
            example: 'include="product.id,product.name"'
          description: RFC 7240 preference header for response inclusion. Alternative to the query parameter.
      responses:
        '200':
          description: The shaped order response
          headers:
            Preference-Applied:
              schema:
                type: string
              description: Acknowledges which preferences were honored.
            Vary:
              schema:
                type: string
                example: Prefer
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/Order'

components:
  schemas:
    Order:
      type: object
      properties:
        id:
          type: string
          example: "ord_992"
        status:
          type: string
          example: "shipped"
        product:
          $ref: '#/components/schemas/Product'
          
    Product:
      type: object
      description: Fields returned depend on the client's inclusion instructions.
      properties:
        id:
          type: string
        name:
          type: string
        price:
          type: number
        description:
          type: string

27.7. Visualizing Response Shaping (Mermaid Diagram)

sequenceDiagram
    autonumber
    actor Client as Client App
    participant API as API Server
    participant DB as Database

    Note over Client,DB: Scenario: Client requests specific fields via Prefer header
    Client->>API: GET /orders/ord_992 (Prefer: include="product.id,product.name")
    
    API->>API: Parse requested shape
    Note right of API: Analyzes "product.id,product.name"<br/>Validates fields against schema
    
    API->>DB: SELECT o.*, p.id, p.name FROM orders o JOIN products p ON o.product_id = p.id WHERE o.id = 'ord_992'
    DB-->>API: Return highly targeted data row
    
    API-->>Client: 200 OK (Vary: Prefer, Preference-Applied: include=...)
    Note over Client: Client receives exact shape requested,<br/>saving parsing time and bandwidth.

Pattern: Response Shaping and Inclusion

27. Pattern: Response Shaping and Inclusion

27.1. Overview

27.2. When to Use Response Shaping

27.3. When NOT to Use Response Shaping

27.4. What the Pattern Looks Like

1. Option A: Using the include Query Parameter (Full Resource)

Request

Response

2. Option B: Using the Prefer Header with Dot Notation (Field Selection)

Request

Response

27.5. Anti-Patterns to Avoid

27.6. OpenAPI Example

27.7. Visualizing Response Shaping (Mermaid Diagram)

1. Option A: Using the `include` Query Parameter (Full Resource)

2. Option B: Using the `Prefer` Header with Dot Notation (Field Selection)