Skip to content
Cloudflare Docs
DirectoryAPISDKsChangelog

Changelog

New updates and improvements at Cloudflare.

Subscribe to RSS View RSS feeds
Workers AI
hero image

  1. Workers AI now supports structured JSON outputs.

    Workers AI

    Workers AI now supports structured JSON outputs with JSON mode, which allows you to request a structured output response when interacting with AI models.

    This makes it much easier to retrieve structured data from your AI models, and avoids the (error prone!) need to parse large unstructured text responses to extract your data.

    JSON mode in Workers AI is compatible with the OpenAI SDK's structured outputs ↗ response_format API, which can be used directly in a Worker:

    JavaScript
    import { OpenAI } from "openai";
    

    // Define your JSON schema for a calendar event
    const CalendarEventSchema = {
      type: "object",
      properties: {
        name: { type: "string" },
        date: { type: "string" },
        participants: { type: "array", items: { type: "string" } },
      },
      required: ["name", "date", "participants"],
    };
    

    export default {
      async fetch(request, env) {
        const client = new OpenAI({
          apiKey: env.OPENAI_API_KEY,
          // Optional: use AI Gateway to bring logs, evals & caching to your AI requests
          // https://edgetunnel-b2h.pages.dev/ai-gateway/usage/providers/openai/
          // baseUrl: "https://gateway.ai.cloudflare.com/v1/{account_id}/{gateway_id}/openai"
        });
    

        const response = await client.chat.completions.create({
          model: "gpt-4o-2024-08-06",
          messages: [
            { role: "system", content: "Extract the event information." },
            {
              role: "user",
              content: "Alice and Bob are going to a science fair on Friday.",
            },
          ],
          // Use the `response_format` option to request a structured JSON output
          response_format: {
            // Set json_schema and provide ra schema, or json_object and parse it yourself
            type: "json_schema",
            schema: CalendarEventSchema, // provide a schema
          },
        });
    

        // This will be of type CalendarEventSchema
        const event = response.choices[0].message.parsed;
    

        return Response.json({
          calendar_event: event,
        });
      },
    };

    To learn more about JSON mode and structured outputs, visit the Workers AI documentation.

  1. Workers AI larger context windows

    Workers AI

    We've updated the Workers AI text generation models to include context windows and limits definitions and changed our APIs to estimate and validate the number of tokens in the input prompt, not the number of characters.

    This update allows developers to use larger context windows when interacting with Workers AI models, which can lead to better and more accurate results.

    Our catalog page provides more information about each model's supported context window.

  1. Workers AI updated pricing

    Workers AI

    We've updated the Workers AI pricing to include the latest models and how model usage maps to Neurons.

    • Each model's core input format(s) (tokens, audio seconds, images, etc) now include mappings to Neurons, making it easier to understand how your included Neuron volume is consumed and how you are charged at scale
    • Per-model pricing, instead of the previous bucket approach, allows us to be more flexible on how models are charged based on their size, performance and capabilities. As we optimize each model, we can then pass on savings for that model.
    • You will still only pay for what you consume: Workers AI inference is serverless, and not billed by the hour.

    Going forward, models will be launched with their associated Neuron costs, and we'll be updating the Workers AI dashboard and API to reflect consumption in both raw units and Neurons. Visit the Workers AI pricing page to learn more about Workers AI pricing.