OpenRouter standardises image input API for vision-capable LLMs across providers

OpenRouter has unified the image-input API across every vision-capable model it supports. The request body is a single messages array where the user message’s content field becomes an array of typed parts: one text object and one image_url object. This pattern is identical regardless of provider, so engineers can swap models by changing only the model slug in the request. The change removes the need to learn multiple vendor-specific schemas and reduces integration code to a single conditional branch for image handling.

The guide distinguishes two ways to send image bytes: public HTTP(S) URLs or base64-encoded data URLs. Public URLs keep the request small and let the provider fetch the image, but they can fail due to access controls, regional blocks, or expired signed links. Base64 encoding adds latency and increases request size, yet it guarantees the bytes are present and avoids external dependencies. Engineers must weigh these trade-offs when choosing between the two formats for their use case.

Model selection becomes a runtime decision rather than a design-time one. Because the request shape is identical, engineers can test different models for strengths such as OCR accuracy, UI screenshot parsing, or chart reasoning without changing integration code. This flexibility allows teams to optimise for cost, latency, or accuracy after the feature is already in production, provided they monitor the tokenisation differences that affect billing and rate limits.

The guide also covers practical limits. Tokenisation of images varies by model, so the same image can consume different numbers of tokens and thus cost different amounts. Multimodal RAG pipelines are possible, but the guide cautions that image-heavy documents can quickly exhaust context windows. Engineers must therefore design their pipelines to chunk documents and images appropriately, balancing retrieval quality against token budget.