Conformance Scenarios

This document describes the process of running DCSA conformance scenarios for measuring the conformance of adopter implementations with DCSA standards.

Starting and stopping scenarios

Each sandbox page lists the scenarios that DCSA considers to be relevant when measuring the conformance of an adopter implementation, often split into meaningfully titled scenario groups.

Scenario status

The button at the left end of each scenario title indicates its running status:

The "play" icon (▶️) indicates that the scenario was either never run, or was reset after being partially or completely run. Click this icon to start a scenario.
The "stop" icon (⏹️) indicates that the scenario is currently running. Click this button to stop the scenario.
The "restart" icon (🔁) indicates that the scenario was completely run. Click this button to delete the previous conformance results and restart the scenario.

Within any sandbox, only one scenario may be running at any given time. When a scenario is running, the buttons of all other scenarios are greyed out.

Conformance result

The second icon at the left of each scenario title indicates the conformance result:

The green check mark (✅) indicates that your application behaved in a fully conformant way throughout the entire execution of the scenario.
The yellow triangle (⚠️) indicates that your application behaved in a partially conformant way during the execution of the scenario.
The red sign (🚫) indicates that your application behaved in a non-conformant way during the execution of the scenario.
The grey question mark (❔) indicates that no API traffic was recorded and analyzed upon which to evaluate the conformance of your application for this scenario.

Running scenario page

The scenario page displays the following information for an active scenario:

The title of the scenario, shown as a series of actions separated by dashes. Actions typically correspond to the use cases of a standard (for example UC3 in Booking - "Update booking request"). You need to be familiar with the standard to understand these actions before running conformance scenarios for it.
The list of remaining actions in the scenario, which is the subset of actions that have not yet been performed in the scenario; this list steadily decreases in length with each action you perform.
The current action in the scenario, displayed prominently above the action buttons.
Action buttons for refreshing the scenario status and to complete the current action.
The conformance status of all scenario actions, updated as each API exchange relevant to the current action is recorded and analyzed.

For an inactive scenario, the page states that the scenario is not currently running, meaning that no actions can be performed within the scenario page.

Supplying scenario parameters

The synthetic counterpart created within the DCSA conformance sandbox needs to exchange API calls with your application in order to measure its conformance. However, it is a generic DCSA reference implementation of the standard that does not have any information about your organization's data. Therefore, as a first step in many scenarios, you are prompted to provide some parameters for the scenario, so that the synthetic counterpart of your application knows how to make API calls that your application will accept.For example, when measuring the conformance of a Carrier within the Booking standard, before the Shipper can send a booking request, you need to tell it for what service contract reference, carrier service name and so on it can submit a booking request that your application can successfully process and accept.This virtually always happens in the first action of most scenarios, and there's a clear prompt that you need to take action (▶️) followed by an example JSON object that you need to customize and submit back. (We use JSON rather than a complex web form so you can easily reuse these parameters between scenarios.)Copy the example JSON into the empty text area, adjust each attribute value as needed by your application, then click "Submit".

Waiting for backend activity

Whenever one or more backend entities is waiting for something, the scenario page displays messages that indicate what needs to happen before you can proceed with the scenario execution.

Refreshing current action status

The frontend web UI does not maintain a continuous connection to the backend through which to receive push notifications, nor does it continuously poll the backend for updates when it is your application's turn to make an API call.Whenever your application has perform an API call that it was expected to perform, for example the Carrier sending a notification to the Shipper within the Booking standard, click the "Refresh status" button to have the UI updated with the latest information available on the backend.

Marking scenario actions completed

When all the exchanges corresponding to a scenario action were completed (for example in Booking, both the primary PUT exchange performed by the Shipper in UC3 and the notification sent back by the Carrier), or when you don't expect further exchange for that action (in the same Booking example, if the notification exchange has not happened but your system does not support sending notifications), you need to click "Action completed" in order to proceed to the next scenario action.In addition to supporting systems who don't make optional API calls, for example Booking Carriers who don't send notifications, this also allows you in many cases to retry an exchange that was reported as less than fully conformant without having to restart the scenario.Important note: To complete the execution of the scenario, you must click "Action completed" after the last action of the scenario; otherwise the scenario remains running and you cannot start other scenarios.

Actions performed by your application

In many scenario actions, the synthetic counterpart of your application that runs in the sandbox sends API requests that your applications handles automatically.In other scenario actions, your application is expected to perform a specific action that usually results in an API call made to its synthetic counterpart running in the sandbox.For example, when measuring the conformance of a Carrier application within the Booking standard, UC5 ("Confirm booking") is a use case that your application needs to perform on its own, typically prompted by human action.In such a case, the fact that it's your application's turn to perform an action is signaled by a "play" icon (▶️) and a clear prompt instructing you what action needs to be performed.Perform the requested action in your application, then click "Refresh status" to check the conformance status, then click "Action completed" when verifying that your application has performed the action in a conformant way.

Completed scenario

When all the actions of a scenario were completed and your application has performed all API requests and responses in a fully conformant way, all actions of the scenario and the scenario itself are marked as fully conformant.Note: Remember to click "Action completed" after the last action of a scenario.